From d5322f10adc032e531b20cf7ce51db8421581b4d Mon Sep 17 00:00:00 2001 From: Vladislav Bolkhovitin Date: Mon, 2 Feb 2009 15:49:39 +0000 Subject: [PATCH] The patch below implements the following changes for the iSCSI-SCST HOWTO document: - Added RHEL / CentOS to the list of officially supported kernels. - Mentioned that regenerating the initial RAM disk is an explicit step on many distro's. - Replaced shell instructions for generating /etc/scst.conf by a sequence of scstadmin commands. - Added note about insserv. Also it adds small HOWTO how to recompile RHEL/CentOS kernel in README_RHEL. Also it implements some docs cleanups. Signed-off-by: Bart Van Assche git-svn-id: http://svn.code.sf.net/p/scst/svn/trunk@658 d57e44dd-8a1f-0410-8b47-8ef2f437770f --- iscsi-scst/README | 18 +-- iscsi-scst/doc/iscsi-scst-howto.txt | 172 ++++++++++++++++++---------- qla2x00t/qla2x00-target/README | 18 +-- scst/README | 30 +++-- scst/README_RHEL | 65 +++++++++++ 5 files changed, 213 insertions(+), 90 deletions(-) create mode 100644 scst/README_RHEL diff --git a/iscsi-scst/README b/iscsi-scst/README index 119f0fc4c..f262848bc 100644 --- a/iscsi-scst/README +++ b/iscsi-scst/README @@ -39,15 +39,15 @@ Installation out of Linux kernel tree Basically as in README-IET, where file names are changed as specified above. -Only vanilla kernels from kernel.org are supported, but it should work -on vendors' kernels, if you manage to successfully compile on them. The -main problem with vendor's kernels is that they often contain patches, -which will appear only in the next version of the vanilla kernel, -therefore it's quite hard to track such changes. Thus, if during -compilation for some vendor kernel your compiler complains about -redefinition of some symbol, you should either switch to vanilla kernel, -or change as necessary the corresponding to that symbol "#if -LINUX_VERSION_CODE" statement. +Only vanilla kernels from kernel.org and RHEL/CentOS 5.2 kernels are +supported, but it should work on other (vendors') kernels, if you manage +to successfully compile on them. The main problem with vendor's kernels +is that they often contain patches, which will appear only in the next +version of the vanilla kernel, therefore it's quite hard to track such +changes. Thus, if during compilation for some vendor's kernel your +compiler complains about redefinition of some symbol, you should either +switch to vanilla kernel, or add or change as necessary the +corresponding to that symbol "#if LINUX_VERSION_CODE" statement. If during compilation you see message like "*** No rule to make target `xxx.h', needed by `yyy.o'. Stop.", then your autogenerated diff --git a/iscsi-scst/doc/iscsi-scst-howto.txt b/iscsi-scst/doc/iscsi-scst-howto.txt index ba2803a84..886c04d97 100644 --- a/iscsi-scst/doc/iscsi-scst-howto.txt +++ b/iscsi-scst/doc/iscsi-scst-howto.txt @@ -1,103 +1,137 @@ -How to set up iSCSI-SCST using scstadmin. +Installing and using iSCSI-SCST with scstadmin +---------------------------------------------- 1. Download, build and install iSCSI-SCST. -Download the source code: +Start with downloading the iSCSI-SCST source code. You can either download the +released version from the following URL: -Released version from http://sourceforge.net/project/showfiles.php?group_id=110471 + http://sourceforge.net/project/showfiles.php?group_id=110471 -or the latest development version by command: +or you can download the latest development version by running the following +command in a shell: -svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk + svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk scst -You need to patch the kernel and rebuild it. Only vanilla kernels from -kernel.org are officially supported, but in most cases your favorite -kernel will work fine as well. Select the patch according to your kernel -version (2.6.18.1 in the example below): +While iSCSI-SCST works fine with an unpatched Linux kernel with the same +or better speed as other open source iSCSI targets, including IET, if +you want even better performance you have to patch and rebuild the +kernel. Select the patch according to your kernel version (2.6.18.1 in +the example below): -cd /usr/src/kernels/linux-2.6.18.1 -patch -p1 < /home/erezz/work/scst/iscsi-scst/kernel/patches/put_page_callback-2.6.18.1.patch -patch -p1 < /home/erezz/work/scst/scst/kernel/scst_exec_req_fifo-2.6.18.patch -make clean + cd /usr/src/kernels/linux-2.6.18.1 + patch -p1 < $HOME/scst/iscsi-scst/kernel/patches/put_page_callback-2.6.18.1.patch + patch -p1 < $HOME/scst/scst/kernel/scst_exec_req_fifo-2.6.18.patch + make clean -Then, build the kernel (i.e. make, make modules_install, make install) -and restart the machine. +Next, build and install the kernel: -In order to build and install SCST and iSCSI, run + make && make modules + make modules_install && make install -make scst scst_install iscsi iscsi_install +For some Linux distributions (a.o. Ubuntu and openSUSE) the above steps do not +rebuild the initial RAM disk. Make sure you regenerate the RAM disk before +rebooting. Please look up the documentation of the update-initrd, +update-initramfs or the mkinitrd command supplied with your distribution. -ISCSI-SCST includes the following components: +Now reboot the system. + +Once the system is again up and running the next step is to build and install +SCST, iSCSI-SCST and scstadmin. You can do this by running the following +command from the SCST directory: + + make scst scst_install iscsi iscsi_install scstadm scstadm_install + +iSCSI-SCST includes the following components: * /etc/init.d/iscsi-scst - service script * iscsi-scstd - daemon - * iscsi-scst-adm - admin tool (not built in 1.0.0) * man pages - * Configuration files (located under iscsi-scst/etc/, need to be copied to /etc if you want to use them): + * Configuration files (located under iscsi-scst/etc/, need to be copied to + /etc if you want to use them): * initiators.allow - used for assigning specific initiators to targets * initiators.deny - used for assigning specific initiators to targets * iscsi-scstd.conf - list of targets and their properties -2. Set up /etc/iscsi/ininitiatorname.iscsi. The most convenient way to -set up this file is to install the open-iscsi package provided by your -Linux distro first and then to run the bash commands shown below. -Verify the contents of the generated file. +2. Set up /etc/iscsi/ininitiatorname.iscsi + +The most convenient way to set up this file is to install the open-iscsi +package provided by your Linux distro first and then to run the shell commands +shown below. Verify the contents of the generated file. + /etc/init.d/open-iscsi stop { echo "InitiatorName=$(/usr/sbin/iscsi-iname)"; echo "InitiatorAlias=$(hostname)"; } >/etc/iscsi/initiatorname.iscsi /etc/init.d/open-iscsi start + cat /etc/iscsi/initiatorname.iscsi -3. Set up /etc/iscsi-scstd.conf, e.g. by running the following shell commands: - cat </etc/iscsi-scstd.conf - Target $(sed -n 's/InitiatorName=//p' /etc/iscsi/initiatorname.iscsi):storage - EOF +3. Set up /etc/iscsi-scstd.conf -4. Set up /etc/scst.conf. The bash statements shown below set up a -configuration file for two vdisks. Replace the device names in the -array 'disk' by those that apply to your system: - { - echo "[HANDLER vdisk]" - echo "#DEVICE ,,," - disk=(/dev/md733 /dev/md734) - for ((i=0;i<${#disk[*]};i++)) - do - echo "DEVICE vdisk$i,${disk[$i]},NV_CACHE,512" - done - echo "[ASSIGNMENT Default]" - echo "#DEVICE ," - for ((i=0;i<${#disk[*]};i++)) - do - echo "DEVICE vdisk$i,$i" - done - } > /etc/scst.conf +You can do this by e.g. running the following shell commands: -Note, that you must have LUN 0. That's a SCSI requirement (documented in SCST's README). + echo "Target $(sed -n 's/InitiatorName=//p' /etc/iscsi/initiatorname.iscsi):storage" >/etc/iscsi-scstd.conf + cat /etc/iscsi-scstd.conf -5. Install scstadmin: - cd scstadmin-x.y.z - make install +4. Set up /etc/scst.conf -6. Edit /etc/init.d/scst: remove the modules you do not need from the -SCST_MODULES variable, e.g. qla2x00tgt. +The scst.conf configuration file is loaded at system boot time by the shell +script /etc/init.d/scst. All devices known by SCST are defined in this file. +The shell statements shown below set up a configuration file for two vdisks. +Replace the device names by names that apply to your system: + + scstadmin -ClearConfig /etc/scst.conf + scstadmin -adddev disk01 -path /dev/ram0 -handler vdisk -options NV_CACHE + scstadmin -adddev disk02 -path /dev/ram1 -handler vdisk -options NV_CACHE + scstadmin -WriteConfig /etc/scst.conf + cat /etc/scst.conf + +Note that for each SCSI device LUN 0 must exist. This is required by the SCSI +protocol and is also documented in SCST's README file. + +5. Edit /etc/init.d/scst + +Remove the modules you do not need from the SCST_MODULES variable, +e.g. qla2x00tgt. + +6. Create soft links in /etc/init.d + +This will make SCST to start automatically upon system boot. Use either +chkconfig or update-rc.d, depending on the Linux distribution you are using. +An example for Debian systems such as Ubuntu: -6. Create soft links in /etc/init.d such that SCST starts up -automatically. Use either chkconfig or update-rc.d, depending on the -Linux distribution you are using. An example for Debian systems: update-rc.d scst defaults update-rc.d iscsi-scst defaults -7. Restart SCST and iSCSI-SCST such that the new settings become effective: +On openSUSE you can let SCST start automatically through the following commands: + + insserv scst + insserv iscsi-scst + +7. Restart SCST and iSCSI-SCST + +If you don't want to reboot now, you can make the above settings effective via +the following commands: + /etc/init.d/iscsi-scst stop /etc/init.d/scst stop /etc/init.d/scst start /etc/init.d/iscsi-scst start -How to install and use iSCSI-SCST without using scstadmin +Installing and using iSCSI-SCST without using scstadmin +------------------------------------------------------- -First, start scst_disk: modprobe scst_disk +First repeat steps 1, 2 and 3 from the previous section but leave out +"scstadm scstadm_install" from the make command in step 1. -/etc/init.d/iscsi-scst can be used to start/stop/restart/check status. +Next load the scst_disk kernel module as follows: -Selecting devices to be used by SCST. + modprobe scst_disk + +The shell script /etc/init.d/iscsi-scst can now be used to start/stop/restart or +check the status of iSCSI-SCST. + + +Selecting devices to be used by SCST +------------------------------------ You can see the list of available devices: @@ -137,7 +171,9 @@ Device (host:ch:id:lun or name) Device handler 5:0:1:3 dev_disk 5:0:1:4 dev_disk -LUN masking. + +LUN masking +----------- SCST defines security groups. For each group, you can add LUNs. @@ -153,7 +189,9 @@ Now, the LUN "5:0:0:8" was added to the "Default" group as LUN #0. Note, that you must have LUN 0. That's a SCSI requirement (documented in SCST's README). + Defining LUN masking +-------------------- In order to associate specific LUNs with specific targets, do the following: @@ -168,7 +206,9 @@ echo "add 4:0:0:8 0" > /proc/scsi_tgt/groups/Default_iqn.2007-05.com.example\:st This will assign the LUN "4:0:0:8" to the target iqn.2007-05.com.example:storage.iscsi-scst-1. Again, you must have LUN 0 for each group. + Deleting a LUN from a group +--------------------------- Run the following command: @@ -176,7 +216,9 @@ echo "del 4:0:0:8 0" > /proc/scsi_tgt/groups/Default_iqn.2007-05.com.example\:st This will remove the LUN "4:0:0:8" from the target iqn.2007-05.com.example:storage.iscsi-scst-1. -Deleting a group. + +Deleting a group +---------------- Run the following command: @@ -185,13 +227,17 @@ echo "del_group Default_iqn.2007-05.com.example:storage.iscsi-scst-1" >/proc/scs For more information about LUN masking, refer to SCST README, section "Access and devices visibility management (LUN masking)". -Creating targets using iscsi-scstd.conf. + +Creating targets using iscsi-scstd.conf +--------------------------------------- The easiest way to create targets is to define them in /etc/iscsi-scstd.conf. An example can be found in etc/iscsi-scstd.conf. You need to have this file under /etc/ before starting iSCSI-SCST. + Assigning targets to specific initiators +---------------------------------------- In order to assign targets to specific initiators, you need to have /etc/initiators.allow and /etc/initiators.deny. You can find diff --git a/qla2x00t/qla2x00-target/README b/qla2x00t/qla2x00-target/README index fbfe74519..6bd1ad14d 100644 --- a/qla2x00t/qla2x00-target/README +++ b/qla2x00t/qla2x00-target/README @@ -30,15 +30,15 @@ features. Installation ------------ -Only vanilla kernels from kernel.org are supported, but it should work -on vendors' kernels, if you manage to successfully compile on them. The -main problem with vendor's kernels is that they often contain patches, -which will appear only in the next version of the vanilla kernel, -therefore it's quite hard to track such changes. Thus, if during -compilation for some vendor kernel your compiler complains about -redefinition of some symbol, you should either switch to vanilla kernel, -or change as necessary the corresponding to that symbol "#if -LINUX_VERSION_CODE" statement. +Only vanilla kernels from kernel.org and RHEL/CentOS 5.2 kernels are +supported, but SCST should work on other (vendors') kernels, if you +manage to successfully compile on them. The main problem with vendors' +kernels is that they often contain patches, which will appear only in +the next version of the vanilla kernel, therefore it's quite hard to +track such changes. Thus, if during compilation for some vendor kernel +your compiler complains about redefinition of some symbol, you should +either switch to vanilla kernel, or add or change as necessary the +corresponding to that symbol "#if LINUX_VERSION_CODE" statement. At first, make sure that the link "/lib/modules/`you_kernel_version`/build" points to the source code for your currently running kernel. diff --git a/scst/README b/scst/README index fe203f5ff..fb87f6ba6 100644 --- a/scst/README +++ b/scst/README @@ -40,18 +40,19 @@ In addition, SCST supports advanced per-initiator access and devices visibility management, so different initiators could see different set of devices with different access permissions. See below for details. + Installation ------------ -Only vanilla kernels from kernel.org are supported, but it should work -on vendors' kernels, if you manage to successfully compile on them. The -main problem with vendor's kernels is that they often contain patches, -which will appear only in the next version of the vanilla kernel, -therefore it's quite hard to track such changes. Thus, if during -compilation for some vendor kernel your compiler complains about -redefinition of some symbol, you should either switch to vanilla kernel, -or change as necessary the corresponding to that symbol "#if -LINUX_VERSION_CODE" statement. +Only vanilla kernels from kernel.org and RHEL/CentOS 5.2 kernels are +supported, but SCST should work on other (vendors') kernels, if you +manage to successfully compile on them. The main problem with vendors' +kernels is that they often contain patches, which will appear only in +the next version of the vanilla kernel, therefore it's quite hard to +track such changes. Thus, if during compilation for some vendor kernel +your compiler complains about redefinition of some symbol, you should +either switch to vanilla kernel, or add or change as necessary the +corresponding to that symbol "#if LINUX_VERSION_CODE" statement. At first, make sure that the link "/lib/modules/`you_kernel_version`/build" points to the source code for your currently running kernel. @@ -131,6 +132,7 @@ IMPORTANT: In the current version simultaneous access to local SCSI devices To uninstall, type 'make scst_uninstall'. + Device handlers --------------- @@ -175,6 +177,7 @@ NOTE: Since "perf" device handlers on READ operations don't touch the was allocated, without even being zeroed. Thus, "perf" device handlers impose some security risk, so use them with caution. + Compilation options ------------------- @@ -282,6 +285,7 @@ For changing VMSPLIT option (CONFIG_VMSPLIT to be precise) you should in memory you have. If it is less than 800MB, you may not touch this option at all. + Module parameters ----------------- @@ -294,6 +298,7 @@ Module scst supports the following parameters: consumed by the SCST commands for data buffers at any given time. By default it is approximately TotalMem/4. + SCST "/proc" commands --------------------- @@ -348,6 +353,7 @@ For example, "echo "assign 1:0:1:0 dev_disk" >/proc/scsi_tgt/scsi_tgt" will assign device handler "dev_disk" to real device sitting on host 1, channel 0, ID 1, LUN 0. + Access and devices visibility management (LUN masking) ------------------------------------------------------ @@ -491,6 +497,7 @@ target driver load and then only add new devices to new groups for new initiators or add new devices to old groups, but not altering existing LUNs in them. + VDISK device handler -------------------- @@ -641,6 +648,7 @@ IMPORTANT: Some disk and partition table management utilities don't support size device, the block size stops matter, any program will work with files on such file system. + BLOCKIO VDISK mode ------------------ @@ -672,6 +680,7 @@ IMPORTANT: Since data in BLOCKIO and FILEIO modes are not consistent between ========= them, if you try to use a device in both those modes simultaneously, you will almost instantly corrupt your data on that device. + Pass-through mode ----------------- @@ -704,6 +713,7 @@ Another way to solve this issue is to build SG entries with more than 1 page each. See the following patch as an example: http://scst.sf.net/sgv_big_order_alloc.diff + User space mode using scst_user dev handler ------------------------------------------- @@ -716,6 +726,7 @@ loads it could be significantly faster, than the regular FILEIO access. All the words about BLOCKIO from above apply to O_DIRECT as well. See fileio_tgt's README file for more details. + Performance ----------- @@ -940,6 +951,7 @@ or inititor-target link, you can increase SCST_MAX_TGT_DEV_COMMANDS in scst_priv.h to 64. Usually initiators don't try to push more commands on the target. + Credits ------- diff --git a/scst/README_RHEL b/scst/README_RHEL new file mode 100644 index 000000000..9d81bc9f2 --- /dev/null +++ b/scst/README_RHEL @@ -0,0 +1,65 @@ +Recompiling a RHEL 5 / CentOS 5 kernel +-------------------------------------- + + - Look up the version of the installed kernel, e.g. through the +following command: + uname -r + + - Choose a RHEL / CentOS mirror + + - Download the source RPM matching the installed kernel version, e.g. +kernel-2.6.18-92.1.22.el5.src.rpm + + - Now generate a kernel source tree from this source RPM by running +the following commands as root: + + useradd mockbuild + rpm -i kernel-2.6.18-92.1.22.el5.src.rpm + +The above commands will create a kernel spec file +/usr/src/redhat/SPECS/kernel-2.6.spec and will create the files for +building the kernel source tree in /usr/src/redhat/SOURCES/. + + - The kernel source tree can now be built by running the commands +below. These commands will not only create a kernel source tree but will +also install a kernel .config file: + + yum install unifdef + rm -rf /usr/src/redhat/BUILD + mkdir -p /usr/src/redhat/BUILD + chown mockbuild /usr/src/redhat/BUILD + sudo -u mockbuild rpmbuild -bp /usr/src/redhat/SPECS/kernel-2.6.spec + + - Next, apply the SCST patches to this kernel source tree: + + cd "/usr/src/redhat/BUILD/kernel-2.6.18/linux-2.6.18.$(uname -m)" + patch -p1 <${SCST_SOURCE_DIR}/scst/kernel/rhel/scst_exec_req_fifo-rhel5.patch + patch -p1 <${SCST_SOURCE_DIR}/iscsi-scst/kernel/patches/rhel/put_page_callback-rhel5.patch + + - Modify EXTRAVERSION in Makefile from "-prep" into "-scst". The text +assigned to the EXTRAVERSION variable will be appended to the kernel +version 2.6.18 and will appear a.o. in the GRUB boot menu. Choosing a +name that refers to the SCST project will help to recognize the +customized kernel. + + - If you are familiar with kernel configuration, you can decrease +kernel compilation time significantly by minimizing the kernel +configuration as follows: + + yum install qt-devel + lsmod + make xconfig & + + - Now compile and install the kernel: + + make && make modules + mkdir -p /lib/modules/2.6.18-scst && make modules_install && make install + + - Edit the bootloader menu such that the newly compiled kernel is +loaded by default: + + vi /boot/grub/menu.lst + +- Boot into the new kernel: + + reboot