diff --git a/iscsi-scst/README_in-tree b/iscsi-scst/README_in-tree index 18abe72ad..d51594888 100644 --- a/iscsi-scst/README_in-tree +++ b/iscsi-scst/README_in-tree @@ -23,9 +23,6 @@ obsolete. Use the sysfs interface facilities instead. It is recommended to use TEST UNIT READY ("tur") command to check if iSCSI-SCST target is alive in MPIO configurations. -IMPORTANT: In the procfs build all LUN information (access control) -========= MUST be configured BEFORE iscsi-scstd started! - Also see SCST README file how to tune for the best performance. CAUTION: Working of target and initiator on the same host isn't fully @@ -35,11 +32,6 @@ CAUTION: Working of target and initiator on the same host isn't fully Sysfs interface --------------- -Starting from 2.0.0 iSCSI-SCST has sysfs interface. You can switch to it -by running "make disable_proc". To switch back to the procfs interface -you should run "make enable_proc". The procfs interface from version -2.0.0 is obsolete and will be removed in one of the next versions. - Root of SCST sysfs interface is /sys/kernel/scst_tgt. Root of iSCSI-SCST is /sys/kernel/scst_tgt/targets/iscsi. It has the following entries: @@ -531,7 +523,7 @@ performance problems. If you use it, it is strongly advised to upgrade. instance as a shared storage with VMware, make sure all your VMs connected to the target via *separate* sessions, i.e. each VM has own connection to the target, not all VMs connected using a single -connection. You can check it using SCST proc or sysfs interface. If you +connection. You can check it using SCST sysfs interface. If you miss it, you can greatly loose performance of parallel access to your target from different VMs. This isn't related to the case if your VMs are using the same shared storage, like with VMFS, for instance. In this diff --git a/scst/README_in-tree b/scst/README_in-tree index 3d9b95c21..ba955b5aa 100644 --- a/scst/README_in-tree +++ b/scst/README_in-tree @@ -232,11 +232,11 @@ your favorite kernel configuration Makefile target, e.g. "make xconfig": Then sometimes get crazy itself. So, this option is disabled by default. - - CONFIG_SCST_MEASURE_LATENCY - if defined, provides in /proc/scsi_tgt/latency - file average commands processing latency. You can clear already - measured results by writing 0 in this file. For the sysfs build you - can find those results in /sys/kernel/scst_tgt and below. Note, you need a - non-preemptible kernel to have correct results. + - CONFIG_SCST_MEASURE_LATENCY - if defined, provides in /sys/kernel/scst_tgt + and below statisctics about average commands processing latency. You + can clear already measured results by writing 0 in the corresponding + file. Note, you need a non-preemptible kernel to have correct + results. HIGHMEM kernel configurations are fully supported, but not recommended for performance reasons, except for scst_user, where they are not @@ -271,249 +271,9 @@ Module scst supports the following parameters: default it is approximately TotalMem/4. -SCST /proc interface --------------------- - -For communications with user space programs SCST provides proc-based -interface in /proc/scsi_tgt directory. This interface is available in -the procfs build only. Starting from version 2.0.0 it is obsolete and -will be removed in one of the next versions. It contains the following -entries. - - - "help" file, which provides online help for SCST commands - - - "scsi_tgt" file, which on read provides information of serving by SCST - devices and their dev handlers. On write it supports the following - command: - - * "assign H:C:I:L HANDLER_NAME" assigns dev handler "HANDLER_NAME" - on device with host:channel:id:lun. The recommended way to find out - H:C:I:L numbers is use of lsscsi utility. - - - "sessions" file, which lists currently connected initiators (open sessions) - - - "sgv" file provides some statistic about with which block sizes - commands from remote initiators come and how effective sgv_pool in - serving those allocations from the cache, i.e. without memory - allocations requests to the kernel. "Size" - is the commands data - size upper rounded to power of 2, "Hit" - how many there are - allocations from the cache, "Total" - total number of allocations. - - - "threads" file, which allows to read and set number of SCST's threads - - - "version" file, which shows version of SCST - - - "trace_level" file, which allows to read and set trace (logging) level - for SCST. See /proc/scsi_tgt/help file for list of commands and - trace levels. If you want to enable logging options, which produce a - lot of events, like "debug", to not loose logged events you should - also: - - * Increase in .config of your kernel CONFIG_LOG_BUF_SHIFT variable - to much bigger value, then recompile it. For example, value 25 - will provide good protection from logging overflow even under - high volume of logging events, but to use it you will need to - modify the maximum allowed value for CONFIG_LOG_BUF_SHIFT in the - corresponding Kconfig file. - - * Change in your /etc/syslog.conf or other config file of your favorite - logging program to store kernel logs in async manner. For example, - I added in my rsyslog.conf line "kern.info -/var/log/kernel" - and added "kern.none" in line for /var/log/messages, so I had: - "*.info;kern.none;mail.none;authpriv.none;cron.none /var/log/messages" - -Each dev handler has own subdirectory. Most dev handler have only two -files in this subdirectory: "trace_level" and "type". The first one is -similar to main SCST "trace_level" file, the latter one shows SCSI type -number of this handler as well as some text description. - -For example, "echo "assign 1:0:1:0 dev_disk" >/proc/scsi_tgt/scsi_tgt" -will assign device handler "dev_disk" to real device sitting on host 1, -channel 0, ID 1, LUN 0. - - -Access and devices visibility management (LUN masking) - /proc interface ------------------------------------------------------------------------- - -Access and devices visibility management allows for an initiator or -group of initiators to see different devices with different LUNs -with necessary access permissions. - -SCST supports two modes of access control: - -1. Target-oriented. In this mode you define for each target devices and -their LUNs, which are accessible to all initiators, connected to that -target. This is a regular access control mode, which people usually mean -thinking about access control in general. For instance, in IET this is -the only supported mode. In this mode you should create a security group -with name "Default_TARGET_NAME", where "TARGET_NAME" is name of the -target, like "Default_iqn.2007-05.com.example:storage.disk1.sys1.xyz" -for target "iqn.2007-05.com.example:storage.disk1.sys1.xyz". Then you -should add to it all LUNs, available from that target. - -2. Initiator-oriented. In this mode you define which devices and their -LUNs are accessible for each initiator. In this mode you should create -for each set of one or more initiators, which should access to the same -set of devices with the same LUNs, a separate security group, then add -to it available devices and names of allowed initiator(s). - -Both modes can be used simultaneously. In this case initiator-oriented -mode has higher priority, than target-oriented. - -When a target driver registers itself in SCST core, it tells SCST core -its name. Then, when there is a new connection from a remote initiator, -the target driver registers this connection in SCST core and tells it -the name of the remote initiator. Then SCST core finds the corresponding -devices for it using the following algorithm: - -1. It searches through all defined groups trying to find group -containing the initiator name. If it succeeds, the found group is used. - -2. Otherwise, it searches through all groups trying to find group with -name "Default_TARGET_NAME". If it succeeds, the found group is used. - -3. Otherwise, the group with name "Default" is used. This group is -always defined, but empty by default. - -Names of both target and initiator you can clarify in the kernel log. In -it SCST reports to which group each session is assigned. - -In /proc/scsi_tgt each group represented as "groups/GROUP_NAME/" -subdirectory. In it there are files "devices" and "names". File -"devices" lists devices and their LUNs in the group, file "names" lists -names of initiators, which allowed to access devices in this group. - -To configure access and devices visibility management SCST provides the -following files and directories under /proc/scsi_tgt: - - - "add_group GROUP_NAME" to /proc/scsi_tgt/scsi_tgt adds group "GROUP_NAME" - - - "del_group GROUP_NAME" to /proc/scsi_tgt/scsi_tgt deletes group "GROUP_NAME" - - - "rename_group OLD_NAME NEW_NAME" to /proc/scsi_tgt/scsi_tgt renames - group "OLD_NAME" to "NEW_NAME". - - - "add H:C:I:L lun [READ_ONLY]" to /proc/scsi_tgt/groups/GROUP_NAME/devices adds - device with host:channel:id:lun with LUN "lun" in group "GROUP_NAME". Optionally, - the device could be marked as read only. The recommended way to find out - H:C:I:L numbers is use of lsscsi utility. - - - "replace H:C:I:L lun [READ_ONLY]" to /proc/scsi_tgt/groups/GROUP_NAME/devices - replaces by device with host:channel:id:lun existing with LUN "lun" - device in group "GROUP_NAME" with generation of INQUIRY DATA HAS - CHANGED Unit Attention. If the old device doesn't exist, this - command acts as the "add" command. Optionally, the device could be - marked as read only. The recommended way to find out H:C:I:L numbers - is use of lsscsi utility. - - - "del H:C:I:L" to /proc/scsi_tgt/groups/GROUP_NAME/devices deletes device with - host:channel:id:lun from group "GROUP_NAME". The recommended way to find out - H:C:I:L numbers is use of lsscsi utility. - - - "add V_NAME lun [READ_ONLY]" to /proc/scsi_tgt/groups/GROUP_NAME/devices adds - device with virtual name "V_NAME" with LUN "lun" in group "GROUP_NAME". - Optionally, the device could be marked as read only. - - - "replace V_NAME lun [READ_ONLY]" to /proc/scsi_tgt/groups/GROUP_NAME/devices - replaces by device with virtual name "V_NAME" existing with LUN - "lun" device in group "GROUP_NAME" with generation of INQUIRY DATA - HAS CHANGED Unit Attention. If the old device doesn't exist, this - command acts as the "add" command. Optionally, the device could - be marked as read only. - - - "del V_NAME" to /proc/scsi_tgt/groups/GROUP_NAME/devices deletes device with - virtual name "V_NAME" from group "GROUP_NAME" - - - "clear" to /proc/scsi_tgt/groups/GROUP_NAME/devices clears the list of devices - for group "GROUP_NAME" - - - "add NAME" to /proc/scsi_tgt/groups/GROUP_NAME/names adds name "NAME" to group - "GROUP_NAME". For NAME you can use simple DOS-type patterns, containing - '*' and '?' symbols. '*' means match all any symbols, '?' means - match only any single symbol. For instance, "blah.xxx" will match - "bl?h.*". - - - "del NAME" to /proc/scsi_tgt/groups/GROUP_NAME/names deletes name "NAME" from group - "GROUP_NAME" - - - "move NAME NEW_GROUP_NAME" to /proc/scsi_tgt/groups/OLD_GROUP_NAME/names - moves name "NAME" from group "OLD_GROUP_NAME" to group "NEW_GROUP_NAME". - - - "clear" to /proc/scsi_tgt/groups/GROUP_NAME/names clears the list of names - for group "GROUP_NAME" - -Examples: - - - "echo "add 1:0:1:0 0" >/proc/scsi_tgt/groups/Default/devices" will - add real device sitting on host 1, channel 0, ID 1, LUN 0 to "Default" - group with LUN 0. - - - "echo "add disk1 1" >/proc/scsi_tgt/groups/Default/devices" will - add virtual VDISK device with name "disk1" to "Default" group - with LUN 1. - -- "echo "21:*:e0:?b:83:*'" >/proc/scsi_tgt/groups/LAB1/names" will - add a pattern, which matches WWNs of Fibre Channel ports from LAB1. - -Consider you need to have an iSCSI target with name -"iqn.2007-05.com.example:storage.disk1.sys1.xyz" (you defined it in -iscsi-scst.conf), which should export virtual device "dev1" with LUN 0 -and virtual device "dev2" with LUN 1, but initiator with name -"iqn.2007-05.com.example:storage.disk1.spec_ini.xyz" should see only -virtual device "dev2" with LUN 0. To achieve that you should do the -following commands: - -# echo "add_group Default_iqn.2007-05.com.example:storage.disk1.sys1.xyz" >/proc/scsi_tgt/scsi_tgt -# echo "add dev1 0" >/proc/scsi_tgt/groups/Default_iqn.2007-05.com.example:storage.disk1.sys1.xyz/devices -# echo "add dev2 1" >/proc/scsi_tgt/groups/Default_iqn.2007-05.com.example:storage.disk1.sys1.xyz/devices - -# echo "add_group spec_ini" >/proc/scsi_tgt/scsi_tgt -# echo "add iqn.2007-05.com.example:storage.disk1.spec_ini.xyz" >/proc/scsi_tgt/groups/spec_ini/names -# echo "add dev2 0" >/proc/scsi_tgt/groups/spec_ini/devices - -It is highly recommended to use scstadmin utility instead of described -in this section low level interface. - -IMPORTANT -========= - -There must be LUN 0 in each security group, i.e. LUs numeration must not -start from, e.g., 1. Otherwise you will see no devices on remote -initiators and SCST core will write into the kernel log message: "tgt_dev -for LUN 0 not found, command to unexisting LU?" - -IMPORTANT -========= - -All the access control must be fully configured BEFORE load of the -corresponding target driver! When you load a target driver or enable -target mode in it, as for qla2x00t driver, it will immediately start -accepting new connections, hence creating new sessions, and those new -sessions will be assigned to security groups according to the -*currently* configured access control settings. For instance, to -"Default" group, instead of "HOST004" as you may need, because "HOST004" -doesn't exist yet. So, one must configure all the security groups before -new connections from the initiators are created, i.e. before target -drivers loaded. - -Access controls can be altered after the target driver loaded as long as -the target session doesn't yet exist. And even in the case of the -session already existing, changes are still possible, but won't be -reflected on the initiator side. - -So, the safest choice is to configure all the access control before any -target driver load and then only add new devices to new groups for new -initiators or add new devices to old groups, but not altering existing -LUNs in them. - - SCST sysfs interface -------------------- -Starting from 2.0.0 SCST has sysfs interface. You can switch to it by -running "make disable_proc". To switch back to the procfs interface you -should run "make enable_proc". - Root of SCST sysfs interface is /sys/kernel/scst_tgt. It has the following entries: @@ -879,103 +639,6 @@ created, i.e. before the target enabled. VDISK device handler -------------------- -/proc interface -~~~~~~~~~~~~~~~ - -This interface starting from version 2.0.0 is obsolete and will be -removed in one of the next versions. - -After loading VDISK device handler creates in /proc/scsi_tgt/ -subdirectories "vdisk" and "vcdrom". They have the following layout: - - - "trace_level" and "type" files as described above - - - "help" file, which provides online help for VDISK commands - - - "vdisk"/"vcdrom" files, which on read provides information of - currently open device files. On write it supports the following - command: - - * "open NAME [PATH] [BLOCK_SIZE] [FLAGS]" - opens file "PATH" as - device "NAME" with block size "BLOCK_SIZE" bytes with flags - "FLAGS". "PATH" could be empty only for VDISK CDROM. "BLOCK_SIZE" - and "FLAGS" are valid only for disk VDISK. The block size must be - power of 2 and >= 512 bytes. Default is 512. Possible flags: - - - WRITE_THROUGH - write back caching disabled. Note, this option - has sense only if you also *manually* disable write-back cache - in *all* your backstorage devices and make sure it's actually - disabled, since many devices are known to lie about this mode to - get better benchmark results. - - - READ_ONLY - read only - - - O_DIRECT - both read and write caching disabled. This mode - isn't currently fully implemented, you should use user space - fileio_tgt program in O_DIRECT mode instead (see below). - - - NULLIO - in this mode no real IO will be done, but success will be - returned. Intended to be used for performance measurements at the same - way as "*_perf" handlers. - - - NV_CACHE - enables "non-volatile cache" mode. In this mode it is - assumed that the target has a GOOD UPS with ability to cleanly - shutdown target in case of power failure and it is - software/hardware bugs free, i.e. all data from the target's - cache are guaranteed sooner or later to go to the media. Hence - all data synchronization with media operations, like - SYNCHRONIZE_CACHE, are ignored in order to bring more - performance. Also in this mode target reports to initiators that - the corresponding device has write-through cache to disable all - write-back cache workarounds used by initiators. Use with - extreme caution, since in this mode after a crash of the target - journaled file systems don't guarantee the consistency after - journal recovery, therefore manual fsck MUST be ran. Note, that - since usually the journal barrier protection (see "IMPORTANT" - note below) turned off, enabling NV_CACHE could change nothing - from data protection point of view, since no data - synchronization with media operations will go from the - initiator. This option overrides WRITE_THROUGH. - - - BLOCKIO - enables block mode, which will perform direct block - IO with a block device, bypassing page-cache for all operations. - This mode works ideally with high-end storage HBAs and for - applications that either do not need caching between application - and disk or need the large block throughput. See also below. - - - REMOVABLE - with this flag set the device is reported to remote - initiators as removable. - - * "close NAME" - closes device "NAME". - - * "resync_size NAME" - refreshes size of device "NAME". Intended to be - used after device resize. - - * "change NAME [PATH]" - changes a virtual CD in the VDISK CDROM. - - * "set_t10_dev_id NAME T10_DEVICE_ID" - sets T10 vendor specific - identifier on Device Identification VPD page (0x83) of device - "NAME" in INQUIRY data. By default VDISK handler always generates - T10_DEVICE_ID for every new created device at creation time. - This parameter allows to overwrite generated by VDISK value of - T10_DEVICE_ID. - -By default, if neither BLOCKIO, nor NULLIO option is supplied, FILEIO -mode is used. - -For example, "echo "open disk1 /vdisks/disk1" >/proc/scsi_tgt/vdisk/vdisk" -will open file /vdisks/disk1 as virtual FILEIO disk with name "disk1". - - -/sys interface -~~~~~~~~~~~~~~ - -Starting from 2.0.0 VDISK device handler has sysfs interface. You can -switch to it by running "make disable_proc". To switch back to the -procfs interface you should run "make enable_proc". The procfs interface -starting from version 2.0.0 is obsolete and will be removed in one of -the next versions. - VDISK has 4 built-in dev handlers: vdisk_fileio, vdisk_blockio, vdisk_nullio and vcdrom. Roots of their sysfs interface are /sys/kernel/scst_tgt/handlers/handler_name, e.g. for vdisk_fileio: @@ -1158,17 +821,17 @@ change virtual CD media in the virtual CDROM device. For example: - echo "" >/sys/kernel/scst_tgt/devices/cdrom/filename - will remove "media" from the virtual CDROM cdrom. -Additionally to the sysfs/procfs interface VDISK handler has module -parameter "num_threads", which specifies count of I/O threads for each -VDISK's device. If you have a workload, which tends to produce rather -random accesses (e.g. DB-like), you should increase this count to a -bigger value, like 32. If you have a rather sequential workload, you -should decrease it to a lower value, like number of CPUs on the target -or even 1. Due to some limitations of Linux I/O subsystem, increasing -number of I/O threads too much leads to sequential performance drop, -especially with deadline scheduler, so decreasing it can improve -sequential performance. The default provides a good compromise between -random and sequential accesses. +Additionally to the sysfs interface VDISK handler has module parameter +"num_threads", which specifies count of I/O threads for each VDISK's +device. If you have a workload, which tends to produce rather random +accesses (e.g. DB-like), you should increase this count to a bigger +value, like 32. If you have a rather sequential workload, you should +decrease it to a lower value, like number of CPUs on the target or even +1. Due to some limitations of Linux I/O subsystem, increasing number of +I/O threads too much leads to sequential performance drop, especially +with deadline scheduler, so decreasing it can improve sequential +performance. The default provides a good compromise between random and +sequential accesses. You shouldn't be afraid to have too many VDISK I/O threads if you have many VDISK devices. Kernel threads consume very little amount of @@ -1469,14 +1132,14 @@ following: instance as a shared storage with VMware, make sure all your VMs connected to the target via *separate* sessions. For instance, for iSCSI it means that each VM has own connection to the target, not all VMs -connected using a single connection. You can check it using SCST proc or -sysfs interface. For other transports you should use available -facilities, like NPIV for Fibre Channel, to make separate sessions for -each VM. If you miss it, you can greatly loose performance of parallel -access to your target from different VMs. This isn't related to the case -if your VMs are using the same shared storage, like with VMFS, for -instance. In this case all your VM hosts will be connected to the target -via separate sessions, which is enough. +connected using a single connection. You can check it using SCST sysfs +interface. For other transports you should use available facilities, +like NPIV for Fibre Channel, to make separate sessions for each VM. If +you miss it, you can greatly loose performance of parallel access to +your target from different VMs. This isn't related to the case if your +VMs are using the same shared storage, like with VMFS, for instance. In +this case all your VM hosts will be connected to the target via separate +sessions, which is enough. 6. For other target and initiator software parts: diff --git a/scst/src/scst_main.c b/scst/src/scst_main.c index 17051a395..4b59c4e0e 100644 --- a/scst/src/scst_main.c +++ b/scst/src/scst_main.c @@ -361,7 +361,7 @@ EXPORT_SYMBOL(scst_unregister_target_template); /** * scst_register_target() - register target * - * Rgisters a target for template vtt and returns new target structure on + * Registers a target for template vtt and returns new target structure on * success or NULL otherwise. */ struct scst_tgt *scst_register(struct scst_tgt_template *vtt,