Small docs update

git-svn-id: http://svn.code.sf.net/p/scst/svn/trunk@1607 d57e44dd-8a1f-0410-8b47-8ef2f437770f
This commit is contained in:
Vladislav Bolkhovitin
2010-04-12 17:38:08 +00:00
parent 70c3602671
commit 32082dc44a
3 changed files with 26 additions and 371 deletions

View File

@@ -23,9 +23,6 @@ obsolete. Use the sysfs interface facilities instead.
It is recommended to use TEST UNIT READY ("tur") command to check if
iSCSI-SCST target is alive in MPIO configurations.
IMPORTANT: In the procfs build all LUN information (access control)
========= MUST be configured BEFORE iscsi-scstd started!
Also see SCST README file how to tune for the best performance.
CAUTION: Working of target and initiator on the same host isn't fully
@@ -35,11 +32,6 @@ CAUTION: Working of target and initiator on the same host isn't fully
Sysfs interface
---------------
Starting from 2.0.0 iSCSI-SCST has sysfs interface. You can switch to it
by running "make disable_proc". To switch back to the procfs interface
you should run "make enable_proc". The procfs interface from version
2.0.0 is obsolete and will be removed in one of the next versions.
Root of SCST sysfs interface is /sys/kernel/scst_tgt. Root of iSCSI-SCST
is /sys/kernel/scst_tgt/targets/iscsi. It has the following entries:
@@ -531,7 +523,7 @@ performance problems. If you use it, it is strongly advised to upgrade.
instance as a shared storage with VMware, make sure all your VMs
connected to the target via *separate* sessions, i.e. each VM has own
connection to the target, not all VMs connected using a single
connection. You can check it using SCST proc or sysfs interface. If you
connection. You can check it using SCST sysfs interface. If you
miss it, you can greatly loose performance of parallel access to your
target from different VMs. This isn't related to the case if your VMs
are using the same shared storage, like with VMFS, for instance. In this

View File

@@ -232,11 +232,11 @@ your favorite kernel configuration Makefile target, e.g. "make xconfig":
Then sometimes get crazy itself. So, this option is disabled by
default.
- CONFIG_SCST_MEASURE_LATENCY - if defined, provides in /proc/scsi_tgt/latency
file average commands processing latency. You can clear already
measured results by writing 0 in this file. For the sysfs build you
can find those results in /sys/kernel/scst_tgt and below. Note, you need a
non-preemptible kernel to have correct results.
- CONFIG_SCST_MEASURE_LATENCY - if defined, provides in /sys/kernel/scst_tgt
and below statisctics about average commands processing latency. You
can clear already measured results by writing 0 in the corresponding
file. Note, you need a non-preemptible kernel to have correct
results.
HIGHMEM kernel configurations are fully supported, but not recommended
for performance reasons, except for scst_user, where they are not
@@ -271,249 +271,9 @@ Module scst supports the following parameters:
default it is approximately TotalMem/4.
SCST /proc interface
--------------------
For communications with user space programs SCST provides proc-based
interface in /proc/scsi_tgt directory. This interface is available in
the procfs build only. Starting from version 2.0.0 it is obsolete and
will be removed in one of the next versions. It contains the following
entries.
- "help" file, which provides online help for SCST commands
- "scsi_tgt" file, which on read provides information of serving by SCST
devices and their dev handlers. On write it supports the following
command:
* "assign H:C:I:L HANDLER_NAME" assigns dev handler "HANDLER_NAME"
on device with host:channel:id:lun. The recommended way to find out
H:C:I:L numbers is use of lsscsi utility.
- "sessions" file, which lists currently connected initiators (open sessions)
- "sgv" file provides some statistic about with which block sizes
commands from remote initiators come and how effective sgv_pool in
serving those allocations from the cache, i.e. without memory
allocations requests to the kernel. "Size" - is the commands data
size upper rounded to power of 2, "Hit" - how many there are
allocations from the cache, "Total" - total number of allocations.
- "threads" file, which allows to read and set number of SCST's threads
- "version" file, which shows version of SCST
- "trace_level" file, which allows to read and set trace (logging) level
for SCST. See /proc/scsi_tgt/help file for list of commands and
trace levels. If you want to enable logging options, which produce a
lot of events, like "debug", to not loose logged events you should
also:
* Increase in .config of your kernel CONFIG_LOG_BUF_SHIFT variable
to much bigger value, then recompile it. For example, value 25
will provide good protection from logging overflow even under
high volume of logging events, but to use it you will need to
modify the maximum allowed value for CONFIG_LOG_BUF_SHIFT in the
corresponding Kconfig file.
* Change in your /etc/syslog.conf or other config file of your favorite
logging program to store kernel logs in async manner. For example,
I added in my rsyslog.conf line "kern.info -/var/log/kernel"
and added "kern.none" in line for /var/log/messages, so I had:
"*.info;kern.none;mail.none;authpriv.none;cron.none /var/log/messages"
Each dev handler has own subdirectory. Most dev handler have only two
files in this subdirectory: "trace_level" and "type". The first one is
similar to main SCST "trace_level" file, the latter one shows SCSI type
number of this handler as well as some text description.
For example, "echo "assign 1:0:1:0 dev_disk" >/proc/scsi_tgt/scsi_tgt"
will assign device handler "dev_disk" to real device sitting on host 1,
channel 0, ID 1, LUN 0.
Access and devices visibility management (LUN masking) - /proc interface
------------------------------------------------------------------------
Access and devices visibility management allows for an initiator or
group of initiators to see different devices with different LUNs
with necessary access permissions.
SCST supports two modes of access control:
1. Target-oriented. In this mode you define for each target devices and
their LUNs, which are accessible to all initiators, connected to that
target. This is a regular access control mode, which people usually mean
thinking about access control in general. For instance, in IET this is
the only supported mode. In this mode you should create a security group
with name "Default_TARGET_NAME", where "TARGET_NAME" is name of the
target, like "Default_iqn.2007-05.com.example:storage.disk1.sys1.xyz"
for target "iqn.2007-05.com.example:storage.disk1.sys1.xyz". Then you
should add to it all LUNs, available from that target.
2. Initiator-oriented. In this mode you define which devices and their
LUNs are accessible for each initiator. In this mode you should create
for each set of one or more initiators, which should access to the same
set of devices with the same LUNs, a separate security group, then add
to it available devices and names of allowed initiator(s).
Both modes can be used simultaneously. In this case initiator-oriented
mode has higher priority, than target-oriented.
When a target driver registers itself in SCST core, it tells SCST core
its name. Then, when there is a new connection from a remote initiator,
the target driver registers this connection in SCST core and tells it
the name of the remote initiator. Then SCST core finds the corresponding
devices for it using the following algorithm:
1. It searches through all defined groups trying to find group
containing the initiator name. If it succeeds, the found group is used.
2. Otherwise, it searches through all groups trying to find group with
name "Default_TARGET_NAME". If it succeeds, the found group is used.
3. Otherwise, the group with name "Default" is used. This group is
always defined, but empty by default.
Names of both target and initiator you can clarify in the kernel log. In
it SCST reports to which group each session is assigned.
In /proc/scsi_tgt each group represented as "groups/GROUP_NAME/"
subdirectory. In it there are files "devices" and "names". File
"devices" lists devices and their LUNs in the group, file "names" lists
names of initiators, which allowed to access devices in this group.
To configure access and devices visibility management SCST provides the
following files and directories under /proc/scsi_tgt:
- "add_group GROUP_NAME" to /proc/scsi_tgt/scsi_tgt adds group "GROUP_NAME"
- "del_group GROUP_NAME" to /proc/scsi_tgt/scsi_tgt deletes group "GROUP_NAME"
- "rename_group OLD_NAME NEW_NAME" to /proc/scsi_tgt/scsi_tgt renames
group "OLD_NAME" to "NEW_NAME".
- "add H:C:I:L lun [READ_ONLY]" to /proc/scsi_tgt/groups/GROUP_NAME/devices adds
device with host:channel:id:lun with LUN "lun" in group "GROUP_NAME". Optionally,
the device could be marked as read only. The recommended way to find out
H:C:I:L numbers is use of lsscsi utility.
- "replace H:C:I:L lun [READ_ONLY]" to /proc/scsi_tgt/groups/GROUP_NAME/devices
replaces by device with host:channel:id:lun existing with LUN "lun"
device in group "GROUP_NAME" with generation of INQUIRY DATA HAS
CHANGED Unit Attention. If the old device doesn't exist, this
command acts as the "add" command. Optionally, the device could be
marked as read only. The recommended way to find out H:C:I:L numbers
is use of lsscsi utility.
- "del H:C:I:L" to /proc/scsi_tgt/groups/GROUP_NAME/devices deletes device with
host:channel:id:lun from group "GROUP_NAME". The recommended way to find out
H:C:I:L numbers is use of lsscsi utility.
- "add V_NAME lun [READ_ONLY]" to /proc/scsi_tgt/groups/GROUP_NAME/devices adds
device with virtual name "V_NAME" with LUN "lun" in group "GROUP_NAME".
Optionally, the device could be marked as read only.
- "replace V_NAME lun [READ_ONLY]" to /proc/scsi_tgt/groups/GROUP_NAME/devices
replaces by device with virtual name "V_NAME" existing with LUN
"lun" device in group "GROUP_NAME" with generation of INQUIRY DATA
HAS CHANGED Unit Attention. If the old device doesn't exist, this
command acts as the "add" command. Optionally, the device could
be marked as read only.
- "del V_NAME" to /proc/scsi_tgt/groups/GROUP_NAME/devices deletes device with
virtual name "V_NAME" from group "GROUP_NAME"
- "clear" to /proc/scsi_tgt/groups/GROUP_NAME/devices clears the list of devices
for group "GROUP_NAME"
- "add NAME" to /proc/scsi_tgt/groups/GROUP_NAME/names adds name "NAME" to group
"GROUP_NAME". For NAME you can use simple DOS-type patterns, containing
'*' and '?' symbols. '*' means match all any symbols, '?' means
match only any single symbol. For instance, "blah.xxx" will match
"bl?h.*".
- "del NAME" to /proc/scsi_tgt/groups/GROUP_NAME/names deletes name "NAME" from group
"GROUP_NAME"
- "move NAME NEW_GROUP_NAME" to /proc/scsi_tgt/groups/OLD_GROUP_NAME/names
moves name "NAME" from group "OLD_GROUP_NAME" to group "NEW_GROUP_NAME".
- "clear" to /proc/scsi_tgt/groups/GROUP_NAME/names clears the list of names
for group "GROUP_NAME"
Examples:
- "echo "add 1:0:1:0 0" >/proc/scsi_tgt/groups/Default/devices" will
add real device sitting on host 1, channel 0, ID 1, LUN 0 to "Default"
group with LUN 0.
- "echo "add disk1 1" >/proc/scsi_tgt/groups/Default/devices" will
add virtual VDISK device with name "disk1" to "Default" group
with LUN 1.
- "echo "21:*:e0:?b:83:*'" >/proc/scsi_tgt/groups/LAB1/names" will
add a pattern, which matches WWNs of Fibre Channel ports from LAB1.
Consider you need to have an iSCSI target with name
"iqn.2007-05.com.example:storage.disk1.sys1.xyz" (you defined it in
iscsi-scst.conf), which should export virtual device "dev1" with LUN 0
and virtual device "dev2" with LUN 1, but initiator with name
"iqn.2007-05.com.example:storage.disk1.spec_ini.xyz" should see only
virtual device "dev2" with LUN 0. To achieve that you should do the
following commands:
# echo "add_group Default_iqn.2007-05.com.example:storage.disk1.sys1.xyz" >/proc/scsi_tgt/scsi_tgt
# echo "add dev1 0" >/proc/scsi_tgt/groups/Default_iqn.2007-05.com.example:storage.disk1.sys1.xyz/devices
# echo "add dev2 1" >/proc/scsi_tgt/groups/Default_iqn.2007-05.com.example:storage.disk1.sys1.xyz/devices
# echo "add_group spec_ini" >/proc/scsi_tgt/scsi_tgt
# echo "add iqn.2007-05.com.example:storage.disk1.spec_ini.xyz" >/proc/scsi_tgt/groups/spec_ini/names
# echo "add dev2 0" >/proc/scsi_tgt/groups/spec_ini/devices
It is highly recommended to use scstadmin utility instead of described
in this section low level interface.
IMPORTANT
=========
There must be LUN 0 in each security group, i.e. LUs numeration must not
start from, e.g., 1. Otherwise you will see no devices on remote
initiators and SCST core will write into the kernel log message: "tgt_dev
for LUN 0 not found, command to unexisting LU?"
IMPORTANT
=========
All the access control must be fully configured BEFORE load of the
corresponding target driver! When you load a target driver or enable
target mode in it, as for qla2x00t driver, it will immediately start
accepting new connections, hence creating new sessions, and those new
sessions will be assigned to security groups according to the
*currently* configured access control settings. For instance, to
"Default" group, instead of "HOST004" as you may need, because "HOST004"
doesn't exist yet. So, one must configure all the security groups before
new connections from the initiators are created, i.e. before target
drivers loaded.
Access controls can be altered after the target driver loaded as long as
the target session doesn't yet exist. And even in the case of the
session already existing, changes are still possible, but won't be
reflected on the initiator side.
So, the safest choice is to configure all the access control before any
target driver load and then only add new devices to new groups for new
initiators or add new devices to old groups, but not altering existing
LUNs in them.
SCST sysfs interface
--------------------
Starting from 2.0.0 SCST has sysfs interface. You can switch to it by
running "make disable_proc". To switch back to the procfs interface you
should run "make enable_proc".
Root of SCST sysfs interface is /sys/kernel/scst_tgt. It has the
following entries:
@@ -879,103 +639,6 @@ created, i.e. before the target enabled.
VDISK device handler
--------------------
/proc interface
~~~~~~~~~~~~~~~
This interface starting from version 2.0.0 is obsolete and will be
removed in one of the next versions.
After loading VDISK device handler creates in /proc/scsi_tgt/
subdirectories "vdisk" and "vcdrom". They have the following layout:
- "trace_level" and "type" files as described above
- "help" file, which provides online help for VDISK commands
- "vdisk"/"vcdrom" files, which on read provides information of
currently open device files. On write it supports the following
command:
* "open NAME [PATH] [BLOCK_SIZE] [FLAGS]" - opens file "PATH" as
device "NAME" with block size "BLOCK_SIZE" bytes with flags
"FLAGS". "PATH" could be empty only for VDISK CDROM. "BLOCK_SIZE"
and "FLAGS" are valid only for disk VDISK. The block size must be
power of 2 and >= 512 bytes. Default is 512. Possible flags:
- WRITE_THROUGH - write back caching disabled. Note, this option
has sense only if you also *manually* disable write-back cache
in *all* your backstorage devices and make sure it's actually
disabled, since many devices are known to lie about this mode to
get better benchmark results.
- READ_ONLY - read only
- O_DIRECT - both read and write caching disabled. This mode
isn't currently fully implemented, you should use user space
fileio_tgt program in O_DIRECT mode instead (see below).
- NULLIO - in this mode no real IO will be done, but success will be
returned. Intended to be used for performance measurements at the same
way as "*_perf" handlers.
- NV_CACHE - enables "non-volatile cache" mode. In this mode it is
assumed that the target has a GOOD UPS with ability to cleanly
shutdown target in case of power failure and it is
software/hardware bugs free, i.e. all data from the target's
cache are guaranteed sooner or later to go to the media. Hence
all data synchronization with media operations, like
SYNCHRONIZE_CACHE, are ignored in order to bring more
performance. Also in this mode target reports to initiators that
the corresponding device has write-through cache to disable all
write-back cache workarounds used by initiators. Use with
extreme caution, since in this mode after a crash of the target
journaled file systems don't guarantee the consistency after
journal recovery, therefore manual fsck MUST be ran. Note, that
since usually the journal barrier protection (see "IMPORTANT"
note below) turned off, enabling NV_CACHE could change nothing
from data protection point of view, since no data
synchronization with media operations will go from the
initiator. This option overrides WRITE_THROUGH.
- BLOCKIO - enables block mode, which will perform direct block
IO with a block device, bypassing page-cache for all operations.
This mode works ideally with high-end storage HBAs and for
applications that either do not need caching between application
and disk or need the large block throughput. See also below.
- REMOVABLE - with this flag set the device is reported to remote
initiators as removable.
* "close NAME" - closes device "NAME".
* "resync_size NAME" - refreshes size of device "NAME". Intended to be
used after device resize.
* "change NAME [PATH]" - changes a virtual CD in the VDISK CDROM.
* "set_t10_dev_id NAME T10_DEVICE_ID" - sets T10 vendor specific
identifier on Device Identification VPD page (0x83) of device
"NAME" in INQUIRY data. By default VDISK handler always generates
T10_DEVICE_ID for every new created device at creation time.
This parameter allows to overwrite generated by VDISK value of
T10_DEVICE_ID.
By default, if neither BLOCKIO, nor NULLIO option is supplied, FILEIO
mode is used.
For example, "echo "open disk1 /vdisks/disk1" >/proc/scsi_tgt/vdisk/vdisk"
will open file /vdisks/disk1 as virtual FILEIO disk with name "disk1".
/sys interface
~~~~~~~~~~~~~~
Starting from 2.0.0 VDISK device handler has sysfs interface. You can
switch to it by running "make disable_proc". To switch back to the
procfs interface you should run "make enable_proc". The procfs interface
starting from version 2.0.0 is obsolete and will be removed in one of
the next versions.
VDISK has 4 built-in dev handlers: vdisk_fileio, vdisk_blockio,
vdisk_nullio and vcdrom. Roots of their sysfs interface are
/sys/kernel/scst_tgt/handlers/handler_name, e.g. for vdisk_fileio:
@@ -1158,17 +821,17 @@ change virtual CD media in the virtual CDROM device. For example:
- echo "" >/sys/kernel/scst_tgt/devices/cdrom/filename - will remove
"media" from the virtual CDROM cdrom.
Additionally to the sysfs/procfs interface VDISK handler has module
parameter "num_threads", which specifies count of I/O threads for each
VDISK's device. If you have a workload, which tends to produce rather
random accesses (e.g. DB-like), you should increase this count to a
bigger value, like 32. If you have a rather sequential workload, you
should decrease it to a lower value, like number of CPUs on the target
or even 1. Due to some limitations of Linux I/O subsystem, increasing
number of I/O threads too much leads to sequential performance drop,
especially with deadline scheduler, so decreasing it can improve
sequential performance. The default provides a good compromise between
random and sequential accesses.
Additionally to the sysfs interface VDISK handler has module parameter
"num_threads", which specifies count of I/O threads for each VDISK's
device. If you have a workload, which tends to produce rather random
accesses (e.g. DB-like), you should increase this count to a bigger
value, like 32. If you have a rather sequential workload, you should
decrease it to a lower value, like number of CPUs on the target or even
1. Due to some limitations of Linux I/O subsystem, increasing number of
I/O threads too much leads to sequential performance drop, especially
with deadline scheduler, so decreasing it can improve sequential
performance. The default provides a good compromise between random and
sequential accesses.
You shouldn't be afraid to have too many VDISK I/O threads if you have
many VDISK devices. Kernel threads consume very little amount of
@@ -1469,14 +1132,14 @@ following:
instance as a shared storage with VMware, make sure all your VMs
connected to the target via *separate* sessions. For instance, for iSCSI
it means that each VM has own connection to the target, not all VMs
connected using a single connection. You can check it using SCST proc or
sysfs interface. For other transports you should use available
facilities, like NPIV for Fibre Channel, to make separate sessions for
each VM. If you miss it, you can greatly loose performance of parallel
access to your target from different VMs. This isn't related to the case
if your VMs are using the same shared storage, like with VMFS, for
instance. In this case all your VM hosts will be connected to the target
via separate sessions, which is enough.
connected using a single connection. You can check it using SCST sysfs
interface. For other transports you should use available facilities,
like NPIV for Fibre Channel, to make separate sessions for each VM. If
you miss it, you can greatly loose performance of parallel access to
your target from different VMs. This isn't related to the case if your
VMs are using the same shared storage, like with VMFS, for instance. In
this case all your VM hosts will be connected to the target via separate
sessions, which is enough.
6. For other target and initiator software parts:

View File

@@ -361,7 +361,7 @@ EXPORT_SYMBOL(scst_unregister_target_template);
/**
* scst_register_target() - register target
*
* Rgisters a target for template vtt and returns new target structure on
* Registers a target for template vtt and returns new target structure on
* success or NULL otherwise.
*/
struct scst_tgt *scst_register(struct scst_tgt_template *vtt,