diff --git a/www/comparison.html b/www/comparison.html index 7080a3bb5..cdac7bcb5 100644 --- a/www/comparison.html +++ b/www/comparison.html @@ -45,13 +45,13 @@

Features comparison between Linux SCSI targets

-

As on October 2009. Partially updated in August 2010.

+

As on September 2010

- + @@ -72,9 +74,11 @@ - + +transfer values (Wide (parallel) SCSI, SAS) - + - - + - + + + + +resized through AENs or Unit Attentions (initiators can instantly see +any target reconfiguration in a PnP-like manner) @@ -138,7 +151,7 @@ any target reconfiguration in a PnP-like fashion) +(Windows 2003 clustering) +13 - - + + + - - - - - - + + + + + + + + + + + + +hard lockup of the target) - + + + + + + @@ -233,9 +267,8 @@ allocate too much memory for buffers and taking it into an OOM state +9 - @@ -248,15 +281,25 @@ allocate too much memory for buffers and taking it into an OOM state + + + + + + + + + @@ -270,12 +313,21 @@ allocate too much memory for buffers and taking it into an OOM state + + + + + + +in modes, other than pass-through < - @@ -316,20 +369,46 @@ devices (VTL) < +session (MS/C) + + + + + + + + + + + + + + + + + + - + + + + + - - - + + + + + + + +
-SCST +SCST 2.0 STGT IET LIO @@ -61,7 +61,9 @@ General
Upstream kernel - Kernel part - - Upstream kernel - Interface with + in-kernel target + drivers - -
Generic Target Engine + + iSCSI only + - Kernel only
Stability + + +Stability Final testing + before release (1.0.1.x + - stable) + 3 - + ? (no released packages) + Probably (no released packages)
Performance 4 ***** @@ -92,40 +96,49 @@ backend drivers +
Support for transports without expecting -transfer values (parallel SCSI, SAS) + - - - + - - -
Interface with user space SysFS/ProcFS Custom - ConfigFS/IOCTL/ProcFS Interface with user space SysFS (or obsolete + ProcFS) Custom - ConfigFS/IOCTL/ProcFS
Major features +Features
Target drivers in kernel space + + - +
Target drivers in user space + + - - Target drivers in user space Via scst_local (e.g. + using STGT + pass-through) + - Via tcm_loop (e.g. + using STGT pass-through)
Backstorage handlers in kernel space + - - +
Backstorage handlers in user space + + - Through STGT Backstorage handlers in user space + + - Via STGT + (under development)
Advanced devices access control 10 + - - +
Automatic sessions reassignment (changes in the +access control immediatelly seen by initiators) + - - -
Support for Asynchronous Event Notifications (AEN) + - - -
Notifications for devices added/removed or -resized through AENs or Unit Attentions (Initiators can instantly see -any target reconfiguration in a PnP-like fashion) + - - - + - - -
Bidirectional Commands + + - - + -
RESERVE/RELEASE - (Windows 2003 clustering) + + + + + + + +
Safe RESERVE/RELEASE implementation according to @@ -147,7 +160,9 @@ SCSI requirements 12 Safe
Safe implementation of Task Management commands -13 Safe Not safe Not safe Not safe Safe Not safe Not safe LUN RESET - safe. + Other TM commands not + implemented.
Support for SCSI task attributes, including @@ -160,30 +175,49 @@ ORDERED commands + + -,
data 14
Persistent Reservations
- (Windows 2008 clustering / RHEL5 I/O fencing)		 + +
(limited +		Persistent (SCSI-3) Reservations +(Windows 2008 clustering / RHEL5 I/O fencing) + +
(limited functionality implemented)		 - +
Durable, i.e. transactional, save of Persistent +Through Power Loss Persistent Reservation data Durable Not supported - Not durable
ALUA - - - +
Failover Clustering + ? + +
Allocated buffers cache + - - -
Failover Clustering + + + +
Different threading models to choose the best +performing + - - -
I/O context grouping between I/O threads (big +performance win with CFQ) + - + -
Per-initiator I/O context grouping (big +performance and fairness win if several initiators access the same +virtual or backend device on the target) + - - -
Protection against commands with wrong transfer size or transfer direction (may lead to crash or -hard lockup) + - - - + - - -
Protection against DoS'ing target by making it -allocate too much memory for buffers and taking it into an OOM state +Protection against crashing target by making it +to allocate too much memory for buffers and go into OOM state + - - -
Caching of allocated buffers + - - -
Latencies measurement facility + - - -
SCSI MIBs - - - +- 15
Local access to emulated backstorage devices -9 + - - + scst_local - - tcm_loop
Supported backstorage
User space FILEIO + + - -
O_DIRECT FILEIO fileio_tgt + - -
Async FILEIO - + - -
Native RAMDISK - - - +
SCSI pass-through 16 + Single initiator only, not enforced, limited - for tapes + functionality for tapes + 17 - Single initiator only, not enforced, - limited for tapes + limited functionality for tapes 17
BLOCKIO BLOCKIO, pass- through
Cache safe11 +FILEIO Safe Safe only RDWR + backend Safe Under development (just added)
Cache safe11 +BLOCKIO Safe - Not safe Under development (just added)
4k sectors support in pass-through mode + - - ?
4k, 2k, 1k and 512 byte sectors emulation -in modes, other than pass-through + - - + + - - Partial
Virtual CD devices emulation from ISO files @@ -302,8 +354,9 @@ devices (VTL) - Experimental - Kernel only
Interface with user space SysFS/IOCTL/ProcFS/ - Netlink - IOCTL/ProcFS/ +Interface with user space SysFS (or obsolete + ProcFS)/ + IOCTL/Netlink - IOCTL/ProcFS/ Netlink ConfigFS/IOCTL/ProcFS
- Experimental -
Multiple connections per - session (MS/C) - - - + - - - +
Max ErrorRecoveryLevel 0 0 0 2
Support for limiting number of initiators +allowed to connect to a target + - + -
Per-portal targets visibility control + - + -
Per-initiators targets visibility control + + + -
Support for AHS + + - -
Support for iSCSI redirects + + + -
Bidirectional Commands + + - -
Extended CDB (size >16 bytes) + + - -
Support for AENs (initiators can instantly see any +target reconfiguration in a PnP-like manner) + - - -
Support for Asynchronous Event Notifications -(AEN) + - - - Support for iSNS + + + -
Safe implementation of Task Management commands +13 Safe Not safe Not safe ABORT TASK - not safe, + LUN RESET - safe, + other TM commands not + implemented.
Safe implementation of connections and sessions @@ -337,30 +416,40 @@ reinstatement 20 Safe N
Safe restart 21 - Safe ? Not safe ? Safe ? Not safe before + v1.4.18. After - + probably safe. ?
Support for -MaxOutstandingR2T >1 (write performance increase on high latency -links) + Partial - +
iSCSI MIBs - - - +- 15
Local access target +
Bidirectional support + - - -
Support for AENs (initiators can instantly see any +target reconfiguration in a PnP-like manner) + - - -

REMARKS:

-

1. STGT has SCSI target engine and memory management in user space with small hooks in the kernel to interact with in-kernel target drivers. - As a direct consequence, fully user space STGT target (e.g. iSCSI) can run without any kernel modules needed.

+

1. STGT has all SCSI target processing and memory management in the user space. In the kernel it has only a small library + to help in-kernel target drivers to interact with it. As a direct consequence, neither iSCSI, nor iSER as well as + any other user space STGT target need any STGT kernel modules loaded to run.

2. All iSCSI management implemented in user space and actual data transfers in kernel space without user space involved.

-

3. ISER target driver has long known (since Feb 2008) data corruption problem, which localization hasn't been determined yet and might be - in the STGT core. See here and - here.

+

3. ISER target driver has long known (since Feb 2008) data corruption problem, which localization hasn't been + determined yet and might be in the STGT core. See + here, + here and + here.

4. The result "in average" is listed. One target can be better somewhere, another one somewhere else. Although manual tuning of target and system parameters tends the restore the difference listed in the comparison. You can find example measurements here, @@ -371,8 +460,9 @@ links) + Partial - +

6. There is no performance data for LIO, which allow to make a direct comparison with other targets. - The conclusion was made by source code study only. LIO should have performance on the IET level or less, - because of more processing overhead. It might be much less for small block sizes.

+ The conclusion was made based on source code study. LIO should have performance on the IET level or less, + because of more processing overhead. It might be much less for small block sizes. Also the single I/O thread per-device + approach LIO has is not optimal in many cases.

7. In SCST data are always passed in zero-copy manner between target and backend drivers without need for any additional kernel patches, except in case, when local access (scst_local) used with user space backend.

@@ -391,6 +481,10 @@ links) + Partial - + +

11. "Cache safe" means that cache synchronization commands (SYNCHRONIZE_CACHE and FUA attribute) from initiators perform + what they expected to perform, i.e. push all the requested blocks from all caches, including devices' caches, + to non-volatile media.

+

12. SCSI requires that if an initiator clears reservation held by another initiator, the reservation holder must be notified about the reservation clearance. Otherwise, several initiators can at the same time change supposed to be protected by the reservation data, which can corrupt them. This is what was called @@ -434,8 +528,7 @@ links) + Partial - + 19. Doesn't need any kernel patch, except in the case, when used with user space backend.

20. Connections and sessions reinstatement is, basically, a kind of Task Management command, because it implies commands aborting. - For instance, open-iscsi uses it as a less intrusive substistute for target reset in eh_target_reset_handler() callback. So, similarly - to the safe task management above, a safe implementation of connections and sessions reinstatement + So, similarly to the safe task management above, a safe implementation of connections and sessions reinstatement must not accept SCSI commands from new connection/session until all the SCSI commands in being reinstated connection/session get into a state, where they can't affect new commands.

diff --git a/www/contributing.html b/www/contributing.html index 992fe250a..a33c9d1e9 100644 --- a/www/contributing.html +++ b/www/contributing.html @@ -36,9 +36,7 @@ -

Contributing in SCST, you investing not only in making Linux the best storage OS, but, if your company - has a product or service, based on SCST, you also investing in the secured future of your product - or service and, hence, in the the secured future of your company.

-

Possible SCST extensions and improvements

Zero-copy FILEIO for READ-direction commands

@@ -282,18 +276,26 @@

Solve SG IO count limitation issue in pass-through mode

-

In the pass-through mode (i.e. using the pass-through device handlers - scst_disk, scst_tape, etc) SCSI commands, coming from remote initiators, +

In the pass-through mode (i.e. using the pass-through device handlers like + scst_tape, etc) SCSI commands, coming from remote initiators, are passed to local SCSI hardware on target as is, without any modifications. As any other hardware, the local SCSI hardware can not handle commands with amount of data and/or segments count in - scatter-gather array bigger some values. If you have this issue you will see - symptoms like small transfers work well, but large ones stall and + scatter-gather array bigger some values. For some commands SCST can + split them on subcommands and, hence, workaround this problem, but it isn't + always possible. For instance, for tapes splitting write commands may mean + corrupting the tape data.

+ +

If you have this issue you will see + symptoms like small transfers work well, but large transfers stall and messages like: "Unable to complete command due to SG IO count limitation" are printed in the kernel logs.

-

In sgv_big_order_alloc.diff you - can find a possible way to solve this issue.

+

+ +

The only complete way to fix this problem is to allocate data buffers with number + of entries inside the SG IO count limitation. In sgv_big_order_alloc.diff + you can find a possible way to solve this issue.

There are also 2 more patches you can look at:

diff --git a/www/index.html b/www/index.html index 2ab08e9f4..2090bb140 100644 --- a/www/index.html +++ b/www/index.html @@ -126,7 +126,8 @@ page-cache for all operations. This mode works well with high-end storage HBAs and for applications that either do not need caching between application and disk or need the large block throughput.
  • User space mode using scst_user device handler, which allows to implement in the - user space virtual SCSI devices in the SCST environment.
    • + user space high performance virtual SCSI devices. Comparing with fully in-kernel dev handlers + this mode has very low overhead (few %%).
  • Performance testing device handlers as well as NULLIO mode to provide a way for direct performance measurements without overhead of actual data transfers from/to underlying SCSI devices. diff --git a/www/scstvsstgt.html b/www/scstvsstgt.html index b320c2e8a..143fdb1ff 100644 --- a/www/scstvsstgt.html +++ b/www/scstvsstgt.html @@ -70,8 +70,8 @@ -

    Also you shouldn't be deceived by the fact that some (small) part of STGT was accepted into the kernel.
    - It doesn't mean that STGT has the "kernel quality". In fact, STGT as a whole similarly to any other +

    Also don't be deceived by the fact that some (small) part of STGT was accepted into the kernel.
    + It doesn't mean that STGT has the "kernel quality". STGT as a whole similarly to any other out-of-kernel-tree project lives on its own, hence has its own quality level, which isn't necessary better, than the quality level of SCST.