scst/scst_local

SCST Local ...
Richard Sharpe, 30-Nov-2008

This is the SCST Local driver. Its function is to allow you to access devices
that are exported via SCST directly on the same Linux system that they are
exported from.

No assumptions are made in the code about the device types on the target, so
any device handlers that you load in SCST should be visible, including tapes
and so forth.

You can freely use any sg, sd, st, etc. devices imported from target,
except the following: you can't mount file systems or put swap on them
for all dev handlers, except BLOCKIO and pass-through, because it can
lead to recursive memory allocation deadlock. This is a limitation of
Linux memory/cache manager. See SCST README file for details. For
BLOCKIO and pass-through dev handlers there's no such limitation, so you
can freely mount file systems over them.

To build, simply issue 'make' in the scst_local directory.

Try 'modinfo scst_local' for a listing of module parameters so far.

Here is how I have used it so far:

1. Load up scst:

  modprobe scst
  modprobe scst_vdisk

2. Create a virtual disk (or your own device handler):

  dd if=/dev/zero of=/some/path/vdisk1.img bs=16384 count=1000000
  echo "add_device vm_disk1 filename=/some/path/vdisk1.img" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt

3. Load the scst_local driver:

  insmod scst_local
  echo "add vm_disk1 0" >/sys/kernel/scst_tgt/targets/scst_local/scst_local_tgt/luns/mgmt

4. Check what you have

   cat /proc/scsi/scsi
  Attached devices:
  Host: scsi0 Channel: 00 Id: 00 Lun: 00
    Vendor: ATA      Model: ST9320320AS      Rev: 0303
    Type:   Direct-Access                    ANSI  SCSI revision: 05
  Host: scsi4 Channel: 00 Id: 00 Lun: 00
    Vendor: TSSTcorp Model: CD/DVDW TS-L632D Rev: TO04
    Type:   CD-ROM                           ANSI  SCSI revision: 05
  Host: scsi7 Channel: 00 Id: 00 Lun: 00
    Vendor: SCST_FIO Model: vm_disk1         Rev:  200
    Type:   Direct-Access                    ANSI  SCSI revision: 04

Or instead of manually "add_device" in (2) and step (3) write a
scstadmin config:

HANDLER vdisk_fileio {
        DEVICE vm_disk1 {
        	filename /some/path/vdisk1.img
        }
}

TARGET_DRIVER scst_local {
	TARGET scst_local_tgt {
		LUN 0 vm_disk1
	}
}

then:

  insmod scst_local
  scstadmin -config conf_file.cfg

More advanced examples:

For (3) you can:

  insmod scst_local add_default_tgt=0
  echo "add_target scst_local_tgt session_name=scst_local_host" >/sys/kernel/scst_tgt/targets/scst_local//mgmt
  echo "add vm_disk1 0" >/sys/kernel/scst_tgt/targets/scst_local/scst_local_tgt/luns/mgmt

Scst_local module's parameter add_default_tgt disables creation of
default target "scst_local_tgt" and session "scst_local_host", so you
needed to create it manually.

There can be any number of targets and sessions created. Each SCST
session corresponds to SCSI host. You can change which LUNs assigned to
each session by using SCST access control. This mode is intended for
user space target drivers (see below).

Alternatively, you can write an scstadmin's config file conf_file.cfg:

HANDLER vdisk_fileio {
        DEVICE vm_disk1 {
        	filename /some/path/vdisk1.img
        }
}

TARGET_DRIVER scst_local {
	TARGET scst_local_tgt {
		session_name scst_local_host

		LUN 0 vm_disk1
	}
}

then:

  insmod scst_local add_default_tgt=0
  scstadmin -config conf_file.cfg

NOTE! Although scstadmin allows to create scst_local's sessions using
"session_name" expression, it doesn't save existing sessions during
writing config file by "write_config" command. If you need this
functionality, feel free to send a request for it in SCST development
mailing list.

5. Have fun.

Some of this was coded while in Santa Clara, some in Bangalore, and some in
Hyderabad. Noe doubt some will be coded on the way back to Santa Clara.

The code still has bugs, so if you encounter any, email me the fixes at:

   realrichardsharpe@gmail.com

I am thinking of renaming this to something more interesting.


Sysfs interface
===============

See SCST's README for a common SCST sysfs description.

Root of this driver is /sys/kernel/scst_tgt/targets/scst_local. It has
the following additional entry:

 - stats - read-only attribute with some statistical information.

Each target subdirectory contains the following additional entries:

 - phys_transport_version - contains and allows to change physical
   transport version descriptor. It determines by which physical
   interface this target will look like. See SPC for more details. By
   default, it is not defined (0).

 - scsi_transport_version - contains and allows to change SCSI
   transport version descriptor. It determines by which SCSI
   transport this target will look like. See SPC for more details. By
   default, it is SAS.

Each session subdirectory contains the following additional entries:

 - transport_id - contains this host's TransportID. This TransportID
   used to identify initiator in Persisten Reservation commands. If you
   change scsi_transport_version for a target, make sure you set for all
   its sessions correct TransportID. See SPC for more details.

 - host - links to the corresponding SCSI host. Using it you can find
   local sg/bsg/sd/etc. devices of this session. For instance, this
   links points out to host12, so you can find your sg devices by:

$ lsscsi -g|grep "\[12:"
[12:0:0:0]   disk    SCST_FIO rd1               200  /dev/sdc  /dev/sg2
[12:0:0:1]   disk    SCST_FIO nullio            200  /dev/sdd  /dev/sg3

They are /dev/sg2 and /dev/sg3.

The following management commands available via /sys/kernel/scst_tgt/targets/scst_local/mgmt:

 - add_target target_name [session_name=sess_name; [session_name=sess_name1;] [...]] -
   creates a target with optionally one or more sessions.

 - del_target target_name - deletes a target.

 - add_session target_name session_name - adds to target target_name
   session (SCSI host) with name session_name.

 - del_session target_name session_name - deletes session session_name
    from target target_name.


Note on performance
===================

Although this driver implemented in the most performance effective way,
including zero-copy passing data between SCSI/block subsystems and SCST,
in many cases it is NOT suited to measure performance as a NULL link.
For example, it is not suited for max IOPS measurements. This is because
for such cases not performance of the link between the target and
initiator is the bottleneck, but CPU or memory speed on the target or
initiator. For scst_local you have both initiator and target on the same
system, which means each your initiator and target are much less
CPU/memory powerful.


User space target drivers
=========================

Scst_local can be used to write full featured SCST target drivers in
user space:

1. For each SCSI target a user space target driver should create an
   scst_local's target using "add_target" command.

2. Then the user space target driver should, if needed, set its SCSI and
   physical transport version descriptors using attributes
   scsi_transport_version and phys_transport_version correspondingly in
   /sys/kernel/scst_tgt/targets/scst_local/target_name directory.

3. For incoming session (I_T nexus) from an initiator the user space
   target driver should create scst_local's session using "add_session"
   command.

4. Then, if needed, the user space target driver should set TransportID
   for this session (I_T nexus) using attribute
   /sys/kernel/scst_tgt/targets/scst_local/target_name/sessions/session_name/transport_id

5. Then the user space target driver should find out sg/bsg devices for
   the LUNs the created session has using link
   /sys/kernel/scst_tgt/targets/scst_local/target_name/sessions/session_name/host
   as described above.

6. Then the user space target driver can start serving the initiator using
   found sg/bsg devices.

For other connected initiators steps 3-6 should be repeated.


Compilation options
===================

There are the following compilation options, that could be commented
in/out in Makefile:

 - CONFIG_SCST_LOCAL_FORCE_DIRECT_PROCESSING - by default, when this option
   is not defined, scst_local reschedules all commands for processing in
   one of the SCST threads. If this option is defined, scst_local tries
   to not do it, if possible (sometimes queuecommand() called under
   various locks held), but instead process them in the submitter's
   context. This is to increase performance, but as on 2.6.37 and below
   Linux block layer doesn't work with such kind of reentrance, hence
   this option disabled by default. Note! At the moment in
   scst_estimate_context*() returning DIRECT contexts disabled, so this
   option doesn't have any real effect.


Change log
==========

V0.1 24-Sep-2008 (Hyderabad) Initial coding, pretty chatty and messy,
                             but worked.

V0.2 25-Sep-2008 (Hong Kong) Cleaned up the code a lot, reduced the log
			     chatter, fixed a bug where multiple LUNs did not
			     work. Also, added logging control. Tested with
			     five virtual disks. They all came up as /dev/sdb
			     through /dev/sdf and I could dd to them. Also
			     fixed a bug preventing multiple adapters.

V0.3 26-Sep-2008 (Santa Clara) Added back a copyright plus cleaned up some
			       unused functions and structures.

V0.4 5-Oct-2008 (Santa Clara) Changed name to scst_local as suggested, cleaned
			      up some unused variables (made them used) and
			      change allocation to a kmem_cache pool.

V0.5 5-Oct-2008 (Santa Clara) Added mgmt commands to handle dev reset and
			      aborts. Not sure if aborts works. Also corrected
			      the version info and renamed readme to README.

V0.6 7-Oct-2008 (Santa Clara) Removed some redundant code and made some
			      changes suggested by Vladislav.

V0.7 11-Oct-2008 (Santa Clara) Moved into the scst tree. Cleaned up some
			       unused functions, used TRACE macros etc.

V0.9 30-Nov-2008 (Mtn View) Cleaned up an additional problem with symbols not
			    being defined in older version of the kernel. Also
			    fixed some English and cleaned up this doc.

V1.0 10-Sep-2010 (Moscow)   Sysfs management added. Reviewed and cleaned up.

V2.1                        Update for kernels up to 3.0. Cleanups.