mirror of
https://github.com/SCST-project/scst.git
synced 2026-08-22 15:16:33 +00:00
Gleb Chesnokov
860ce48072
README.performance runs fio against a block device, exports LUNs, logs in iSCSI sessions and may enable multipath. Its placeholders look copyable, but using the wrong path or host can overwrite storage and change both target and initiator state. Put an explicit safety boundary before the first command. Require authorization for the exact hosts, transports and configuration, and allow the workflow only with verified disposable backing devices.
73 lines
3.0 KiB
Plaintext
73 lines
3.0 KiB
Plaintext
SCST Performance
|
|
================
|
|
|
|
A question that is asked often is how to tune performance. This means how to
|
|
improve the IOPS and/or bandwidth measured at the initiator side. In this
|
|
README it is explained how to optimize storage performance.
|
|
|
|
WARNING: The examples below issue direct I/O, create and export LUNs, create
|
|
initiator sessions and may activate multipath. Run them only with explicit
|
|
authorization for the target and initiator hosts, transports, configuration
|
|
and disposable backing devices. Verify every placeholder before use.
|
|
|
|
Local storage
|
|
-------------
|
|
Start with measuring the performance of the local block device(s). IOPS can be
|
|
measured e.g. as follows:
|
|
|
|
fio --ioengine=libaio --rw=randread --ioscheduler=none --numjobs=$(nproc) \
|
|
--runtime=60 --group_reporting=1 --gtod_reduce=1 --norandommap \
|
|
--thread --buffered=0 --iodepth=256 --iodepth_batch=128 --bs=4k \
|
|
--name=bdev --filename=/dev/...
|
|
|
|
The bandwidth supported by a block device can be measured by increasing the
|
|
block size to a larger value in the above command, e.g. --bs=1M.
|
|
|
|
Storage network
|
|
---------------
|
|
Start with measuring the network bandwidth using your favorite tool, e.g.
|
|
netperf for non-RDMA networks or ib_write_bw for RDMA networks.
|
|
|
|
Next, add a nullio LUN to SCST, e.g. by adding the following in /etc/scst.conf:
|
|
|
|
HANDLER vdisk_nullio {
|
|
DEVICE disk09 {
|
|
blocksize 4096
|
|
size_mb 256
|
|
}
|
|
}
|
|
|
|
TARGET_DRIVER ... {
|
|
TARGET ... {
|
|
LUN ... disk09
|
|
}
|
|
}
|
|
|
|
After a nullio LUN has been added, verify that this LUN is visible at the
|
|
initiator side. If it is not visible at the initiator side, consider
|
|
rescanning LUNs or disconnecting and reconnecting the initiator system to the
|
|
SCST server.
|
|
|
|
Once the nullio LUN is visible at the initiator side, measure IOPS and
|
|
bandwidth. The bandwidth should be close to the network bandwidth. When using
|
|
the Linux iSCSI initiator this may require configuring multiple iSCSI sessions
|
|
and activating multipathd on top of the multiple iSCSI sessions. The sequence
|
|
for logging in with iSCSI and activating multiple iSCSI sessions is as follows:
|
|
|
|
iscsiadm -m iface -I iface2 -o new
|
|
iscsiadm -m iface -I iface2 -o update -n iface.initiatorname -v ${iqn2}
|
|
iscsiadm -m discovery -t st -p ${scst_ip_address}
|
|
iscsiadm -m discovery -t st -p ${scst_ip_address} -I iface2
|
|
iscsiadm -m node -p ${scst_ip_address} -l
|
|
iscsiadm -m node -p ${scst_ip_address} -I iface2 -l
|
|
|
|
SCST Configuration
|
|
------------------
|
|
If the number of IOPS measured at the initiator side is significantly lower
|
|
than the minimum of the IOPS supported by the local storage and the storage
|
|
network, further tuning is required. Look up in /proc/interrupts which CPU
|
|
cores process the most network and storage interrupts and configure the SCST
|
|
kernel threads such that these run on other CPU cores than those that process
|
|
the most interrupts by configuring the cpu_mask attribute. More information
|
|
about the SCST cpu_mask sysfs attribute is available in the SCST README.
|