diff --git a/iscsi-scst/README b/iscsi-scst/README index 71501c323..b2227d25a 100644 --- a/iscsi-scst/README +++ b/iscsi-scst/README @@ -952,6 +952,39 @@ disable_sendpage parameter for drbd module. But in this case it might be better for performance simply not apply put_page_callback patch. +Background information about zero-copy data sending +--------------------------------------------------- + +As explained above the most efficient operation of the iSCSI-SCST target +driver is achieved when the following two conditions are met: +* Data is sent from target to initiator in a zero-copy fashion. +* Data buffers are cached for reuse (by the so-called sgv pool). +Unfortunately the zero-copy API in the Linux kernel (proto.sendpage() / +tcp_sendpage()) does not yet support completion notifications. Hence the +put_page_callback patch which adds completion notification support to +tcp_sendpage(). However, since the put_page_callback patch increases the size +of the page structure in the Linux kernel that patch is considered +unacceptable for integration in the mainline kernel [1]. Another approach, +called skb paged fragment destructors, might get merged in the mainline kernel +in the future. Version five of that patch series has been posted in May 2012 +[2]. + +Notes: +* The current implementation of scst_user is such that iSCSI-SCST only sends + data allocated by scst_user in a zero-copy fashion if the put_page_callback + patch has been applied. This holds even if scst_user did not use the sgv + pool to allocate a data buffer. +* Zero-copy sending is only possible with network interface drivers that + support scatter/gather and checksumming (NETIF_F_SG and NETIF_F_ALL_CSUM + respectively). + +References: +[1] James Bottomley, Linux Kernel Mailing List, December 2008, + http://lkml.org/lkml/2008/12/11/213. +[2] Ian Campbell, [PATCH v5 0/9] skb paged fragment destructors, May 3 2012, + linux-netdev, http://marc.info/?l=linux-netdev&m=133605705111948&w=2. + + Credits ------- diff --git a/scst/README.drbd b/scst/README.drbd new file mode 100644 index 000000000..dbb6cc6a5 --- /dev/null +++ b/scst/README.drbd @@ -0,0 +1,46 @@ +SCST, DRBD and Dual Primary Mode +******************************** + +Introduction +------------ +One possible approach to build a highly available storage solution is to +combine SCST with DRBD. While designing such a solution one has to choose an +operation mode for DRBD. While single primary mode is the easiest to set up, +with dual primary mode it becomes possible to balance the load for a single +LUN over the clustered servers. However, SCST and DRBD settings have to be +chosen carefully in order to avoid data corruption with dual primary +mode. One must make sure that SCST only confirms a write as finished to the +multipath initiator after the data involved in write operation has been +passed on to DRBD on both nodes. + +SCST Configuration +------------------ +In order to avoid data corruption it is crucial to use the following SCST +settings for each DRBD block device exported via SCST: +* One can use either the vdisk_blockio device handler or vdisk_fileio. +* Caching MUST be disabled: set the nv_cache parameter to 0 in scst.conf. +* Write-through MUST be enabled: set the write_through parameter to 1 in + scst.conf. + +DRBD Configuration +------------------ +As documented in the DRBD manual, dual primary mode only works with the DRBD +replication protocol C and not with protocol A nor with protocol B. + +Initiator Configuration +----------------------- +When using VMware, do NOT use multipath I/O in round-robin mode. This mode +namely depends on persistent reservations to guarantee data +consistency. When running an SCST cluster, the SCST persistent reservation +state is not shared between the two nodes. + +References +---------- +* DRBD authors, [Dual Primary Mode], DRBD Manual, 2011. +* VMware, [iSCSI SAN Configuration Guide 4.1], VMware website, September 2010. +* Phil White, [VMware ESX, MPIO, SCST and DRBD in Dual Primary Mode], SCST + Developers Mailing List, June 2010. +* Stefan Berger, [SCST and DRBD in Dual Primary Mode], DRBD users mailing + list, June 2010. +* Bart Van Assche [SCST and DRBD in Dual Primary Mode], SCST Developers + Mailing List, May 2011.