From 3e639ad97af0a97d534d2238661b68934c24e5c6 Mon Sep 17 00:00:00 2001 From: Bart Van Assche Date: Wed, 17 Oct 2012 17:53:48 +0000 Subject: [PATCH] scst/README: Move information from the SCST wiki into the Subversion tree The wiki has been announced to be shut down in 2012Q4 toghether with all other hosted SourceForge apps, so move the information from there into the SCST source tree. git-svn-id: http://svn.code.sf.net/p/scst/svn/trunk@4563 d57e44dd-8a1f-0410-8b47-8ef2f437770f --- iscsi-scst/README | 33 +++++++++++++++++++++++++++++++++ scst/README.drbd | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+) create mode 100644 scst/README.drbd diff --git a/iscsi-scst/README b/iscsi-scst/README index 71501c323..b2227d25a 100644 --- a/iscsi-scst/README +++ b/iscsi-scst/README @@ -952,6 +952,39 @@ disable_sendpage parameter for drbd module. But in this case it might be better for performance simply not apply put_page_callback patch. +Background information about zero-copy data sending +--------------------------------------------------- + +As explained above the most efficient operation of the iSCSI-SCST target +driver is achieved when the following two conditions are met: +* Data is sent from target to initiator in a zero-copy fashion. +* Data buffers are cached for reuse (by the so-called sgv pool). +Unfortunately the zero-copy API in the Linux kernel (proto.sendpage() / +tcp_sendpage()) does not yet support completion notifications. Hence the +put_page_callback patch which adds completion notification support to +tcp_sendpage(). However, since the put_page_callback patch increases the size +of the page structure in the Linux kernel that patch is considered +unacceptable for integration in the mainline kernel [1]. Another approach, +called skb paged fragment destructors, might get merged in the mainline kernel +in the future. Version five of that patch series has been posted in May 2012 +[2]. + +Notes: +* The current implementation of scst_user is such that iSCSI-SCST only sends + data allocated by scst_user in a zero-copy fashion if the put_page_callback + patch has been applied. This holds even if scst_user did not use the sgv + pool to allocate a data buffer. +* Zero-copy sending is only possible with network interface drivers that + support scatter/gather and checksumming (NETIF_F_SG and NETIF_F_ALL_CSUM + respectively). + +References: +[1] James Bottomley, Linux Kernel Mailing List, December 2008, + http://lkml.org/lkml/2008/12/11/213. +[2] Ian Campbell, [PATCH v5 0/9] skb paged fragment destructors, May 3 2012, + linux-netdev, http://marc.info/?l=linux-netdev&m=133605705111948&w=2. + + Credits ------- diff --git a/scst/README.drbd b/scst/README.drbd new file mode 100644 index 000000000..dbb6cc6a5 --- /dev/null +++ b/scst/README.drbd @@ -0,0 +1,46 @@ +SCST, DRBD and Dual Primary Mode +******************************** + +Introduction +------------ +One possible approach to build a highly available storage solution is to +combine SCST with DRBD. While designing such a solution one has to choose an +operation mode for DRBD. While single primary mode is the easiest to set up, +with dual primary mode it becomes possible to balance the load for a single +LUN over the clustered servers. However, SCST and DRBD settings have to be +chosen carefully in order to avoid data corruption with dual primary +mode. One must make sure that SCST only confirms a write as finished to the +multipath initiator after the data involved in write operation has been +passed on to DRBD on both nodes. + +SCST Configuration +------------------ +In order to avoid data corruption it is crucial to use the following SCST +settings for each DRBD block device exported via SCST: +* One can use either the vdisk_blockio device handler or vdisk_fileio. +* Caching MUST be disabled: set the nv_cache parameter to 0 in scst.conf. +* Write-through MUST be enabled: set the write_through parameter to 1 in + scst.conf. + +DRBD Configuration +------------------ +As documented in the DRBD manual, dual primary mode only works with the DRBD +replication protocol C and not with protocol A nor with protocol B. + +Initiator Configuration +----------------------- +When using VMware, do NOT use multipath I/O in round-robin mode. This mode +namely depends on persistent reservations to guarantee data +consistency. When running an SCST cluster, the SCST persistent reservation +state is not shared between the two nodes. + +References +---------- +* DRBD authors, [Dual Primary Mode], DRBD Manual, 2011. +* VMware, [iSCSI SAN Configuration Guide 4.1], VMware website, September 2010. +* Phil White, [VMware ESX, MPIO, SCST and DRBD in Dual Primary Mode], SCST + Developers Mailing List, June 2010. +* Stefan Berger, [SCST and DRBD in Dual Primary Mode], DRBD users mailing + list, June 2010. +* Bart Van Assche [SCST and DRBD in Dual Primary Mode], SCST Developers + Mailing List, May 2011.