From f407c425c0e5338de9a78e683d6e1a9e1ba71913 Mon Sep 17 00:00:00 2001 From: Bart Van Assche Date: Sat, 8 Oct 2011 11:17:37 +0000 Subject: [PATCH] Reran performance measurements git-svn-id: http://svn.code.sf.net/p/scst/svn/branches/2.1.0.x@3893 d57e44dd-8a1f-0410-8b47-8ef2f437770f --- srpt/Measurement-Results.txt | 126 +++++++++++------------------------ 1 file changed, 40 insertions(+), 86 deletions(-) diff --git a/srpt/Measurement-Results.txt b/srpt/Measurement-Results.txt index 136ea8979..ee6b3d4b9 100644 --- a/srpt/Measurement-Results.txt +++ b/srpt/Measurement-Results.txt @@ -2,117 +2,71 @@ Setup details ============= Initiator system hardware: -* IB HCA: QDR Mellanox ConnectX MT26428 -* CPU: Intel Core2 Duo E6750 @ 2.66 GHz. +* IB HCA: QDR Mellanox ConnectX MT26428. +* CPU: Intel Core2 Duo E8400 @ 3.00 GHz. Initiator system hardware configuration: * CPU frequency scaling was disabled. * IRQ affinity for mlx4_core: not bound to a CPU core (smp_affinity=3). Initiator system software: -* Kernel 2.6.34.1 +* Kernel 2.6.34.10 +* fio version fio-1.59-2-g5efce23 Initiator system software configuration: * Runlevel: 3. -* SRP initiator was loaded with parameter srp_sg_tablesize=128. +* SRP initiator was loaded with parameter srp_sg_tablesize=255. * SRP initiator block device I/O scheduler: noop. Target system hardware: -* IB HCA: QDR Mellanox ConnectX MT26428 -* CPU: Intel Core2 Duo E8400 @ 3.00 GHz. +* IB HCA: QDR Mellanox ConnectX MT26428. +* CPU: Intel Core i5-2400 @ 3.10 GHz. Target system hardware configuration: * CPU frequency scaling was disabled. -* IRQ affinity for mlx4-comp-0: not bound to a CPU core (smp_affinity=3). +* IRQ affinity for mlx4-comp-0: not bound to a CPU core (smp_affinity=f). Target system software: -* 2.6.34 kernel with SCST patches applied and with kernel debugging disabled. +* 2.6.34.10 kernel with SCST patches applied and with kernel debugging disabled. * Mainstream kernel InfiniBand drivers (no OFED). -* SCST revision 1857 built in release mode (make debug2release) and - with SCST_MAX_TGT_DEV_COMMANDS equal to 48. +* SCST revision 3883 built in performance mode (make debug2perf) and with + SCST_MAX_TGT_DEV_COMMANDS equal to 64. Target system software configuration: * Runlevel: 3. -* ib_srpt kernel module was loaded with the parameter thread=0. +* ib_srpt kernel module was loaded with these kernel module parameters: + srp_max_req_size=4200 thread=1 * A 1 GB file residing on a tmpfs filesystem was exported towards the - initiator system. - + initiator system. From scst.conf: + HANDLER vdisk_fileio { + DEVICE disk01 { + filename /dev/vdisk + nv_cache 1 + threads_num 1 + } + } Measurement results =================== -+ blockdev-perftest -f -d -s 30 -m 12 /dev/sdb -blocksize W W W W(avg, W(std, W R R R R(avg, R(std, R - (bytes) (s) (s) (s) MB/s) MB/s) (IOPS) (s) (s) (s) MB/s) MB/s) (IOPS) - 67108864 1.250 1.251 1.252 818.484 0.471 12.789 0.602 0.601 0.603 1701.977 2.425 26.593 - 33554432 1.221 1.222 1.220 838.449 0.483 26.202 0.610 0.610 0.608 1680.218 2.441 52.507 - 16777216 1.200 1.208 1.200 851.360 2.694 53.210 0.624 0.624 0.623 1642.135 1.226 102.633 - 8388608 1.154 1.153 1.155 887.313 0.716 110.914 0.676 0.670 0.651 1539.438 24.668 192.430 - 4194304 1.085 1.094 1.089 940.192 3.360 235.048 0.699 0.727 0.702 1444.004 25.142 361.001 - 2097152 1.127 1.130 1.130 906.980 0.975 453.490 0.795 0.783 0.783 1301.447 9.198 650.724 - 1048576 1.221 1.225 1.219 837.981 1.744 837.981 0.891 0.891 0.895 1147.564 2.127 1147.564 - 524288 1.449 1.450 1.449 706.579 0.141 1413.158 1.045 1.040 1.038 983.418 2.570 1966.836 - 262144 1.585 1.581 1.585 646.580 0.803 2586.320 1.110 1.109 1.110 922.567 0.557 3690.269 - 131072 1.751 1.753 1.763 583.286 1.783 4666.290 1.244 1.239 1.240 825.115 1.416 6600.917 - 65536 2.180 2.185 2.174 469.835 0.961 7517.363 1.592 1.562 1.563 651.305 5.637 10420.879 - 32768 2.924 2.959 2.918 349.049 2.165 11169.563 2.221 2.227 2.215 461.049 0.970 14753.581 - 16384 4.374 4.395 4.396 233.338 0.533 14933.643 3.437 3.409 3.416 299.369 1.022 19159.585 - 8192 7.400 7.436 7.445 137.874 0.360 17647.884 5.913 5.932 5.930 172.820 0.247 22120.931 - 4096 13.079 13.268 13.207 77.670 0.464 19883.400 10.686 10.648 10.656 96.028 0.148 24583.280 - -+ blockdev-perftest -f -a -s 30 -m 12 /dev/sdb -blocksize W W W W(avg, W(std, W R R R R(avg, R(std, R - (bytes) (s) (s) (s) MB/s) MB/s) (IOPS) (s) (s) (s) MB/s) MB/s) (IOPS) - 67108864 2.018 2.046 2.006 506.134 4.091 7.908 1.515 1.511 1.516 676.438 0.814 10.569 - 33554432 1.998 2.022 1.997 510.577 2.933 15.956 1.508 1.512 1.511 678.075 0.823 21.190 - 16777216 2.011 1.997 1.998 511.558 1.636 31.972 1.515 1.510 1.517 676.415 1.280 42.276 - 8388608 1.998 2.019 2.003 510.279 2.266 63.785 1.489 1.497 1.486 687.019 2.047 85.877 - 4194304 1.948 1.979 1.949 522.812 3.882 130.703 1.382 1.385 1.385 739.782 0.823 184.945 - 2097152 1.707 1.695 1.696 602.627 1.853 301.313 1.274 1.270 1.284 802.520 3.848 401.260 - 1048576 1.385 1.411 1.401 731.984 5.613 731.984 1.148 1.190 1.175 874.622 12.953 874.622 - 524288 1.319 1.308 1.338 774.982 7.152 1549.965 1.136 1.139 1.140 899.357 1.243 1798.714 - 262144 1.310 1.305 1.324 779.899 4.803 3119.597 1.132 1.129 1.129 906.074 1.313 3624.295 - 131072 1.324 1.323 1.333 771.847 2.702 6174.780 1.135 1.132 1.132 903.715 1.237 7229.716 - 65536 1.304 1.316 1.302 783.338 3.731 12533.402 1.145 1.143 1.132 898.495 4.641 14375.927 - 32768 1.316 1.351 1.316 771.388 9.403 24684.407 1.139 1.141 1.135 899.459 1.842 28782.694 - 16384 1.349 1.314 1.312 772.877 9.936 49464.155 1.149 1.152 1.144 891.722 2.721 57070.239 - 8192 1.332 1.320 1.355 766.724 8.241 98140.706 1.162 1.144 1.175 882.757 9.511 112992.880 - 4096 1.374 1.364 1.389 744.407 5.620 190568.208 1.185 1.173 1.189 865.959 5.002 221685.510 +Numer of read IOPS reported by fio with block size 4 KB and I/O depth 32: +195000 + blockdev-perftest -f -j -d -s 30 -m 12 /dev/sdb -blocksize W W W W(avg, W(std, W R R R R(avg, R(std, R - (bytes) (s) (s) (s) MB/s) MB/s) (IOPS) (s) (s) (s) MB/s) MB/s) (IOPS) - 67108864 0.726 0.727 0.730 1407.245 3.283 21.988 0.600 0.600 0.599 1707.616 1.343 26.682 - 33554432 0.738 0.731 0.720 1403.526 14.290 43.860 0.605 0.605 0.605 1692.562 0.000 52.893 - 16777216 0.725 0.726 0.731 1407.901 5.069 87.994 0.619 0.616 0.617 1658.754 3.349 103.672 - 8388608 0.746 0.745 0.756 1367.216 9.025 170.902 0.639 0.640 0.640 1600.835 1.180 200.104 - 4194304 0.775 0.783 0.782 1312.848 6.009 328.212 0.672 0.675 0.678 1517.057 5.505 379.264 - 2097152 0.863 0.871 0.865 1182.011 4.628 591.006 0.741 0.740 0.738 1384.411 2.336 692.206 - 1048576 0.964 0.960 0.973 1060.441 5.956 1060.441 0.840 0.837 0.849 1216.196 7.342 1216.196 - 524288 1.197 1.185 1.198 858.122 4.262 1716.243 0.981 1.001 0.994 1032.330 8.649 2064.661 - 262144 1.312 1.311 1.313 780.488 0.486 3121.952 1.079 1.112 1.109 931.082 12.730 3724.326 - 131072 1.536 1.538 1.533 666.813 0.893 5334.501 1.268 1.260 1.261 810.774 2.280 6486.196 - 65536 1.943 1.937 1.941 527.745 0.679 8443.924 1.654 1.644 1.646 621.363 1.626 9941.816 - 32768 2.755 2.753 2.745 372.229 0.585 11911.334 2.321 2.286 2.303 444.590 2.758 14226.886 - 16384 4.371 4.325 4.366 235.191 1.117 15052.245 3.644 3.680 3.644 280.094 1.296 17925.987 - 8192 7.553 7.589 7.531 135.493 0.428 17343.092 6.333 6.223 6.248 163.379 1.222 20912.470 - 4096 13.793 13.573 13.802 74.626 0.579 19104.136 11.348 11.551 11.287 89.870 0.885 23006.749 - -+ blockdev-perftest -f -j -a -s 30 -m 12 /dev/sdb -blocksize W W W W(avg, W(std, W R R R R(avg, R(std, R - (bytes) (s) (s) (s) MB/s) MB/s) (IOPS) (s) (s) (s) MB/s) MB/s) (IOPS) - 67108864 1.505 1.508 1.504 680.098 0.767 10.627 1.345 1.345 1.345 761.338 0.000 11.896 - 33554432 1.515 1.503 1.503 679.505 2.544 21.235 1.342 1.342 1.340 763.420 0.537 23.857 - 16777216 1.506 1.512 1.504 679.349 1.530 42.459 1.337 1.337 1.336 766.085 0.270 47.880 - 8388608 1.514 1.505 1.504 679.201 2.022 84.900 1.321 1.325 1.323 774.000 0.955 96.750 - 4194304 1.511 1.518 1.506 677.405 2.204 169.351 1.246 1.239 1.239 824.925 2.189 206.231 - 2097152 1.498 1.491 1.503 683.890 2.249 341.945 1.035 1.037 1.042 986.520 2.794 493.260 - 1048576 1.405 1.414 1.406 727.107 2.075 727.107 0.950 0.954 0.960 1072.646 4.613 1072.646 - 524288 1.308 1.345 1.308 775.696 10.152 1551.392 0.904 0.907 0.921 1124.525 9.103 2249.050 - 262144 1.316 1.294 1.299 785.920 5.657 3143.678 0.907 0.907 0.904 1130.246 1.766 4520.982 - 131072 1.248 1.239 1.237 824.932 3.172 6599.453 0.882 0.874 0.878 1166.303 4.338 9330.425 - 65536 1.278 1.266 1.257 808.246 5.481 12931.929 0.887 0.883 0.885 1157.066 2.135 18513.057 - 32768 1.311 1.299 1.302 785.288 3.064 25129.218 0.891 0.891 0.890 1149.701 0.609 36790.429 - 16384 1.391 1.399 1.378 737.073 4.599 47172.659 0.935 0.934 0.937 1094.799 1.459 70067.124 - 8192 1.550 1.533 1.529 666.112 3.931 85262.303 1.011 1.010 1.012 1012.859 0.818 129645.980 - 4096 1.864 1.847 1.862 551.238 2.257 141117.020 1.289 1.290 1.287 794.621 0.769 203422.850 +blocksize W W W W(avg, W(std, W R R R R(avg, R(std, R + (bytes) (s) (s) (s) MB/s) MB/s) (IOPS) (s) (s) (s) MB/s) MB/s) (IOPS) + 67108864 0.564 0.559 0.569 1815.698 13.143 28.370 0.372 0.371 0.372 2755.161 3.498 43.049 + 33554432 0.558 0.558 0.565 1827.547 10.718 57.111 0.373 0.372 0.374 2745.321 6.010 85.791 + 16777216 0.552 0.548 0.552 1859.586 6.383 116.224 0.385 0.382 0.383 2671.333 8.681 166.958 + 8388608 0.523 0.525 0.527 1950.495 6.067 243.812 0.412 0.408 0.411 2495.575 10.360 311.947 + 4194304 0.545 0.544 0.543 1882.357 2.825 470.589 0.432 0.436 0.431 2364.955 11.764 591.239 + 2097152 0.597 0.606 0.610 1694.567 15.304 847.283 0.520 0.512 0.513 1988.444 13.679 994.222 + 1048576 0.699 0.695 0.697 1469.162 3.442 1469.162 0.589 0.588 0.582 1746.496 9.239 1746.496 + 524288 0.948 0.955 0.955 1074.890 3.732 2149.781 0.815 0.794 0.796 1277.515 14.960 2555.031 + 262144 1.096 1.084 1.105 935.218 7.357 3740.870 0.994 0.897 0.895 1105.299 53.127 4421.198 + 131072 1.258 1.280 1.249 811.282 8.329 6490.257 1.083 1.066 1.069 954.676 6.566 7637.404 + 65536 1.601 1.609 1.641 633.343 6.726 10133.494 1.383 1.308 1.383 754.571 20.014 12073.138 + 32768 2.318 2.312 2.333 441.196 1.676 14118.257 1.960 1.957 1.892 528.975 8.669 16927.201 + 16384 3.769 3.776 3.779 271.283 0.301 17362.084 3.092 3.010 3.096 334.042 4.357 21378.686 + 8192 6.807 6.728 6.827 150.875 0.954 19312.035 5.256 5.225 5.282 194.891 0.865 24945.996 + 4096 12.260 12.277 12.258 83.490 0.058 21373.349 9.851 9.737 9.587 105.309 1.173 26959.019