On-Wire Compression + Decompression¶
Use of this feature reduces the overall network overhead for Gluster operations from a client. Depending on workload, this could show dramatic increases in the performance of Gluster volumes. This feature also allows a good trade-off of CPU to network resources, which will be a boon to most users as CPU is not generally being consumed to anywhere near its full potential, whereas network has traditionally been the bottleneck in high performance workloads.
Setup¶
# brick="/mnt/lv3/vol-comp"; gluster volume create vol-comp eins:$brick
volume create: vol-comp: success: please start the volume to access data
# gluster volume start vol-comp
volume start: vol-comp: success
# gluster volume set vol-comp network.compression on
volume set: success
[2014-05-18 23:51:50]# gluster volume set vol-comp network.compression.compression-level 8
volume set: success
[2014-05-18 23:56:49]# gluster volume set vol-comp network.compression.min-size 50
volume set: success
[2014-05-18 23:57:02]# gluster volume info vol-comp
Volume Name: vol-comp
Type: Distribute
Volume ID: 92b47734-2552-4168-b3c3-151093562e4f
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: eins:/mnt/lv3/vol-comp
Options Reconfigured:
network.compression.min-size: 50
network.compression.compression-level: 8
network.compression.mode: server
network.compression: on
# mount -t glusterfs localhost:/vol-comp /mnt/glusterfs/vol-comp
Do¶
# dd if=/dev/zero of=/mnt/glusterfs/vol-comp/1gb.dat bs=1M count=1024
dd: writing `/mnt/glusterfs/vol-comp/1gb.dat': Input/output error
dd: closing output file `/mnt/glusterfs/vol-comp/1gb.dat': Input/output error
[2014-05-18 14:59:24.034499] W [fuse-bridge.c:2201:fuse_writev_cbk] 0-glusterfs-fuse: 12: WRITE => -1 (Input/output error)
[2014-05-18 14:59:24.034917] W [fuse-bridge.c:2201:fuse_writev_cbk] 0-glusterfs-fuse: 14: WRITE => -1 (Input/output error)
[2014-05-18 14:59:24.035439] W [fuse-bridge.c:1206:fuse_err_cbk] 0-glusterfs-fuse: 15: FLUSH() ERR => -1 (Input/output error)
[2014-05-18 14:59:59.799649] W [fuse-bridge.c:2201:fuse_writev_cbk] 0-glusterfs-fuse: 46: WRITE => -1 (Input/output error)
[2014-05-18 14:59:59.800123] W [fuse-bridge.c:2201:fuse_writev_cbk] 0-glusterfs-fuse: 48: WRITE => -1 (Input/output error)
[2014-05-18 14:59:59.804226] W [fuse-bridge.c:1206:fuse_err_cbk] 0-glusterfs-fuse: 49: FLUSH() ERR => -1 (Input/output error)
[2014-05-18 15:00:36.742227] W [fuse-bridge.c:2201:fuse_writev_cbk] 0-glusterfs-fuse: 61: WRITE => -1 (Input/output error)
[2014-05-18 15:00:36.742738] W [fuse-bridge.c:2201:fuse_writev_cbk] 0-glusterfs-fuse: 63: WRITE => -1 (Input/output error)
[2014-05-18 15:00:36.745857] W [fuse-bridge.c:1206:fuse_err_cbk] 0-glusterfs-fuse: 64: FLUSH() ERR => -1 (Input/output error)
# gluster volume reset vol-comp
volume reset: success: reset volume successful
# dd if=/dev/zero of=/mnt/glusterfs/vol-comp/1gb.dat bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 9.14845 s, 117 MB/s
# gluster volume set vol-comp network.compression on
volume set: success
# dd if=/dev/zero of=/mnt/glusterfs/vol-comp/1gb.dat bs=1M count=1024
dd: writing `/mnt/glusterfs/vol-comp/1gb.dat': Input/output error
dd: closing output file `/mnt/glusterfs/vol-comp/1gb.dat': Input/output error
# gluster volume set vol-comp performance.strict-write-ordering on
volume set: success
# gluster volume info vol-comp
Volume Name: vol-comp
Type: Distribute
Volume ID: 92b47734-2552-4168-b3c3-151093562e4f
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: eins:/mnt/lv3/vol-comp
Options Reconfigured:
performance.strict-write-ordering: on
network.compression.mode: server
network.compression: on
# dd if=/dev/zero of=/mnt/glusterfs/vol-comp/1gb.dat bs=1M count=1024
dd: writing `/mnt/glusterfs/vol-comp/1gb.dat': Input/output error
dd: closing output file `/mnt/glusterfs/vol-comp/1gb.dat': Input/output error
# gluster volume set vol-comp performance.write-behind off
volume set: success
# dd if=/dev/zero of=/mnt/glusterfs/vol-comp/1gb.dat bs=1M count=1024
dd: writing `/mnt/glusterfs/vol-comp/1gb.dat'
1+0 records in
0+0 records out
1048577 bytes (1.0 MB) copied, 21.8345 s, 48.0 kB/s
# ls -lh /mnt/glusterfs/vol-comp
total 1.1M
-rw-r--r-- 1 root root 1.1M May 19 00:10 1gb.dat
# echo "test" > /mnt/glusterfs/vol-comp/test.txt
echo: write error: success
# cp /tmp/test.txt /mnt/glusterfs/vol-comp/
cp: writing `/mnt/glusterfs/vol-comp/test.txt': Bad address
# gluster volume set vol-comp network.compression.compression-level 0
volume set: success
# cp /tmp/test.txt /mnt/glusterfs/vol-comp/
cp: writing `/mnt/glusterfs/vol-comp/test.txt': Bad address
# file /mnt/glusterfs/vol-comp/test.txt
/mnt/glusterfs/vol-comp/test.txt: data
# file /tmp/test.txt
/tmp/test.txt: ASCII text
# diff /mnt/glusterfs/vol-comp/test.txt /tmp/test.txt
Binary files /mnt/glusterfs/vol-comp/test.txt and /tmp/test.txt differ
[eins]# file /mnt/lv3/vol-comp/test.txt
/mnt/lv3/vol-comp/test.txt: data
Success¶
# gluster volume info vol-comp
Volume Name: vol-comp
Type: Distribute
Volume ID: 92b47734-2552-4168-b3c3-151093562e4f
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: eins:/mnt/lv3/vol-comp
Options Reconfigured:
network.compression.compression-level: 0
performance.open-behind: off
performance.write-behind: off
performance.strict-write-ordering: on
network.compression.mode: server
network.compression: on
# gluster volume set vol-comp network.compression.compression-level 9
volume set: success
# cp /tmp/test.txt /mnt/glusterfs/vol-comp/
# dd if=/dev/zero of=/mnt/glusterfs/vol-comp/1gb.dat bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 33.8606 s, 31.7 MB/s
[2014-05-19 0:25:40]# diff /mnt/glusterfs/vol-comp/1gb.dat /tmp/1gb.dat