<< | Thread Index | >> ]    [ << | Date Index | >> ]

Subject: Proposal: Compression of large packets
From: Josef Drexler <jdrexler,AT,josefsbox,DOT,cjb,DOT,net>
Date: Fri, 30 Nov 2001 20:11:30 +0100

Previously I used to have a VPN tunneled by ssh, with all the problems
that made Olaf Titz develop CIPE.  I like CIPE a lot, but the one
advantage that ssh had was it could compress the data stream.

I understand that doing this based on datagrams is more difficult, however
I did some statistics on my connection, using tcpdump and a perl script
that did the compression from tcpdumped packets to see how much they could
have been compressed.  The results from about a day of traffic are at the
end of this email.

What I found is that compression is only efficient for very large packets
(no surprise there);  basically only packets close to the MTU had a
significant compression ratio.  However, these make up for more than 50%
of the traffic, while only being less than 10% of the packets.  My tests
showed that compressing these gives a ratio of around 50-60%, good enough
to improve the bandwidth significantly.

Of course this would require a relatively fast computer, but that's true
for compressing with ssh as well.  However, by selecting a minimum packet
size for which compression is attempted, the performance penalty would
only occur for large packets, and for these it might well be an increase
in performance, depending on the bandwidth (i. e., is it faster to
transfer an uncompressed packet, or to compress and transfer the
compressed packet).

It would also be useful to make compression asymmetric, for example for
ADSL where only one direction is so slow that it would benefit from
compression.

CONCLUSION

Compression of large packets can significantly improve bandwidth while
keeping latency low for small packets.

WHAT NEEDS TO BE DONE

- decide what compression scheme to use
- make a new option to enable compression and choose minimum packet size
- add a protocol to negotaite compression with peer (probably in a
  CT_CONFREQ or CT_CONF
- define a bit in P to indicate whether a packet is compressed
- do the actual compression and decompression
- anything else?

STATISTICAL RESULTS

Legend:
Size    Size of the packets in bytes (octets); up to the next entry
Number  How many packets were transferred with this size
Pct     Percentage of packets with this size
PBytes  Percentage of total bytes to all traffic
AvgComp Average compression ratio using Zlib's "compress" function, in percent
PctComp Percentage of packets that could successfully be compressed
TotalB  Total bytes transferred
BSaved  Bytes saved through compression (only where successful)
PSaved  Percentage of bytes saved to all traffic

Size    Number  Pct     PBytes  AvgComp PctComp TotalB  BSaved  PSaved
0       4991     37      11     118       0     255181  0        11
64      6503     49      22     113       0     501071  31       22
128     337       2       2     106       1     50481   21        2
192     106       0       1     104       1     22649   5         1
256     101       0       1      99      15     28552   1089      1
320     77        0       1     100      15     26930   521       1
384     72        0       1      91      55     29328   2921      1
448     32        0       0      98      18     15151   549       0
512     12        0       0      99      16     6516    143       0
576     8         0       0      75      62     4869    1217      0
640     23        0       0      81      52     15325   3005      0
704     11        0       0      97       9     8136    303       0
768     14        0       0      93      21     11310   857       0
832     11        0       0      98       9     9357    273       0
896     7         0       0      90      28     6532    695       0
960     6         0       0      90      16     5895    586       0
1024    14        0       0      98       7     14681   423       0
1088    10        0       0      91      20     11275   1028      0
1152    5         0       0      74      60     5922    1540      0
1216    14        0       0      82      42     17398   3157      0
1280    6         0       0      79      50     7867    1679      0
1344    10        0       0     100      10     13818   11        0
1408    819       6      52      57      90     1171427 504134   29

-- 
   Josef Drexler                 |    http://publish.uwo.ca/~jdrexler/
---------------------------------+---------------------------------------
 Please help Conserve Gravity    | Email address is *valid*.
 Carry a helium balloon.         | Don't remove the "nospam" part.





<< | Thread Index | >> ]    [ << | Date Index | >> ]