<< | Thread Index | >> ]    [ << | Date Index | >> ]

Subject: Re: Proposal: Compression of large packets
From: ewheeler,AT,kaico,DOT,com
Date: Fri, 30 Nov 2001 21:58:47 +0100
In-reply-to: <Pine.LNX.4.31.0111301310170.27669-100000@josefsbox.cjb.net>

Josef --

  I would help with this porject!  I have part of this implemented
already, and zlib working within the kernel (i know, zlib is kinda
bloated). See this: http://xeon.eboch.com/cipe/output.c

I've added a cipe_compress function which only compresses outgoing
packets, but I can't figure out where to put the code to decompress the
incoming packet!

Ideas?

On Fri, 30 Nov 2001, Josef Drexler wrote:

> 
> Previously I used to have a VPN tunneled by ssh, with all the problems
> that made Olaf Titz develop CIPE.  I like CIPE a lot, but the one
> advantage that ssh had was it could compress the data stream.
> 
> I understand that doing this based on datagrams is more difficult, however
> I did some statistics on my connection, using tcpdump and a perl script
> that did the compression from tcpdumped packets to see how much they could
> have been compressed.  The results from about a day of traffic are at the
> end of this email.
> 
> What I found is that compression is only efficient for very large packets
> (no surprise there);  basically only packets close to the MTU had a
> significant compression ratio.  However, these make up for more than 50%
> of the traffic, while only being less than 10% of the packets.  My tests
> showed that compressing these gives a ratio of around 50-60%, good enough
> to improve the bandwidth significantly.
> 
> Of course this would require a relatively fast computer, but that's true
> for compressing with ssh as well.  However, by selecting a minimum packet
> size for which compression is attempted, the performance penalty would
> only occur for large packets, and for these it might well be an increase
> in performance, depending on the bandwidth (i. e., is it faster to
> transfer an uncompressed packet, or to compress and transfer the
> compressed packet).
> 
> It would also be useful to make compression asymmetric, for example for
> ADSL where only one direction is so slow that it would benefit from
> compression.
> 
> 
> CONCLUSION
> 
> Compression of large packets can significantly improve bandwidth while
> keeping latency low for small packets.
> 
> 
> WHAT NEEDS TO BE DONE
> 
> - decide what compression scheme to use
> - make a new option to enable compression and choose minimum packet size
> - add a protocol to negotaite compression with peer (probably in a
>   CT_CONFREQ or CT_CONF
> - define a bit in P to indicate whether a packet is compressed
> - do the actual compression and decompression
> - anything else?
> 
> 
> STATISTICAL RESULTS
> 
> Legend:
> Size  Size of the packets in bytes (octets); up to the next entry
> Number        How many packets were transferred with this size
> Pct   Percentage of packets with this size
> PBytes  Percentage of total bytes to all traffic
> AvgComp       Average compression ratio using Zlib's "compress" function, 
>in percent
> PctComp Percentage of packets that could successfully be compressed
> TotalB  Total bytes transferred
> BSaved  Bytes saved through compression (only where successful)
> PSaved  Percentage of bytes saved to all traffic
> 
> Size    Number  Pct     PBytes  AvgComp PctComp TotalB  BSaved  PSaved
> 0       4991     37      11     118       0     255181  0        11
> 64      6503     49      22     113       0     501071  31       22
> 128     337       2       2     106       1     50481   21        2
> 192     106       0       1     104       1     22649   5         1
> 256     101       0       1      99      15     28552   1089      1
> 320     77        0       1     100      15     26930   521       1
> 384     72        0       1      91      55     29328   2921      1
> 448     32        0       0      98      18     15151   549       0
> 512     12        0       0      99      16     6516    143       0
> 576     8         0       0      75      62     4869    1217      0
> 640     23        0       0      81      52     15325   3005      0
> 704     11        0       0      97       9     8136    303       0
> 768     14        0       0      93      21     11310   857       0
> 832     11        0       0      98       9     9357    273       0
> 896     7         0       0      90      28     6532    695       0
> 960     6         0       0      90      16     5895    586       0
> 1024    14        0       0      98       7     14681   423       0
> 1088    10        0       0      91      20     11275   1028      0
> 1152    5         0       0      74      60     5922    1540      0
> 1216    14        0       0      82      42     17398   3157      0
> 1280    6         0       0      79      50     7867    1679      0
> 1344    10        0       0     100      10     13818   11        0
> 1408    819       6      52      57      90     1171427 504134   29
> 
> 
> 

-- 

Eric Wheeler
Network Administrator
KAICO
20417 SW 70th Ave.
Tualatin, OR 97062
www.kaico.com
Voice: 503.692.5268





<< | Thread Index | >> ]    [ << | Date Index | >> ]