Compression and decompression shoot out

New benchmark!

I decided to do another test suite, this time including decompression. The data is a concatenation of two tar archives. The first tar contains 3+ GB of C:\Program Files from a Windows XP installation. The second tar is a 3+ GB Ubuntu installation, sans /usr which is on a separate partition. In total this amounts to 6.53 GB.

The test host is my girfriend's 3 GHz P4 with 1GB RAM and 2MB L2 cache. It is rated at ~ 6000 bogomips. The computer is running Ubuntu 8.04 with a custom 2.6.25.4 kernel. All files were on an fuse mounted ntfs partition.

In addition to processing time I tried to measure RAM usage with GNU time but I didn't get any meaningful results. I did manage to record page faults,and I may add those statistics later.

This time lha is out. It doesn't seem to like files this big. Since I was also going to benchmark decompression I decided to include lzo, which is allegedly very fast at decompression.

Complete list of the contestants:

• bzip2 1.0.4 (-9k)
  
• gzip 1.3.12 (-9c)
  
• lrzip 0.23 (-w 9 -q)
  
• LZMA SDK 4.43 (-9kq)
  
• LZO library 1.08 (-9 -k)
  
• RAR 3.71 (-m5)
  

gzip was invoked in redirect mode (-c) because I didn't want it to throw away the source file. This shouldn't really affect compression ratio or processing times.

Below are the results broken down into compressed size and ratio, compression time and decompression time.

Compressed size, from worst to best:

• uncompressed: 7,011,041,280 (6.53 GB), ratio 1.000
• lzo_________: 4,719,645,902 (4.40 GB), ratio 1.486
• gzip________: 4,563,292,811 (4.25 GB), ratio 1.536
• bzip2_______: 4,428,910,323 (4.12 GB), ratio 1.583
• rar_________: 4,125,923,141 (3.84 GB), ratio 1,699
• lzma________: 3,840,213,621 (3.58 GB), ratio 1,826
• lrzip_______: 3,585,069,056 (3.34 GB), ratio 1.955

Wall clock compression time, from slowest to fastest:

• lzma_: 8,409 s
• lrzip: 7,904 s
• rar__: 5,906 s
• lzo__: 3,487 s
• bzip2: 3,034 s
• gzip_: 1,598 s
• cat__: __111 s (for reference)
  

Wall clock decompression time, from slowest to fastest:

• lrzip: 2,830 s
• bzip2: 1,491 s
• lzma_: __981 s
• rar__: __604 s
• gzip_: __503 s
• lzo__: __449 s
• cat__: __111 s (for reference)

The shell script used to gather these statistics is here.

Compression shoot out

I had some time on my hands the other day and a 2.9/4 GiB debian lenny ext3 filesystem sitting rather idly on one of my hard drives (This fs serves as a backup OS in case my bleeding edge gentoo bellies up on me). This fs includes Xorg, Firefox, Xfce4, GNOME among other installed packages. Having just observed the coreutils distribution switch to lzma I was really curious about lzma's performance. A file system filled with linux applications and data was a good candidate for compression testing. I decided to round up gzip, bzip2, lha, zip, rar, lzma and lrzip to see what they were made of.

I copied the blocks to a file using dd, loop mounted the file and started copying from /dev/zero to a file on the loop mounted fs so as to prepare the unused blocks with redundant data. Eventually the dd command stopped with a file system full warning, so I synced the fs and deleted the zero stuffed file. I unmounted the looped back fs and started compressing...

Several CPU hours later I had an impressive amount of digits which I have decided to share with you through this blog. Here are the hard numbers:

Original file: __4,293,563,904 (3.999 GiB) (4096 MiB down to nearest cylinder boundary)
Zero file size: _1,150,640,128 (1.072 GiB)
Net fs size: ____3,142,923,776 (2.927 GiB) (This number is used as the uncompressed size)
fs usage per df: 3,065,671,680 (2.856 GiB)
ext3fs overhead: ___77,252,096 (2.458 %)

compressor _________size ______________ratio__________CPU usage
none_______3,142,923,776_(2.9271 GiB)_1.0000
gzip_-9____1,177,245,158_(1.0964 GiB)_2.6697______593s (sys 24s)
zip_-9_____1,177,245,231_(1.0964 GiB)_2.6697______554s (sys 25s)
lha_-o7____1,152,406,262_(1.0733 GiB)_2.7273____1,492s (sys 39s)
bzip2_-9___1,082,698,303_(1.0083 GiB)_2.9029____2,316s (sys 25s)
rar_-m5______942,002,518_(0.8773 GiB)_3.3364____3,481s (sys 30s)
lzma_-9______871,912,252_(0.8120 GiB)_3.6046___10,334s (sys 23s, wall 10,471s)
lrzip_-w9____849,062,862_(0.7908 GiB)_3.7016____4,483s (sys 35s, wall 7,000s)

(Sorry about the formatting...)

Wall clock times are not much different from user times except for lzma and lrzip. This is because these beasts use large windows which results in swapping.

Observations:

• gzip and Zip use the same algorithm, ie. PKZIP deflate. The file size difference is in headers. Zip compression is noticeably faster.
• lrzip spent 42 minutes (!) swapping. I need more RAM!
• There is an almost perfect correlation between CPU time spent and compression ratio achieved. The exception is that lrzip is a lot quicker than lzma (which lrzip uses, incidently) at highest settings.
• It would be interesting to measure RAM usage during compression, but I haven't found a simple way to do that.
• It would be interesting to benchmark decompression too. To be done.
  

Bench mark host and contestants information:

AMD Athlon XP 2400+ @ 2GHz,256 kB L2
1 GB DDR PC2700 @ 266 MHz
Linux 2.6.24

• gzip 1.3.12
• Zip 2.32
• LHa 1.14i-ac20050924p1
• bzip2 1.0.5
• RAR 3.71
• LZMA SDK 4.32.6
• lrzip 0.23
  

All built on host with gcc 4.2.3 using CFLAGS="-march=athlon-xp -O2 -pipe".