zstandard support

Michael Stone via Argus-info argus-info at lists.andrew.cmu.edu
Mon Oct 24 14:12:13 EDT 2016


On Mon, Oct 24, 2016 at 01:46:55PM -0400, I wrote:
>The patch below adds zstandard support to argus (to allow transparent 
>use of zstd-compressed argus files). Why?
>
>(raw)	2702MB	9s
>bz2	851MB	260s
>gz	943MB	72s
>xz	608MB	160s
>zst-3	880MB	41s
>zst-10	834MB	41s
>zst-19	685MB	45s
>
>In my testing, zstd consistently produces files that are both faster 
>to read and smaller than gzip, without the performance penalty of bz2 
>or xz. The zstd-3 is the default compression option, and -10 and -19 
>reflect the current middle and maximum compression settings. It's 
>worth noting that the data in this testing was all cached, so the times do 
>not reflect disk I/O.

I'd meant to include 
https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/
for some context as to what zstandard is.

Mike Stone



More information about the argus mailing list