zstandard support
Michael Stone via Argus-info
argus-info at lists.andrew.cmu.edu
Mon Oct 24 14:12:13 EDT 2016
On Mon, Oct 24, 2016 at 01:46:55PM -0400, I wrote:
>The patch below adds zstandard support to argus (to allow transparent
>use of zstd-compressed argus files). Why?
>
>(raw) 2702MB 9s
>bz2 851MB 260s
>gz 943MB 72s
>xz 608MB 160s
>zst-3 880MB 41s
>zst-10 834MB 41s
>zst-19 685MB 45s
>
>In my testing, zstd consistently produces files that are both faster
>to read and smaller than gzip, without the performance penalty of bz2
>or xz. The zstd-3 is the default compression option, and -10 and -19
>reflect the current middle and maximum compression settings. It's
>worth noting that the data in this testing was all cached, so the times do
>not reflect disk I/O.
I'd meant to include
https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/
for some context as to what zstandard is.
Mike Stone
More information about the argus
mailing list