zstandard support

Michael Stone via Argus-info argus-info at lists.andrew.cmu.edu
Mon Oct 24 13:46:55 EDT 2016


The patch below adds zstandard support to argus (to allow transparent 
use of zstd-compressed argus files). Why?

(raw)	2702MB	9s
bz2	851MB	260s
gz	943MB	72s
xz	608MB	160s
zst-3	880MB	41s
zst-10	834MB	41s
zst-19	685MB	45s

In my testing, zstd consistently produces files that are both faster to 
read and smaller than gzip, without the performance penalty of bz2 or 
xz. The zstd-3 is the default compression option, and -10 and -19 
reflect the current middle and maximum compression settings. It's worth
noting that the data in this testing was all cached, so the times do not
reflect disk I/O.

Mike Stone


--- argus-clients-3.0.8.2.orig/common/argus_util.c
+++ argus-clients-3.0.8.2/common/argus_util.c
@@ -24305,6 +24305,7 @@ ArgusReadConnection (struct ArgusParserS
                      if (((ptr[0] == 0x1F) && ((ptr[1] == 0x8B) || (ptr[1] == 0x9D))) ||
                          ((ptr[0] == 0xFD) &&  (ptr[1] == 0x37) && (ptr[2] == 0x7A) && (ptr[3] == 0x58) &&  (ptr[4] == 0x5A) && (ptr[5] == 0x00)) ||
                          ((ptr[0] == 0xFD) && ((ptr[1] == 0x37) || (ptr[2] == 0x7A))) ||
+                         ((ptr[0] == 0x28) && (ptr[1] == 0xB5) && (ptr[2] == 0x2F) && (ptr[3] == 0xFD)) ||
                          ((ptr[0] == 'B') && (ptr[1] == 'Z') && (ptr[2] == 'h'))) {
                         char cmd[256];
                         bzero(cmd, 256);
@@ -24316,10 +24317,13 @@ ArgusReadConnection (struct ArgusParserS
                            strncpy(cmd, "bzip2 -dc ", 11);
                         else
                         if (ptr[1] == 0x8B)
-                           strncpy(cmd, "gzip -dc ", 11);
+                           strncpy(cmd, "gzip -dc ", 10);
                         else
                         if ((ptr[0] == 0xFD) && (ptr[1] == 0x37) && (ptr[2] == 0x7A) && (ptr[3] == 0x58) &&  (ptr[4] == 0x5A) && (ptr[5] == 0x00))
                            strncpy(cmd, "xzcat ", 7);
+                        else
+                        if (ptr[0] == 0x28)
+                           strncpy(cmd, "zstd -dc ", 10);
                         else
                            strncpy(cmd, "zcat ", 6);





More information about the argus mailing list