[ARGUS] Argus v5.0.0 and packet size distributions

[Carter Bullard]

Gentle persons,
We’re getting closer to release time, and I’m finalizing the features that will be included in v5.0.0.

In support of a bunch of applications, I’ve migrated support for packet size distribution reporting, on all flows.  This is in addition to reporting the max and min packet sizes seen.
This feature generates a 32-bit representation of the relative packet size distribution, where each nibble is a column representing a log2 scale of packet sizes starting at 40 (min packet size), and going to 4K and above tallied in the last column.

This is turned on in the argus.conf file, using ARGUS_PACKET_SIZE_HISTOGRAM=yes.

This has been used for traffic analysis, flow classification (big elephant, interactive, with control), transaction classification (full data push, jumbo packet use, poll, etc), anomaly detection and to add additional support for other behavioral measures, like keystroke detection.

argus-clients v5.0.0 print these out as a 32-bit hex value for both src and dst (spktsz, dpktsz), and aggregation (racluster) should combine the histograms correctly.
argus-clients v3.x has code support for working with these distributions, but argus v3.x didn’t generate the data ...

Working with histograms is an interesting statistical effort. distance, divergence, skew, chi-squared tests, this data can support many of these methods ... its not perfect … but … it doesn’t add more than 12-16 bytes of data per flow record, so it meets a lot of criteria for adding to argus records.

As more people want to use this data, hopefully the group will contribute support and ideas on how we can get the most out of packet size features.

Suggestions, opinions, reactions, comments, reactions, are all welcome, of course.

Hope all is most excellent,


Carter

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1385 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20240530/649f9e78/attachment.bin>