Some useful Argus scripts for detecting traffic anomalies in ranges of data

[Michael Hornung]

Hi Terry, I really appreciate you sharing your work with the group.  I 
was wondering how the *heck* you were going to get what you wanted with 
"syn and not tcp" but then I looked in your scripts and saw that you 
really meant to say "syn OR not tcp".  :)

These user-contributed scripts are helpful to me, and I'm not even a 
total Argus newbie.  Keep 'em coming!

-Mike


Terry Burton wrote:
> Hi all.
> 
> Attached are some scripts that use the Argus client tools with ranges
> of Argus data in order to create a ranked list of flows satisfying
> interesting traffic patterns. I have been successfully using wrappers
> for these to sample five minute intervals of captured data via
> (rastream or cron jobs) in order to generate alerts over XMPP whenever
> certain predefined thresholds are exceeded.
> 
> They are provided in the hope that they may be useful and I welcome
> any feedback that you all may have.
> 
> 
> A couple of examples:
> 
> * arg_scan_hit_darkhost *
> 
> This ranks hosts based on the number of unregistered IP addresses that
> they have touched. (We carefully maintain a list of every registered
> network device and so we are aware of precisely which IP addresses
> ought to be dark).
> 
> For example:
> 
> $ arg_scan_hit_darkhost -r /srv/argus/flows/2008-02-08/0.0.0.0-11\:20\:05.arg.gz
> 
>   1128 11:20:00.000000 11:25:00.000000 300.000000       69.61.230.22
>     28 11:20:03.049115 11:24:52.327789 289.278687      71.205.62.200
>     14 11:20:28.460944 11:24:04.000071 215.539124      85.214.70.203
>      8 11:23:05.943371 11:24:54.101835 108.158463      220.174.93.18
>      2 11:20:21.726881 11:23:58.291126 216.564240        24.64.4.221
>      2 11:20:08.378429 11:23:58.291376 229.912949       24.64.166.59
>      2 11:23:12.514529 11:23:58.277644  45.763115      24.64.177.114
>      2 11:20:28.685934 11:24:58.351166 269.665222      212.91.252.10
>      1 11:23:12.169240 11:23:12.170106   0.000866       12.8.175.253
>      1 11:21:01.065079 11:21:01.065079   0.000000         24.64.3.15
> ...
> 
> The above example indicates that host 69.61.230.22 hit 1128 distinct
> unregistered addresses between 11:20-11:25 (a 300 second interval),
> i.e. it was actively fingerprinting our network.
> 
> * arg_scan_by_port *
> 
> This ranks hosts based on the number of /distinct/ hosts that they
> have contacted on the /same/ uncommon destination port (not http,
> smtp, etc.). This highlights traffic that is symptomatic of exploit
> searching, etc. [Aside: The corresponding script arg_scan_by_host
> ranks based on connections to /distinct/ ports on a /single/ host,
> i.e. targeted port scanning.]
> 
> $ arg_scan_by_port -r
> /srv/argus/flows/2008-02-08/0.0.0.0-11\:20\:05.arg.gz - src net
> 123.123.0.0/16
> 
>   2837 11:20:00.000000 11:25:00.000000 300.000000       123.123.230.22
>    tcp imap2
>     10 11:21:45.846137 11:24:52.652832 186.806702      123.123.209.83    tcp 22
>      9 11:23:28.410986 11:24:13.083116  44.672131      123.123.40.19    tcp 4662
>      5 11:24:24.327964 11:24:29.065500   4.737536     123.123.79.231    tcp 1030
> ...
> 
> In the above example I have limited the input to flows that originate
> from the local network (123.123.0.0/16) in order to detect possibly
> compromised local hosts or rogue users. [Aside: You can add similar
> filters to the input provided to any of my scripts since they pass all
> parameters directly to ra.] It is clear that 123.123.230.22 is
> compromised having attempted connections to 2837 different hosts on
> the tcp/imap2 port within a 300 second interval. Similarly
> 123.123.209.83 might be worth investigation as it may be slowly
> scanning, or possibly be a ordinary hyperactive user ;-).
> 
> * arg_talkative_hosts - useful for spotting peer-to-peer activity
> * arg_unregistered_talkers - lit IP that should be ranked by distinct
> host connections
> 
> 
> A general improvement that still needs to be made: Since the direction
> of the flows is of key importance in several of the scripts I use a
> "(syn and not tcp)" filter in order to reduce the number of flows for
> which the direction is uncertain - the main culprits being the TCP
> connections that were ongoing at the start of the data range. This
> filter is not ideal because it excludes some data that would otherwise
> form a useful part of the results (and the filter solution does not
> help with UDP-based flows, etc.). I think that a better solution to
> this problem would be to "warm-up" Argus by reading an additional 15
> mins or so of data prior to the analysis window in order to prime the
> flow directions, and then to define the analysis window using the "-t
> <timerange>" ra option. It this the best solution to the directions
> problem?
> 
> Finally, might I suggest that once the Argus 3.0 clients are released
> that there be an official repository of useful examples and scripts
> that people can contribute to and refine. Perhaps the wiki is the best
> starting point for this?
> 
> Thanks in advance for anyone's feedback.
> 
> 
> Have fun,
> 
> Tez