Some useful Argus scripts for detecting traffic anomalies in ranges of data

[Carter Bullard]

Hey Terry,
The direction aspects to this can be taken care of in a number
of ways, the most full-proof is to track the smac and dmac fields,
and to provide filters that deal with all the combinations.  The reason
this is a good strategy, is generally because the number of ethernet
addresses a give probe sees is small, so the list of filters is going
to be bounded, but, ...., it does mean that filters are very specific  
for
each installation, which makes maintenance etc... a bit of a chore.

The TCP direction logic is there so you can figure out what is the
service port, which in many analyses, is very important, but for
scan analysis, it is not such a big deal.  "someone sends a TCP
synack to port xxxxx, to see if they can get a RST back".  Well this
is not "backscatter", it is a synack probe, and so the TCP direction
actually gets in the way.

So, in my radark.pl script, I try to do similar things as you but  
without
worrying about direction.  My assumption is that if anything hits a
dark address, regardless of protocol state, then more than likely,
its a part of a scan.  By aggregating out the 'proto' you lose the TCP
direction modification data, and so you can get the addresses of
your scanners without having to deal with who is what.

So I use rafilteraddr() to give me flows that hit dark addresses,
and then pipe that through "racluster -M rmon -m smac saddr"
and then filter them for addresses that are not local:

    rafilteraddr -w - | racluster -M rmon -m smac saddr -w - | ra -  
src net not $localaddr

This will give me any address that hit a dark address that is not
one of mine.   But I do end up using filters like:
    (not src net $localnet and dst net $localnet)

In order to figure out did local address ever respond to this scanner,
which I think is the real question.  Who cares if someone is scanning,
the question is did they find anything, ..., new.

In support of your scripts, having a script that generates the darkhost
list or the inverse, the lithost list would be very helpful for some.   
Do
you have a script, or do we need to generate one?  My radark.pl
script does some of this, and we can carve out that piece to provide
a ralight() like program?

Is this a good idea?  How were/are you generating your IP address list?

Carter


On Feb 8, 2008, at 1:42 PM, Terry Burton wrote:

> Hi all.
>
> Attached are some scripts that use the Argus client tools with ranges
> of Argus data in order to create a ranked list of flows satisfying
> interesting traffic patterns. I have been successfully using wrappers
> for these to sample five minute intervals of captured data via
> (rastream or cron jobs) in order to generate alerts over XMPP whenever
> certain predefined thresholds are exceeded.
>
> They are provided in the hope that they may be useful and I welcome
> any feedback that you all may have.
>
>
> A couple of examples:
>
> * arg_scan_hit_darkhost *
>
> This ranks hosts based on the number of unregistered IP addresses that
> they have touched. (We carefully maintain a list of every registered
> network device and so we are aware of precisely which IP addresses
> ought to be dark).
>
> For example:
>
> $ arg_scan_hit_darkhost -r /srv/argus/flows/2008-02-08/0.0.0.0-11\: 
> 20\:05.arg.gz
>
>  1128 11:20:00.000000 11:25:00.000000 300.000000       69.61.230.22
>    28 11:20:03.049115 11:24:52.327789 289.278687      71.205.62.200
>    14 11:20:28.460944 11:24:04.000071 215.539124      85.214.70.203
>     8 11:23:05.943371 11:24:54.101835 108.158463      220.174.93.18
>     2 11:20:21.726881 11:23:58.291126 216.564240        24.64.4.221
>     2 11:20:08.378429 11:23:58.291376 229.912949       24.64.166.59
>     2 11:23:12.514529 11:23:58.277644  45.763115      24.64.177.114
>     2 11:20:28.685934 11:24:58.351166 269.665222      212.91.252.10
>     1 11:23:12.169240 11:23:12.170106   0.000866       12.8.175.253
>     1 11:21:01.065079 11:21:01.065079   0.000000         24.64.3.15
> ...
>
> The above example indicates that host 69.61.230.22 hit 1128 distinct
> unregistered addresses between 11:20-11:25 (a 300 second interval),
> i.e. it was actively fingerprinting our network.
>
> * arg_scan_by_port *
>
> This ranks hosts based on the number of /distinct/ hosts that they
> have contacted on the /same/ uncommon destination port (not http,
> smtp, etc.). This highlights traffic that is symptomatic of exploit
> searching, etc. [Aside: The corresponding script arg_scan_by_host
> ranks based on connections to /distinct/ ports on a /single/ host,
> i.e. targeted port scanning.]
>
> $ arg_scan_by_port -r
> /srv/argus/flows/2008-02-08/0.0.0.0-11\:20\:05.arg.gz - src net
> 123.123.0.0/16
>
>  2837 11:20:00.000000 11:25:00.000000 300.000000       123.123.230.22
>   tcp imap2
>    10 11:21:45.846137 11:24:52.652832 186.806702       
> 123.123.209.83    tcp 22
>     9 11:23:28.410986 11:24:13.083116  44.672131       
> 123.123.40.19    tcp 4662
>     5 11:24:24.327964 11:24:29.065500   4.737536      
> 123.123.79.231    tcp 1030
> ...
>
> In the above example I have limited the input to flows that originate
> from the local network (123.123.0.0/16) in order to detect possibly
> compromised local hosts or rogue users. [Aside: You can add similar
> filters to the input provided to any of my scripts since they pass all
> parameters directly to ra.] It is clear that 123.123.230.22 is
> compromised having attempted connections to 2837 different hosts on
> the tcp/imap2 port within a 300 second interval. Similarly
> 123.123.209.83 might be worth investigation as it may be slowly
> scanning, or possibly be a ordinary hyperactive user ;-).
>
> * arg_talkative_hosts - useful for spotting peer-to-peer activity
> * arg_unregistered_talkers - lit IP that should be ranked by distinct
> host connections
>
>
> A general improvement that still needs to be made: Since the direction
> of the flows is of key importance in several of the scripts I use a
> "(syn and not tcp)" filter in order to reduce the number of flows for
> which the direction is uncertain - the main culprits being the TCP
> connections that were ongoing at the start of the data range. This
> filter is not ideal because it excludes some data that would otherwise
> form a useful part of the results (and the filter solution does not
> help with UDP-based flows, etc.). I think that a better solution to
> this problem would be to "warm-up" Argus by reading an additional 15
> mins or so of data prior to the analysis window in order to prime the
> flow directions, and then to define the analysis window using the "-t
> <timerange>" ra option. It this the best solution to the directions
> problem?
>
> Finally, might I suggest that once the Argus 3.0 clients are released
> that there be an official repository of useful examples and scripts
> that people can contribute to and refine. Perhaps the wiki is the best
> starting point for this?
>
> Thanks in advance for anyone's feedback.
>
>
> Have fun,
>
> Tez
> < 
> arg_scan_by_host 
> > 
> < 
> arg_scan_by_port 
> > 
> <arg_scan_hit_darkhost><arg_talkative_hosts><arg_unregistered_talkers>