Correlation rules

[Matt Brown]

Hello Jaime,

This is a response in reference to:
http://thread.gmane.org/gmane.network.argus/10094  (How did I miss that
thread?!)

Beyond what Carter said, I wanted to respond with my opinion and some
ideas.  Not sure of their value, but wanted to share.


A few argus clients are interesting for this sort of thing from what I can
see: ralabel, radark, raports and routers, raservices/rauserdata, rarpwatch
(http://nsmwiki.org/Argus#List_of_clients).  Take a look at their source
and see if you can make some new clients.


In general, I am also excited about the potential to datamine argus data to
find outliers, etc.

It appears that your list can be classified into three categories
(enrichment of data, indicator of security issue, which may be an
outlier/anomaly):


Connection to reputation IP, domain, url: ENRICHMENT

Connection to unprivileged ports: INDICATOR
Activation of privileged ports in our network: INDICATOR
IP sweep: INDICATOR
Port scan: INDICATOR
Change in ip profile, from consumer to producer or the other way around:
INDICATOR
Direct connection to outside dns, Web, mail when existing inside: INDICATOR
Change in gateway IP: INDICATOR
Botnet C&C: INDICATOR
Worm propagation: INDICATOR

Link saturation / loss: OUTLIER/ANOMALY
Outlier detection in time series: OUTLIER/ANOMALY
Excess of SIN packets: OUTLIER/ANOMALY
Surge in error packets: OUTLIER/ANOMALY
Changes of entropy in certain variables both globally and per IP:
OUTLIER/ANOMALY... needs more static definition, this is like a gigantic
iceberg
Weird usage of light protocols like dns, icmp or voip: OUTLIER/ANOMALY
Change in usual traffic ratios like syn to syn/ack or tcp/total, successful
sessions: OUTLIER/ANOMALY
Abuse of dns and authentication protocols (LDAP,...): OUTLIER/ANOMALY,
INDICATOR


The "Port Scan" indicator can be derived from a correlation rule of... a
single (many?) IP(s), hitting a bunch of ports on a given IP.

For this specific example, it's difficult to argue that spending time
mining argus data is the best way to invest effort to get this result (my
opinion), particularly if there are other systems that are already written
to handle such tracking and events (consider the exact things that need to
occur to track a Port Scan).

Meaning, instead, consider what I'm beginning to shift to:
-http://mbrownnyc.wordpress.com/2013/12/24/a-quick-post-about-logs/
-
http://mbrownnyc.wordpress.com/2013/12/31/a-more-effective-monitoring-architecture/

This will lay the groundwork for an IDS or IPS event tracker (not a "SIEM"
necessarily, since I think that also requires a full issue life-cycle
architecture):
Basically: feed me data, I check that data against some regexes, match =
action.


Consider processing output of an argus client as strings (don't use '-w
-'), and leveraging a correlation engine (such as SEC
http://simple-evcorr.sourceforge.net/) for pattern matching.
With SEC, you can (probably) write a definition that indicates a "Port
Scan" event.  Feed this INDICATOR into your IDS system.

But, why expend time with any of this when this INDICATOR can already be
detected with Bro, Snort... etc... Then fed into your IDS event tracker?


I think time would be better spent with some simple stats analysis to find
outlying data.
Here's some stuff I started in on:
-
http://mbrownnyc.wordpress.com/technology-solutions/anomaly-detection-in-argus-data/part-1-introduction/
-
http://mbrownnyc.wordpress.com/technology-solutions/anomaly-detection-in-argus-data/part-2-locating-outliers-using-an-empirical-method-in-python-with-scipys-mquantile/
-
http://mbrownnyc.wordpress.com/technology-solutions/anomaly-detection-in-argus-data/part-3-qualifying-data-as-anomalous/

Let me know what you think.


Will you be integrating this into the eneo redBorder IDS?  I'm not sure
about the argus usage license, but Carter can speak to this.


Thanks,

Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140103/d97cfaf1/attachment.html>