Correlation rules

[Carter Bullard]

Hey Matt,
I have built slow scan detectors with argus data that Bro or Snort could never detect.
I think that justifies using Argus over Bro or Snort for any type of scan detection.
Why would you defer to something that has documented flaws ???

Carter


On Jan 3, 2014, at 1:44 PM, Matt Brown <matthewbrown at gmail.com> wrote:

> Hello Jaime,
> 
> This is a response in reference to: http://thread.gmane.org/gmane.network.argus/10094  (How did I miss that thread?!)
> 
> Beyond what Carter said, I wanted to respond with my opinion and some ideas.  Not sure of their value, but wanted to share.
> 
> 
> A few argus clients are interesting for this sort of thing from what I can see: ralabel, radark, raports and routers, raservices/rauserdata, rarpwatch (http://nsmwiki.org/Argus#List_of_clients).  Take a look at their source and see if you can make some new clients.
> 
> 
> In general, I am also excited about the potential to datamine argus data to find outliers, etc.
> 
> It appears that your list can be classified into three categories (enrichment of data, indicator of security issue, which may be an outlier/anomaly):
> 
> 
> Connection to reputation IP, domain, url: ENRICHMENT
> 
> Connection to unprivileged ports: INDICATOR
> Activation of privileged ports in our network: INDICATOR
> IP sweep: INDICATOR
> Port scan: INDICATOR
> Change in ip profile, from consumer to producer or the other way around: INDICATOR
> Direct connection to outside dns, Web, mail when existing inside: INDICATOR
> Change in gateway IP: INDICATOR
> Botnet C&C: INDICATOR
> Worm propagation: INDICATOR
> 
> Link saturation / loss: OUTLIER/ANOMALY
> Outlier detection in time series: OUTLIER/ANOMALY
> Excess of SIN packets: OUTLIER/ANOMALY
> Surge in error packets: OUTLIER/ANOMALY
> Changes of entropy in certain variables both globally and per IP: OUTLIER/ANOMALY... needs more static definition, this is like a gigantic iceberg
> Weird usage of light protocols like dns, icmp or voip: OUTLIER/ANOMALY
> Change in usual traffic ratios like syn to syn/ack or tcp/total, successful sessions: OUTLIER/ANOMALY
> Abuse of dns and authentication protocols (LDAP,...): OUTLIER/ANOMALY, INDICATOR
> 
> 
> The "Port Scan" indicator can be derived from a correlation rule of... a single (many?) IP(s), hitting a bunch of ports on a given IP.
> 
> For this specific example, it's difficult to argue that spending time mining argus data is the best way to invest effort to get this result (my opinion), particularly if there are other systems that are already written to handle such tracking and events (consider the exact things that need to occur to track a Port Scan).
> 
> Meaning, instead, consider what I'm beginning to shift to:
> -http://mbrownnyc.wordpress.com/2013/12/24/a-quick-post-about-logs/
> -http://mbrownnyc.wordpress.com/2013/12/31/a-more-effective-monitoring-architecture/
> 
> This will lay the groundwork for an IDS or IPS event tracker (not a "SIEM" necessarily, since I think that also requires a full issue life-cycle architecture):
> Basically: feed me data, I check that data against some regexes, match = action.
> 
> 
> Consider processing output of an argus client as strings (don't use '-w -'), and leveraging a correlation engine (such as SEC http://simple-evcorr.sourceforge.net/) for pattern matching.
> With SEC, you can (probably) write a definition that indicates a "Port Scan" event.  Feed this INDICATOR into your IDS system.
> 
> But, why expend time with any of this when this INDICATOR can already be detected with Bro, Snort... etc... Then fed into your IDS event tracker?
> 
> 
> I think time would be better spent with some simple stats analysis to find outlying data.
> Here's some stuff I started in on:
> -http://mbrownnyc.wordpress.com/technology-solutions/anomaly-detection-in-argus-data/part-1-introduction/
> -http://mbrownnyc.wordpress.com/technology-solutions/anomaly-detection-in-argus-data/part-2-locating-outliers-using-an-empirical-method-in-python-with-scipys-mquantile/
> -http://mbrownnyc.wordpress.com/technology-solutions/anomaly-detection-in-argus-data/part-3-qualifying-data-as-anomalous/
> 
> Let me know what you think.
> 
> 
> Will you be integrating this into the eneo redBorder IDS?  I'm not sure about the argus usage license, but Carter can speak to this.
> 
> 
> Thanks,
> 
> Matt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140103/a3ade04f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140103/a3ade04f/attachment.bin>