Alerting on certain traffic using Argus?

[Drew Dixon]

Hey Carter, apologies for the delay, finally getting back to this-
responses in-line:

On Fri, May 11, 2018 at 6:00 PM, Carter Bullard <carter at qosient.com> wrote:

> Hey Drew,
> So what do you mean by ‘traffic alerts’?  Are you thinking about SIEM
> style alarms / alerts ??  I’m not really an alarm or alert kind of cyber
> guy, so … but they are somewhat easy to do with the tools we offer ...
>

I was asking about implementing traffic alerts using the tools you
offer/wrote : ) we have so much flow data that dumping it into our SIEM
would be crazy expensive and not feasible...

>
> The argus -> radium combo I would assume to be a part of any real
> installation.  The purpose of radium is to provide multiple access to the
> argus stream, and I usually expect there to be multiple reasons to process
> independent copies of the stream, using filters, aggregators, labelers, etc
> to build up a network awareness task.   Racluster.1 is a way to minimize
> raw argus data.  For some its summarize data, for others its to generate
> differing views.  But racluster.1 by itself probably doesn’t provide enough
> semantics / analytics to be used to generate alerts other than tipping
> (looking for a specific IP address in the stream).  It also doesn’t provide
> you with enough control over time to generate traffic types of
> alarms/alerts, like for instantaneous peak bandwidth utilization data.  For
> that I generally turn to rabins.1, which is racluster.1 but confined to a
> prescribed time range (bin).
>

Yea, eventually I'd like us to deploy the argus server portion but that's
unfortunately out of scope for us for the time being.  So still sending
IPFIX  from routers to radium then processing with racluster (still not
seeing what I would expect the results to be after processing with
racluster so I need to ask you about that in another thread sometime soon
here, btw, not sure whats up) then archiving into .xz archives for best
compression.

>
> The design for tagging flows of interest for alarm / alerts is centered
> around ralabel.1, which tags / colors / labels flows based on a fall
> through criteria (filters).  This is the metadata support that enables the
> argus system to tag flows or aggregations of flows, with metadata that can
> be used to say, oh this is bad, or its too much or its too little. Its
> these labels that help to generate alerts. Is the total bandwidth of a flow
> greater than some threshold?  Is this flow a big elephant ??.  Use your
> radium -> ralabel to assign a “too high” or “big elephant” like label to
> the flows, and then use any ra* tool to look for the “too high” string or
> the “big elephant" in the label.  Is the instantaneous peak bandwidth over
> a 60 second period for a given video flow too low, use radium -> rabins.1
> to get the data into 60 second reports and use ralabel to label the flows
> with “too low”, if the rate or load drops below a threshold.  This will
> generate a stream of 60 second flow records which you can run through
> racluster.1 to consolidate it down to one flow record.   Because the labels
> are preserved during aggregation, if there was one 60 sec period where the
> video stream was “too low”, the “too low” label will be in the aggregate
> for the whole set of flow records.  If in your ralabel.1 conf file also
> labeled records with “jitter too high”, or “loss to high”, and there were
> 60s flows that hit those, the single flow record's label after the final
> raclusteer.1 would have all 3 strings included, but only one copy of each
> string.
>

I am somewhat interested in doing a few of the things you've mentioned
here, probably around the elephant flow identification potentially so this
sort of thing is cool to hear about being possible.

>
> You can build simple IDS like functionality by structuring sets of
> ralabel.1 configurations, since STIX/TAXI signatures are just snort like
> filters, which are easily translated to argus filters, or if you can write
> C, its easy to convert snort to argus, or to just use the snort filters
> directly.  You can also think of signatures as policy statements, so using
> tools like rapolicy.1 can be used to build fast tests for access control
> violations, or if your test is a set of IP watch lists, you can use
> rafilteraddr.1 to label those flows that have matches, which can be
> selected to generate an alarm.
>

THIS, yes this last paragraph is the type of thing I was inquiring about,
specifically at the moment I'm most interested in importing a list of IP
addresses probably using ralabel/rafilteraddr and labeling the flows that
have matches for hits on these IP addresses, then alerting on those label
matches.  However, what I'm not immediately seeing though, is how the alarm
generation logic and functional aspects of the alerting are typically
constructed in this type of setup? AKA, how do you recommend (or how have
others already implemented) the actual alerting piece on the labels to say
fire off an email alert etc.?

I'm trying to fully understand how I should properly place this into my
existing flow data processing pipeline (described above)...If I had to take
a guess at it off the top of my head maybe something like configuring
ralabel to pull in the addresses list using an ralable.conf file (I don't
see what option I should use to specify my address list in looking at the
man page for ralabel.conf, I do see mention of an address.spec file in the
man page for ralabel, can I do something like that in ralabel.conf?) then
attaching ralabel to radium (can ralabel run as a systemd service like
radium?) then what would you recommend to use to do the email alerting?

Maybe I'm off the mark here so don't want to guess too much on the best
approach here and will let the expert(s) weigh-in on this.  I'd certainly
like to discuss the IDS signature conversion to Argus filters sometime as
well but for now my priority is the alerting on specific IP lists.


> Are these the types of traffic alerts you’re interested in ???
>
> Carter
>

Many thanks,

-Drew

>
> On May 11, 2018, at 1:35 PM, Drew Dixon <dwdixon at umich.edu> wrote:
>
> Hi there,
>
> Somewhat of a general question I've been meaning to pose to the mailing
> list for a while but just now getting around to it...
>
> How many of you are using Argus/Radium/Racluster flow data to generate
> traffic alerts?
>
> If you are doing this, I would be extremely interested in hearing all the
> details of how you are doing this and how it's working for you.
>
> Many thanks,
>
> -Drew
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20180515/9b0709ad/attachment.html>