Help with queries

[Carter Bullard]

Hey Paul,
Incantations and explanations in-line.
Carter

On May 3, 2013, at 8:36 AM, Paul Halliday <paul.halliday at gmail.com> wrote:

> Hi,
> 
>  I have been banging around with ra and racluster and while I can get close to what I want I am still not very comfortable with the commands and not entirely sure I am using them correctly. Getting really close to greping, awking, sorting and uniq -c'ing  out the stuff I need which I doubt I need to do..
> 
> Anyway any help would be appreciated. I am trying to get:
> 
> The Top 20 Sources and then Destinations (sum bytes) matching 10.1.0.0/16 for the current day

Because you want to report on a single identifier, rather than the traditional flow identifiers,
you will need to convert the flow data to RMON style data, so that you can get metrics for individual
objects.

    racluster -M rmon -r argus.file(s).daily -m saddr -w argus.daily.ip.addrs.out - ip

This will generate a list of argus records where the saddr field holds the unique IP addresses
seen in the file.  The daddr field will hold an interesting aggregation of all the IP addresses
the saddr talked to.  Sometimes that is a single address, sometimes its a CIDR address,
depending on how many hosts the saddr communicated with.

Argus clients keep the RMON style data in flow records, in order to improve forensics and
data mining, but once you create an rmon style of flow record with racluster(), only the
' s ' (source) objects should be used, but all the metrics are valid.  In our specific example,
the ' s ' metrics are those generated by the 's'object, so for bytes and packets, they are
the ones transmitted (sent), and the ' d ' metrics are the ones for traffic received (recv'd)
by the 's'object.

So to get your Top 20 Sources, i.e. the IP addresses that " sent " the most bytes, you are
interested in working with the " sbytes " metric.  Just sort the ip address file, based on
sbytes, and then printout the first 20 entries that match the filter you're interested.

    rasort -m sbytes -r argus.daily.ip.addrs.out -No20 -s stime dur saddr spkts dpkts sbytes dbytes - net 10.1.0.0/16

The " -No20 " option indicates to number  records to printout after the sorting is done.  In this case
you want 20, so " -No20 ".

We use the " -s fields " option to printout the fields of interest.  I always think the time and duration
are important, but the real fields for you are " saddr sbytes ".  If you want the " rank ", add that 
field to the list, usually first.

To get the Top 20 Destinations, in bytes:

    rasort -m dbytes -r argus.daily.ip.addrs.out -No20 -s stime dur saddr spkts dpkts sbytes dbytes - net 10.1.0.0/16

> Count of distinct IP's matching 10.20.0.0/22 in the last hour

Again, because you want to report on a single identifier, rather than the traditional flow identifiers,
you will need to convert the flow data to RMON style data, so that you can get metrics for individual
objects.  You don't have to store that data in a file, you can pipe it into programs that will generate the
number you're interested in.  If you just want the number, each line of the racluster() output represents
a unique IP address, you will need to pick out the 10.20.0.0./22 addresses you're interested in.

The simplest solution, is to print the lines to something like wc.1, and have it do the counting.

    racluster -M rmon -m saddr -r argus.last.hour.file(s) -w - - ip | ra -L-1 - src net 10.20.0.0/22 | wc

In my network, you would get this:

thoth:tmp carter$ racluster -r argus.2013.05.02.*out -M rmon -m saddr -w - - ip | ra -L-1 - src net 192.168.0.0/24 | wc
      13     156    2119

The "-L-1" suppresses printing of the column labels, which you don't want to count.

> 
> Lastly, I am trying to produce a rate chart with ragraph that portrays inbound/outbound, something like this:
> 
> http://www.pintumbler.org/ex.png

   ragraph sbytes dbytes srcid -M time 5m -r argus.file.to.graph



If you are using argus-clients-3.0.7.5 or higher, be sure to set this variable to off in your .rarc files.
This is a temporary requirement.

RA_GENERATE_BIN_MAR_RECORDS=no

Send email if you have any problems.

Carter

> 
> possible?
>  
> Thanks!
> 
> -- 
> Paul Halliday
> http://www.pintumbler.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130503/fcfce162/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ragraph.png
Type: image/png
Size: 21973 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130503/fcfce162/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4399 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130503/fcfce162/attachment.bin>