rabins and racluster.conf

[Mark E. Mallett]

On Mon, Apr 30, 2012 at 09:38:53AM -0400, Carter Bullard wrote:
> Hey Mark,
> Sorry for the delayed response !!!!! 

Not a problem.


> Please give these a test, and if all is well, I'll put out the first round of argus-client-3.0.6 fixes !!!
> I've included a new copy of rabins.c if you aren't comfortable with patch files.

Looks better but still not completely joyful. I applied your patches (and also
did a diff against your rabins.c to make sure that one was ok) and did a
complete remake and reinstall.

It now does the aggregation for the first few lines in my c3.conf, which
it did not do before, but still does not for the final "catch-all"
filter line.

Taking your insight that racluster should do the same thing as rabins with
one bin, here's what I find with your argus.simple.data.out file

$ racluster -n -f c3.conf -r argus.simple.data.out -s stime proto bytes dport
         StartTime  Proto   TotBytes  Dport 
   17:48:36.592155    tcp      50823 80
   17:48:36.589949    udp        439 0
   17:48:36.589413    arp        680

$ rabins -n -f c3.conf -r argus.simple.data.out -M time 1d -s stime proto bytes dport
         StartTime  Proto   TotBytes  Dport 
   17:48:36.592155    tcp      50823 80
   17:48:36.589949    udp        439 0

If you don't get the same results then maybe I've done something wrong
here.

This is a little clearer perhaps, using one of my own files with data
just within one day:

$ racluster -n -f c3.conf -r infile -s stime proto bytes dport
         StartTime  Proto   TotBytes  Dport 
   00:20:53.200000    tcp   70126187 80
   00:21:44.447000    tcp   71531614 443
   00:26:50.436603    tcp    3169863 22
   00:20:28.010000    udp   12408866 0
   00:20:53.221000    tcp  168594597 0
   00:25:22.955191    arp     120540
   00:33:44.486000   icmp     200541 0x0000
   00:57:58.110098    llc        300 0
   08:36:56.807000   igmp         28

$ rabins -n -f c3.conf -r infile -M time 1d -s stime proto bytes dport
         StartTime  Proto   TotBytes  Dport 
   00:20:53.200000    tcp   70126187 80
   00:21:44.447000    tcp   71531614 443
   00:26:50.436603    tcp    3169863 22
   00:20:28.010000    udp   12408866 0
   00:20:53.221000    tcp  168594597 0

You can see that the catch-all filter rule is not getting anything with
rabins, whereas it is with racluster.  Just for completeness, the c3.conf
file is:

$ cat c3.conf
# racluster.conf file, testing using this with rabins to
#  get some specific usage breakdowns.  Aggregation by proto and dport
#  may let us get protocol-based usage for some protocols.
RACLUSTER_PRESERVE_FIELDS=no
#
filter="tcp and dst port eq 80" model="proto dport"
filter="tcp and dst port eq 443" model="proto dport"
filter="tcp and dst port eq 22" model="proto dport"
filter="tcp or udp" model="proto"
filter="" model="proto"

As an aside: in the commands above I am displaying enough (proto and
dport) to try to show which filter rules matched. Is there a way to add
a tag of some sort, that would make it easier to extract aggregated data
for that rule later? (Perhaps even create a tag and aggregate around the
tag value?)

-mm-