rabins and racluster.conf

[Mark E. Mallett]

On Mon, Apr 30, 2012 at 03:50:27PM -0400, Carter Bullard wrote:
> Hey Mark,
> May not be a complete solution still, but give this additional patch a try.
> This moves the variable declarations for filter matching into the nested
> aggregator loop.

That does seem to fix it for this test.  Thanks!

mm


> 
> ==== //depot/argus/clients/clients/rabins.c#71 - /Volumes/Users/carter/argus/clients/clients/rabins.c ====
> 996d995
> <    int retn = 0, fretn = -1, lretn = -1;
> 1001a1001
> >       int retn = 0, fretn = -1, lretn = -1;
> 
> 
> I get this kind of result from argus.simple.data.out and your configuration file.
> 
> thoth:clients carter$ ../bin/rabins -r ~/argus/data/argus.simple*out -f /tmp/racluster.conf \
>       -M time 1d -s stime dur proto dport spkts dpkts sbytes dbytes state
> 
>                  StartTime        Dur  Proto  Dport  SrcPkts  DstPkts     SrcBytes     DstBytes State 
> 2012/02/13.17:48:36.592155  27.203648    tcp http         38       47         5922        44901   FIN
> 2012/02/13.17:48:36.589949   0.000608    udp *             2        2          148          291   CON
> 2012/02/13.17:48:36.589413 213.552917    arp               8        4          424          256   CON
> 
> Carter 
> 
> 
> 
> 
> On Apr 30, 2012, at 2:50 PM, Mark E. Mallett wrote:
> 
> > On Mon, Apr 30, 2012 at 09:38:53AM -0400, Carter Bullard wrote:
> >> Hey Mark,
> >> Sorry for the delayed response !!!!! 
> > 
> > Not a problem.
> > 
> > 
> >> Please give these a test, and if all is well, I'll put out the first round of argus-client-3.0.6 fixes !!!
> >> I've included a new copy of rabins.c if you aren't comfortable with patch files.
> > 
> > Looks better but still not completely joyful. I applied your patches (and also
> > did a diff against your rabins.c to make sure that one was ok) and did a
> > complete remake and reinstall.
> > 
> > It now does the aggregation for the first few lines in my c3.conf, which
> > it did not do before, but still does not for the final "catch-all"
> > filter line.
> > 
> > Taking your insight that racluster should do the same thing as rabins with
> > one bin, here's what I find with your argus.simple.data.out file
> > 
> > $ racluster -n -f c3.conf -r argus.simple.data.out -s stime proto bytes dport
> >         StartTime  Proto   TotBytes  Dport 
> >   17:48:36.592155    tcp      50823 80
> >   17:48:36.589949    udp        439 0
> >   17:48:36.589413    arp        680
> > 
> > $ rabins -n -f c3.conf -r argus.simple.data.out -M time 1d -s stime proto bytes dport
> >         StartTime  Proto   TotBytes  Dport 
> >   17:48:36.592155    tcp      50823 80
> >   17:48:36.589949    udp        439 0
> > 
> > If you don't get the same results then maybe I've done something wrong
> > here.
> > 
> > This is a little clearer perhaps, using one of my own files with data
> > just within one day:
> > 
> > $ racluster -n -f c3.conf -r infile -s stime proto bytes dport
> >         StartTime  Proto   TotBytes  Dport 
> >   00:20:53.200000    tcp   70126187 80
> >   00:21:44.447000    tcp   71531614 443
> >   00:26:50.436603    tcp    3169863 22
> >   00:20:28.010000    udp   12408866 0
> >   00:20:53.221000    tcp  168594597 0
> >   00:25:22.955191    arp     120540
> >   00:33:44.486000   icmp     200541 0x0000
> >   00:57:58.110098    llc        300 0
> >   08:36:56.807000   igmp         28
> > 
> > $ rabins -n -f c3.conf -r infile -M time 1d -s stime proto bytes dport
> >         StartTime  Proto   TotBytes  Dport 
> >   00:20:53.200000    tcp   70126187 80
> >   00:21:44.447000    tcp   71531614 443
> >   00:26:50.436603    tcp    3169863 22
> >   00:20:28.010000    udp   12408866 0
> >   00:20:53.221000    tcp  168594597 0
> > 
> > You can see that the catch-all filter rule is not getting anything with
> > rabins, whereas it is with racluster.  Just for completeness, the c3.conf
> > file is:
> > 
> > $ cat c3.conf
> > # racluster.conf file, testing using this with rabins to
> > #  get some specific usage breakdowns.  Aggregation by proto and dport
> > #  may let us get protocol-based usage for some protocols.
> > RACLUSTER_PRESERVE_FIELDS=no
> > #
> > filter="tcp and dst port eq 80" model="proto dport"
> > filter="tcp and dst port eq 443" model="proto dport"
> > filter="tcp and dst port eq 22" model="proto dport"
> > filter="tcp or udp" model="proto"
> > filter="" model="proto"
> > 
> > As an aside: in the commands above I am displaying enough (proto and
> > dport) to try to show which filter rules matched. Is there a way to add
> > a tag of some sort, that would make it easier to extract aggregated data
> > for that rule later? (Perhaps even create a tag and aggregate around the
> > tag value?)
> > 
> > -mm-
> 



-- 
Mark E. Mallett