rabins bug?

[Carter Bullard]

Hey Jesper Skou Jensen,

Looks OK to me, and your experience may not be bugs.

If we look at the various calls, your rabins.1 call seems to do this:

> rabins -r /arguslogs/2012-02/14/*.gz -t c2012/02/14 -m proto dport -M hard time 30s -w 2012-02-14.rabins - 'net 10.0.0.0/24 and ((tcp and syn and synack) or udp)'

    For TCP flows that we know the direction, and all UDP flows
         that start and stop within a given day (-t c2012/02/14)
     1. aggregate within each 30 second bin using the proto
         and the dst port as the flow keys.

This should give you for every 30sec period in a day (there should no more than
2880 of these) flows that represent the active dst ports by protocol.

> racluster -r 2012-02-14.rabins -w- - not frag |rasort -m pkts -w-|ra -N 100 -s dport -n -L -1 > top100ports.txt

You take that output, and you aggregate everything but frag flows, using the default
5-tuple flow model, which is valid, and then sort the records by packet count,
and then use ra.1 to print the first 100 records that come out, printing out the dst port value.

Seems OK to me, however, you're seeing multiple lines with repeating port values.
You should print out the protocol as well, as you are tracking "proto dport".
You could get as possible outputs, something like "tcp 53" and "udp 53", but you're
only printing the port value.   That may explain the repeats?

The second issue:
   
> racluster -r 2012-02-14.rabins -n - not frag and src port not 0


You're thinking that this should return no flow records because rabins.1 should
have set all the src port values to 0 ?    

When you aggregate with a subset of the 5-tuple aggregation key, by default, the
non-key fields are not zero'd out.  They get modified as aggregation occurs when the
"non-key" fields don't agree, but maybe not zero'd out.  The is called field preservation.

Field preservation, is usually "if the values are not the same, zero the value ou"
but for some fields, it is pretty cleaver.  For IP addresses, as an example, we
preserve the ' longest prefix match '.  

We do this to get some additional semantics out of aggregation, maybe for free,
so to speak.  The address preservation is designed to provide this info:

   "all of these aggregated records come from the same subnet"

The standard port preservation algorithm is designed to provide this info:
   
   "all of these aggregated records used the same source port"

If you want the non-key fields to be zero'd out, use the " -f racluster.conf "
option and the configuration should have this line in it.

RACLUSTER_PRESERVE_FIELDS=no

If this is important, I'll could add a "-M nopreserve" to the command line so you can
do this without a configuration file.

Without this directive, the value of a non-key attribute can definitely not be zero.

Hope this helps, if anything comes up, don't hesitate to email !!!!!!

Carter



On Feb 15, 2012, at 8:11 AM, Jesper Skou Jensen wrote:

> Hi guys,
> 
> I'm playing a bit with rabins and ragraph and I've run into a small problem, maybe a bug?
> 
> I'm using argus clients Version 3.0.5.30 (also tried 3.0.5.14 with the same result).
> 
> The source Argus logfiles are 24 hours log, split into 15 minute .gz files.
> 
> Here's the problem:
> 
> ***
> rabins -r /arguslogs/2012-02/14/*.gz -t c2012/02/14 -m proto dport -M hard time 30s -w 2012-02-14.rabins - 'net 10.0.0.0/24 and ((tcp and syn and synack) or udp)'
> 
> racluster -r 2012-02-14.rabins -w- - not frag |rasort -m pkts -w-|ra -N 100 -s dport -n -L -1 > top100ports.txt
> 
> ragraph bytes dport -M time 30s -r 2012-02-14.rabins -w 2012-02-14.rabins.rabins.top100ports_bytes.graph.png - port \(`sed -e "2,\\$s/^/or /" top100ports.txt` \)
> ***
> 
> Depending on the timeperiod of the soruce argus files, I sometimes get repeating lines in the top100ports.txt file, and I've investigated that a bit further.
> 
> racluster -r 2012-02-14.rabins -n - not frag and src port not 0
> 
> This actually produces a handfull sessions/flows. This seems like a bug? Shouldn't the "rabins ... dport" prevent exactly this?
> 
> 
> -- 
> 
> Regards
> Jesper Skou Jensen

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120215/300bbf49/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120215/300bbf49/attachment.bin>