Duration sum bug

Mon Mar 21 13:50:45 EDT 2011

Reran as requested.  These do not generate consistent results.
Unfortunately, they generate the same results I previously posted.

On Mon, Mar 21, 2011 at 1:42 PM, Carter Bullard <carter at qosient.com> wrote:
> So, the "-m srcid" removes all the flow identifiers, so there won't be any IP addresses, or ports or protos.
> Well, these results are not consistent.   Did you apply the same filter, that you did in the first run?
>
> Try this:
>   ra -r files -w /tmp/ra.out - host 1.2.3.4
>
>   racluster -u -r /tmp/ra.out -s stime ltime dur mean trans saddr dir daddr spkt dpkts
>   racluster -u -m srcid -r /tmp/ra.out -s stime ltime dur mean trans saddr dir daddr spkt dpkts
>
> Do these generate consistent results?
> Carter
>
>
> On Mar 21, 2011, at 1:33 PM, Digital Ninja wrote:
>
>> With -m srcid I'm back to the 72k seconds and no IPs listed:
>> 20:57:18.761336 18:57:33.788103 79215.0234 10 0.015230 0.0.0.0 <-> 0.0.0.0 10 10
>>
>> On Mon, Mar 21, 2011 at 1:29 PM, Carter Bullard <carter at qosient.com> wrote:
>>> And because you are interested in comparing the times, you may want to
>>> use the "-u" option, so that the time is printed in seconds.  Makes it easier
>>> for comparisons.
>>> Carter
>>>
>>>
>>> On Mar 21, 2011, at 1:26 PM, Digital Ninja wrote:
>>>
>>>> racluster -r <files> -s stime ltime dur trans mean saddr dir daddr
>>>> spkts dpkts - host 1.2.3.4 produces:
>>>>
>>>> 03:57:23.529664    03:57:23.544301   0.014637      1   0.014637
>>>> 5.6.7.5   <->            1.2.3.4        1        1
>>>> 09:57:27.624699 09:57:27.639307   0.014608      1   0.014608
>>>> 5.6.7.5   <->            1.2.3.4        1        1
>>>> 12:57:29.660667    12:57:29.676479   0.015812      1   0.015812
>>>> 5.6.7.5   <->            1.2.3.4        1        1
>>>> 13:57:30.339190 13:57:30.354246   0.015056      1   0.015056
>>>> 5.6.7.5   <->            1.2.3.4        1        1
>>>> 14:57:31.030849 14:57:31.046388   0.015539      1   0.015539
>>>> 5.6.7.5   <->            1.2.3.4        1        1
>>>> 16:57:32.385680 16:57:32.400769   0.015089      1   0.015089
>>>> 5.6.7.5   <->            1.2.3.4        1        1
>>>> 18:57:33.772816 18:57:33.788103   0.015287      1   0.015287
>>>> 5.6.7.5   <->            1.2.3.4        1        1
>>>> 20:57:18.761336    20:57:18.776750   0.015414      1   0.015414
>>>> 5.6.7.5   <->            1.2.3.4        1        1
>>>> 20:57:18.793793    20:57:18.808984   0.015191      1   0.015191
>>>> 5.6.7.6   <->            1.2.3.4        1        1
>>>> 23:57:20.806478    23:57:20.822145   0.015667      1   0.015667
>>>> 5.6.7.5   <->            1.2.3.4        1        1
>>>>
>>>> On Mon, Mar 21, 2011 at 1:19 PM, Carter Bullard <carter at qosient.com> wrote:
>>>>> Hmmm, it doesn't look like you have any bugs getting in your way,
>>>>> so you should be able to do what you want.  So what does this generate?
>>>>>
>>>>>   racluster -r files -s stime ltime dur trans mean saddr dir daddr spkts dpkts  host 1.2.3.4
>>>>>
>>>>> Do these numbers look reasonable?
>>>>> Carter
>>>>>
>>>>> On Mar 21, 2011, at 12:56 PM, Digital Ninja wrote:
>>>>>
>>>>>> Ok, so going back to what Rafael said about the 5-tuple aggregation...
>>>>>> When I run ra against the files with the following flags:
>>>>>>
>>>>>> ra -nn -c "," -r <file> <file> <file> ... -L0 -s stime proto saddr dir
>>>>>> daddr dur sport dport - host 1.2.3.4
>>>>>>
>>>>>> I get the following:
>>>>>>
>>>>>> 03:57:23.529664,17,5.6.7.5,<->,1.2.3.4,0.014637,30416,53
>>>>>> 09:57:27.624699,17,5.6.7.5,<->,1.2.3.4,0.014608,29294,53
>>>>>> 12:57:29.660667,17,5.6.7.5,<->,1.2.3.4,0.015812,49771,53
>>>>>> 13:57:30.339190,17,5.6.7.5,<->,1.2.3.4,0.015056,6923,53
>>>>>> 14:57:31.030846,17,5.6.7.5,<->,1.2.3.4,0.015539,31211,53
>>>>>> 16:57:32.385680,17,5.6.7.5,<->,1.2.3.4,0.015089,14851,53
>>>>>> 18:57:33.772816,17,5.6.7.5,<->,1.2.3.4,0.015287,1052,53
>>>>>> 20:57:18:761336,17,5.6.7.5,<->,1.2.3.4,0.015414,6004,53
>>>>>> 20:57:18:793793,17,5.6.7.5,<->,1.2.3.4,0.015191,31141,53
>>>>>> 23:57:20.806478,17,5.6.7.5,<->,1.2.3.4,0.015667,30562,53
>>>>>>
>>>>>> Eyeballing the total time from beginning to end looks to be ~20 hours,
>>>>>> with each connection actually lasting < .02 seconds.  The 72k seconds
>>>>>> from the racluster works out to about 22 hours, which would make sense
>>>>>> if there were overlaps in connection time, but there aren't.  What am
>>>>>> I missing here? Is there a way to get the aggregated results I
>>>>>> expected (in the original email) from racluster without summing them
>>>>>> external to argus?
>>>>>>
>>>>>> Can it then be assumed then, based on my racluster flags, racluster is
>>>>>> aggregating all sessions for the 1.2.3.4 IP based on the 5-tuple of
>>>>>> 1.2.3.4:53 -> 0.0.0.0:0?
>>>>>>
>>>>>> Thanks for all your help!
>>>>>>
>>>>>> On Mon, Mar 21, 2011 at 11:05 AM, Rafael Barbosa <rrbarbosa at gmail.com> wrote:
>>>>>>> racluster(), by default, aggregates all records with the same 5-tuple in a
>>>>>>> single one. The resulting record has the start time of the first record and
>>>>>>> the end time of the last one.
>>>>>>> In your example the duration should be the end time of the last record (the
>>>>>>> one with 96 bytes) minus the start time of the first one (with 213 bytes).
>>>>>>> However without the files you are using is hard to say for sure.
>>>>>>> Best regards,
>>>>>>> Rafael Barbosa
>>>>>>> http://www.vf.utwente.nl/~barbosarr/
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Mar 21, 2011 at 3:34 PM, Digital Ninja <dn1nj4 at gmail.com> wrote:
>>>>>>>>
>>>>>>>> I ran across something with racluster v3.0.2 & v3.0.4 that I can't
>>>>>>>> quite explain and need some help.  I have 9 different argus files.  I
>>>>>>>> am running racluster with the following options:
>>>>>>>>
>>>>>>>> racluster -M rmon -nn -c "," -m saddr proto sport -r <file> -L0 -s
>>>>>>>> saddr proto sport sbytes dur dbytes - not arp
>>>>>>>>
>>>>>>>> When I run this command on the 9 files separately, for a single IP I
>>>>>>>> get something like this:
>>>>>>>>
>>>>>>>> 1.2.3.4,17,53,289,0.47648,213
>>>>>>>> 1.2.3.4,17,53,133,0.015667,117
>>>>>>>> 1.2.3.4,17,53,133,0.014637,117
>>>>>>>> 1.2.3.4,17,53,133,0.014608,117
>>>>>>>> 1.2.3.4,17,53,133,0.015812,117
>>>>>>>> 1.2.3.4,17,53,133,0.015056,117
>>>>>>>> 1.2.3.4,17,53,133,0.015539,117
>>>>>>>> 1.2.3.4,17,53,133,0.015089,117
>>>>>>>> 1.2.3.4,17,53,133,0.015287,96
>>>>>>>>
>>>>>>>> Summing the bytes and duration columns up, I would expect the totals to
>>>>>>>> be:
>>>>>>>> 1.2.3.4,17,53,1376,0.169343,1128
>>>>>>>>
>>>>>>>> However, when I run racluster on all 9 files simultaneously (-r <file>
>>>>>>>> <file> <file>...etc) I get the following results for the above data:
>>>>>>>> 1.2.3.4,17,53,1376,79215.023438,1128
>>>>>>>>
>>>>>>>> What's going on with the duration field??
>>>>>>>>
>>>>>>>> Thanks in advance.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>