Duration sum bug

[Carter Bullard]

The original 9 values that you printed out in your original racluster() runs are not
mean transaction durations, they are range values, as Rafael indicated.
 
You should print the 'mean' value for each entry from the 9 files.  Printing out the
'trans' field (N, or number of transactions) can be very helpful to realize the nature of
the actual numbers racluster() generates.  'trans' will show how many samples were used to
generate the duration metric that you are reporting.

This next description is a bit complex, if you haven't thought about how racluster() generates
and reports its results.  Understand that for every argus record that racluster() creates, it has
inserted an aggregation dsr (data set record) that holds the N, sum and sum of squares for
the metric it is tracking (duration is the default).   Additional runs of racluster() uses any existing
aggregation dsr, and accumulates within that stat.  That allows you to track how each subsequent
run contributes to stats like 'mean', 'stddev'.

If, however, as an example, you want to report the mean for the specific 9 values that you
originally reported, then you need to throw away the aggregation dsr's that are in the records, so
that you treat each of the 9 aggregates as if they were a single metric:

   racluster -M dsrs="-agr"

That will throw away the N, sum, and sum of squares for the metrics of interest that were
created in your first run of racluster().  By throwing those values away, you are saying that
you want the accumulated values to be treated as a single metric.  This is perfectly valid, but
your statistic will be different than what the original data reflects, say if your interested in
transaction duration times.  Something like this:

    racluster -M rmon -m saddr proto sport -r <file> -nn -c "," -m saddr proto sport -L0 \
           -s dur trans mean saddr proto sport sbytes dbytes 

Will highlight some of what is going on.

Then when run racluster() against the clustered data, you can see that the 'trans'
value is increasing in one run, then use the "-M dsrs='-agr'" option, to see how
the 'trans' goes.

Carter


On Mar 21, 2011, at 11:39 AM, Digital Ninja wrote:

> (Correction: The duration in the first row should have been 0.047648
> not 0.47648)
> 
> For the same flags, swapping "mean" for "dur" produces:
> 1.2.3.4,17,53,1376,0.015230,1128.  That seems incorrect as well.
> Shouldn't the mean be 0.018815889 (sum all durations/number of
> durations)?
> 
> On Mon, Mar 21, 2011 at 11:17 AM, Carter Bullard <carter at qosient.com> wrote:
>> If you were to print out the 'mean', you should get the value you expect.
>> The 'mean', 'max', 'min', and 'stddev' default to track the duration metric.
>> 
>> Carter
>> 
>> 
>> On Mar 21, 2011, at 10:34 AM, Digital Ninja wrote:
>> 
>>> I ran across something with racluster v3.0.2 & v3.0.4 that I can't
>>> quite explain and need some help.  I have 9 different argus files.  I
>>> am running racluster with the following options:
>>> 
>>> racluster -M rmon -nn -c "," -m saddr proto sport -r <file> -L0 -s
>>> saddr proto sport sbytes dur dbytes - not arp
>>> 
>>> When I run this command on the 9 files separately, for a single IP I
>>> get something like this:
>>> 
>>> 1.2.3.4,17,53,289,0.47648,213
>>> 1.2.3.4,17,53,133,0.015667,117
>>> 1.2.3.4,17,53,133,0.014637,117
>>> 1.2.3.4,17,53,133,0.014608,117
>>> 1.2.3.4,17,53,133,0.015812,117
>>> 1.2.3.4,17,53,133,0.015056,117
>>> 1.2.3.4,17,53,133,0.015539,117
>>> 1.2.3.4,17,53,133,0.015089,117
>>> 1.2.3.4,17,53,133,0.015287,96
>>> 
>>> Summing the bytes and duration columns up, I would expect the totals to be:
>>> 1.2.3.4,17,53,1376,0.169343,1128
>>> 
>>> However, when I run racluster on all 9 files simultaneously (-r <file>
>>> <file> <file>...etc) I get the following results for the above data:
>>> 1.2.3.4,17,53,1376,79215.023438,1128
>>> 
>>> What's going on with the duration field??
>>> 
>>> Thanks in advance.
>>> 
>> 
>> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110321/c88faff6/attachment.bin>