Racluster ordering

[Carter Bullard]

Hey Rafael,
I have reworked racluster() to better deal with the problem you reported.
New software should be up later today/tomorrow.

The original algorithm was designed to take in unsorted data, and generate good cluster data based on the configuration file.
I've now got unsorted input discovery, and I'll make adjustments accordingly.

racluster() can't guarantee time order, especially with 3 criteria for reporting a flow, 2 based on stime and 1
based on ltime.  It works much better, and if you only have one criteria, I think I can get the data in some order.

Carter

On Jul 27, 2011, at 4:22 AM, Rafael Barbosa wrote:

> Is it does, and I think I ran into this behavior before.
> 
> But shouldn't the output be order by ltime then? 
> 
> Rafael Barbosa
> http://www.vf.utwente.nl/~barbosarr/
> 
> 
> 
> On Wed, Jul 27, 2011 at 12:51 AM, Carter Bullard <carter at qosient.com> wrote:
> When you specify an idle time, you should expect the output to be unordered.  As an example four flow records:
>    A. stime=0 dur=10 flow=1
>    B. stime=1 dur=5   flow=2
>    C. stime=200 dur=10  flow=1
>    D. stime=300 dur=10  flow=1
> 
> racluster() wil read and cache A, read and cache B, read C., at which point it will match with flow A and aggregate it, then it will read D., where it will realize that B needs to be written out as idle, then it will match with flow A, and aggregate.  When the input is done (EOF), it will flush out the aggregated A. record, and the two outputs will be out of order.
> 
> Does that make sense?
> 
> Carter
> 
> 
> 
> 
> On Jul 26, 2011, at 10:35 AM, Rafael Barbosa <rrbarbosa at gmail.com> wrote:
> 
>> Hi,
>> 
>> Once again, a question about the ordering of ra() data. I am trying to obtain unique flows (no status report) using racluster.
>> 
>> $ racluster -r test.argus -w test.argus.merged -f ~/config/racluster.conf
>> 
>> Where racluster.conf simple contais:
>> filter="" status=0 idle=300
>> 
>> The problem is that while the input is 'stime' ordered, the output is not. 
>> 
>> I found the issue at clients 3.0.5.15, but they also appear at the latest 3.0.5.17. I upload an example file "test.argus", that shows the behavior.
>> 
>> Regards,
>> Rafael Barbosa
>> http://www.vf.utwente.nl/~barbosarr/
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110808/14c3124b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110808/14c3124b/attachment.bin>