Big O Impact of Filters

[Carter Bullard]

Hey Jason,
Could you give this version of racluster() a run to see if it does
what you want ???  The principal difference is that the output of
this new racluster() will have records a bit more out of order
that the other version.  

With streaming data, you may not get status reports timely (like
within 0.25 seconds of the status timer expiration) but you will
get correct status record reporting, driven by the idle timeout
period.  I’ll improve this behavior later today.

Sorry for any inconvenience, and thanks for pushing on this !!!!

Carter



On May 15, 2014, at 10:20 AM, Carter Bullard <carter at qosient.com> wrote:

> Hey Jason,
> Found the problem, and its a poor design assumption on my part.
> Its a kind of a thrash between the status timer and the idle timer.
> This does not affect rabins() or radium(), just racluster().
> 
> Fixing it now.
> 
> Carter
> 
> On May 14, 2014, at 5:53 PM, Jason <dn1nj4 at gmail.com> wrote:
> 
>> Hi Carter,
>> 
>> So I asked a very similar question last year (http://comments.gmane.org/gmane.network.argus/9110), but I can't seem to find a response.  I apologize if I'm just missing something or have just forgotten.
>> 
>> I am trying once again to understand why there is such a significant impact on the length of time it takes to run racluster when leveraging filters.  Here is the racluster.conf file I am testing: 
>> 
>> filter="udp and port domain" model="saddr daddr proto sport dport" status=600 idle=10
>> filter="udp" model="saddr daddr proto sport dport" status=600 idle=60
>> filter="" model="saddr daddr proto sport dport" status=600 idle=600
>> 
>> And here are two runs against a single argus file.  The only difference is whether or not I'm using the racluster.conf:
>> 
>> $ time racluster -f racluster.conf -r infile.bin -w outfile.bin -M rmon -u -c "," -m saddr proto sport dport -L0 -Z s -s stime saddr proto sport dport sbytes runtime dbytes trans state - not arp 
>>  
>> real    2m42.935s 
>> user    2m39.274s 
>> sys     0m3.288s 
>>  
>> $ time racluster -r infile.bin -w outfile.bin -M rmon -u -c "," -m saddr proto sport dport -L0 -Z s -s stime saddr proto sport dport sbytes runtime dbytes trans state - not arp 
>>  
>> real    0m1.054s 
>> user    0m0.944s 
>> sys     0m0.108s
>> 
>> Why does the filtered option take exponentially longer?
>> 
>> Thanks!
>> Jason
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140515/fd1baee8/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: racluster.c
Type: application/octet-stream
Size: 41181 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140515/fd1baee8/attachment.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140515/fd1baee8/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140515/fd1baee8/attachment.sig>