Huge argus files and racluster

Marco listaddr at gmail.com
Tue Feb 7 04:33:30 EST 2012


Hi, is there any limit on the size an argus file can have? I'm trying
to synthesize about 50GB of pcap data into argus data.
Doing a plain "argus -r hugefile.pcap -w argusdata.argus" works and
produces a 8GB argus file, but then if I try to run racluster on it to
reduce it further, it just seems to hang after a while.

I did some experiment, and I tried creating multiple argus files and
merge tem in successive iteration, but the final merge always fails.
When I say "fails", I mean that it looks like racluster is working up
to a point (ie, disk activity, strace shows it's reading data), but
from a certain point on, it just appears to be hung. Strace, shows
nothing, ltrace shows lots of stuff like

gettimeofday(0xcdd866b8, NULL)
                                               = 0
pthread_mutex_lock(0x2303268, 1, 435251, 0, 0x2303268)
                                               = 0
pthread_mutex_unlock(0x2303268, 1, 0xcdd866a0, 0xcdd866a0, 0x2303268)
                                               = 0
pthread_mutex_lock(0x2303268, 0xcdd87830, 1, 0x4f2be088, 0)
                                               = 0
pthread_mutex_unlock(0x2303268, 0xcdd87830, 1, 0xcdd866a0, 0x2303268)
                                               = 0
gettimeofday(0xcdd87848, NULL)
                                               = 0
pthread_mutex_lock(0x2303268, 1, 435628, 0, 0x2303268)
                                               = 0
pthread_mutex_unlock(0x2303268, 1, 0xcdd87830, 0xcdd87830, 0x2303268)
                                               = 0
pthread_mutex_lock(0x2303268, 0xcdd889c0, 1, 0x4f2be088, 0)
                                               = 0
pthread_mutex_unlock(0x2303268, 0xcdd889c0, 1, 0xcdd87830, 0x2303268)
                                               = 0
...

Currently, I've reduced the problem to the merging of two files:

$ ra -n -r 1_40.argus | wc -l
959586
$ ra -n -r 41_50.argus | wc -l
399927

Doing

$ racluster -r 1_40.argus -r 41_50.argus -w 1_50.argus

produces the behavior described above, and I have to kill racluster
with "kill -9".

Any help appreciated, thanks.



More information about the argus mailing list