a memory oddness (possibly a Mac oddness)

[Peter Van Epp]

On Mon, Feb 05, 2007 at 08:44:43PM -0500, Carter Bullard wrote:
> Hmmm, racluster will aggregate across files, and so you may just be 
> running out
> of memory.  There are two things to consider.  Try running racluster() 
> with a conf
> file that sets an idle time.  This will flush out the very short tcp 
> flows and scanners.
> The other thing to do is have racluster() process each file 
> individually, rather than
> aggregate across all the files, by using the "-M ind" option, "process 
> files independantly".
> That will help, but it will not merge data that crosses files.
> 
> Carter
> 

	I think this is a Mac (and/or memory) issue. The files were being
done one at a time (24 one hour files) from a shell script but it appears
to manage to eat memory permanently between invocations. If I reboot the 
machine it will generally process a file that it just choked on, and get a 
file or two further before running out of swap again. 
	The IBM (with 4 gigs of memory though) under Linux was running fine
until it ran out of disk space :-). Having discovered that racluster (as well
as the rest of argus :-)) does odd things on the one arm routed VLANs on our
backbone I'm trying to make sure it is compressing correctly (rather than 
just well :-)) on our more sane inbound links (no VLANs). I'm currently 
capturing user data on there as well and the size gets large but racluster
knocks it down nicely (as long as it is also correct :-)). 

Peter Van Epp / Operations and Technical Support 
Simon Fraser University, Burnaby, B.C. Canada