a memory oddness (possibly a Mac oddness)

[Carter Bullard]

Hmmm, racluster will aggregate across files, and so you may just be 
running out
of memory.  There are two things to consider.  Try running racluster() 
with a conf
file that sets an idle time.  This will flush out the very short tcp 
flows and scanners.
The other thing to do is have racluster() process each file 
individually, rather than
aggregate across all the files, by using the "-M ind" option, "process 
files independantly".
That will help, but it will not merge data that crosses files.

Carter


Peter Van Epp wrote:

>	I'm trying to run racluster against an archive. I started with a 
>script that does 24 hours (one hour at a time) and discovered that my 2 gig
>Mac ran out of memory a couple of files in. Rebooting (which seems to be 
>required to free VM) and running them individually seems to still run out
>of memory (perhaps indicating a memory leak) on about the second file:
>
>test4:/var/log/argus vanepp$ /usr/local/bin/racluster -r  /archive/argus3/com_argus.archive/2007/01/30/com_argus.2007.01.30.09.00.00.0.gz -w  /archive/argus3c/com_argus.archive/2007/01/30/com_argus.2007.01.30.09.00.00.0
>test4:/var/log/argus vanepp$ /usr/local/bin/racluster -r  /archive/argus3/com_argus.archive/2007/01/30/com_argus.2007.01.30.10.00.01.0.gz -w  /archive/argus3c/com_argus.archive/2007/01/30/com_argus.2007.01.30.10.00.01.0
>racluster(364) malloc: *** vm_allocate(size=1069056) failed (error code=3)
>racluster(364) malloc: *** error: can't allocate region
>racluster(364) malloc: *** set a breakpoint in szone_error to debug
>Bus error
>
>	I just started moving the source archive from the Mac to one of the 
>IBMs with 4 gigs of ram to see if the same thing will happen there. I'd 
>think that the racluster task ending should release all the memory its holding
>which doesn't seem to be happening (but may be a Mac issue rather than 
>argus). 
>
>Peter Van Epp / Operations and Technical Support 
>Simon Fraser University, Burnaby, B.C. Canada
>
>  
>