racluster crashes, and memory utilization

[Peter Van Epp]

On Mon, Feb 26, 2007 at 11:10:28AM -0800, Michael Hornung wrote:
> Hi, new member to the list here and a new argus user.  I'm running an 
> argus probe and a separate collector, retrieving info via a SASL 
> connection between the two, and the collector is writing files to disk.
> 
> My collector is OpenBSD 4.0 on a P4 2.8Ghz with 1GB physical RAM and many 
> times that in swap.  I'm running RC39.  When I try to combine several 
> logs' worth of data (the log being archived when it reaches a given size) 
> into one argus stream using 'racluster' I continually run out of memory 
> when I do not expect to.  See an example:
> 
> % ls -l ../archive/20070226-[45]
> -rw-r--r--  1 argus  argus  287077860 Feb 26 09:50 ../archive/20070226-4
> -rw-r--r--  1 argus  argus  295809628 Feb 26 10:00 ../archive/20070226-5
> 
> % racluster -M rmon -m saddr -r ../archive/20070226-[45] -w clustered
> racluster[11726]: 11:04:40.200048 ArgusMallocListRecord ArgusMalloc Cannot 
> allocate memory
> racluster[11726]: 11:04:40.200563 ArgusNewSorter ArgusCalloc error Cannot 
> allocate memory
> Segmentation fault (core dumped)
> 
> (gdb) bt
> #0  0x1c04473b in ArgusNewSorter ()
> #1  0x1c00262f in RaParseComplete ()
> #2  0x1c003b92 in ArgusShutDown ()
> #3  0x1c028d69 in ArgusLog ()
> #4  0x1c028b9e in ArgusMallocListRecord ()
> #5  0x1c03afe2 in ArgusCopyRecordStruct ()
> #6  0x1c002d0a in RaProcessThisRecord ()
> #7  0x1c0029f5 in RaProcessRecord ()
> #8  0x1c01adb8 in ArgusHandleDatum ()
> #9  0x1c038713 in ArgusReadStreamSocket ()
> #10 0x1c038adb in ArgusReadFileStream ()
> #11 0x1c003829 in main ()
> 
> Any ideas?  Thanks.
> 
> _____________________________________________________
>  Michael Hornung          Computing & Communications
>  hornung at washington.edu   University of Washington

	Have you boosted the memory rlimits in your kernel? I don't remember
the default, but I think its 512K. On FreeBSD I'm using these:

options MAXDSIZ="(1380*1024*1024)"
options DFLDSIZ="(1380*1024*1024)"
options MAXSSIZ="(1024*1024*1024)"

Peter Van Epp / Operations and Technical Support 
Simon Fraser University, Burnaby, B.C. Canada