new clients rc.62 on the server - description of rastream()

[Carter Bullard]

Hey Tez,
Sorry for the inconvenience!!  I haven't seen this, but I'll try to reproduce.  A few questions.  Platform (intel/ppc/sparc), available memory, estimated record load and how many probes?

Thanks for the email!!!

Carter

Carter Bullard
QoSient LLC
150 E. 57th Street Suite 12D
New York, New York 10022
+1 212 588-9133 Phone
+1 212 588-9134 Fax

-----Original Message-----
From: "Terry Burton" <tez at terryburton.co.uk>

Date: Wed, 31 Oct 2007 12:52:56 
To:Argus <argus-info at lists.andrew.cmu.edu>
Cc:"Carter Bullard" <carter at qosient.com>
Subject: Re: [ARGUS] new clients rc.62 on the server - description of rastream()


On 10/30/07, Carter Bullard <carter at qosient.com> wrote:
> rastream() is now ready to use in production situations.  It is the
> replacement
> for argusarchive, and is designed to make archive generation much
> easier.
> It is a designed as a persistent raspilt() that can process archive
> files as they
> are closed.  The example shell script that is included in the
> distribution
<...snip...>

Hi,

rastream is an excellent tool and I have switched over to using this
for our logging and post-processing, rather than using the "-w ...."
option to argus/radium.

It does appear to leak memory though as the process fell over after
about 8 hours leaving the following in syslog:

Oct 31 08:25:15 mink rastream[24204]: 08:25:15.963144 ArgusRunScript
(/srv/argus/archive/2007-10-31/xxx.yyy.7.1-08:20:00.arg) fork() error
Cannot allocate memory
Oct 31 08:25:16 mink rastream[24204]: 08:25:16.169338 ArgusRunScript
(/srv/argus/archive/2007-10-31/xxx.yyy.7.1-00:10:00.arg) fork() error
Cannot allocate memory

i686 GNU/Linux, invoked as rastream -X -S localhost:569 -M time 5m -B
10s -f /bin/true -w
/srv/argus/archive/\$srcid/%Y/%m/%d/argus.%Y.%m.%d.%H.%M.%S

I will investigate this further, hopefully later today.

A suggestion:

Would it be possible to present the template variables, ($srcid, %Y,
%M, et.al.) to the "-f script", perhaps via environment variables?
Currently the post-processing script has to reparse these out of its
"-r parameter".


Thanks again,

Tez