how to use rastream properly

[Carter Bullard]

Jason,
You must add a "-B secs" option to rastream().  It needs to know how  
long to wait after
the time period before it is suppose to launch the shell.  So this  
should work:

> rastream -B 15s -S probe-01:561 -M time 5m -w "/data/argus/\$srcid/ 
> %Y/%m/%d/%H/%Y.%m.%d.%H.%M.%S.argus" -f /scripts/compress.sh

So this will wait until 15s after the end of each 5 minute boundary,  
close the file(s) and then launch the scripts.
You need to wait long enough for all the argus records to show up, so  
that when you process the file,
you don't have new records coming in.  If you're argus FAR_STATUS  
interval is the standard 5 seconds, then
15 seconds should be overkill.

rasplit() can split with the source id in the path description:
    rasplit S probe-01:561 -M time 5m -w /data/argus/\$srcid/%Y/%m/%d/ 
%H/%Y.%m.%d.%H.%M.%S.argus

The only thing rastream() is suppose to add to the picture is the "-B  
secs" and "-f shell" options.

Carter

On Aug 10, 2009, at 1:59 PM, Jason Carr wrote:

> Hello,
>
> I'm using Rasplit Version 3.0.2.beta.11 on an amd64 architecture.
>
> I'm trying to use rastream to capture all packets from an argus  
> stream into 5 minute files.  This is my current command line:
>
> rastream -S probe-01:561 -M time 5m -w "/data/argus/\$srcid/%Y/%m/%d/ 
> %H/%Y.%m.%d.%H.%M.%S.argus" -f /scripts/compress.sh
>
> Right now, very very small files are being produced, approximately  
> around 20k or so.  It only has a few packets listed, most of the  
> packets are exactly at the 5 minute mark.  If I try with rasplit:
>
> rasplit -S probe-01:561 -M time 5m -w /data/argus/core/%Y/%m/%d/%H/ 
> %Y.%m.%d.%H.%M.%S.argus
>
> This works fine and produces 300MB files for each 5 minutes like it  
> should.  I would prefer to use rastream because I want the ability  
> to split the stream out by source ID and the ability to run a script  
> to parse the file and compress it.
>
> What am I doing wrong?
>
> - Jason
>
>





-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090810/bdaebd23/attachment.bin>