rabins -w extended format doesn't seem to work

[Carter Bullard]

Hey Corey,
These are features of rasplit() and rastream(), but no real reason they shouldn't
be in rabins(). I'll fix/add these output features in the next few days.

rasplit()/rastream() and rabins() are much different in this mode of operation, and you
may want to consider these differences.  rasplit()/rastream() will not buffer any input
records, and so it will write the data records immediately upon receiving
them.   This is a huge benefit if you are dealing with a high volume of data.
rastream(), can be used to run scripts against the files it generates, so that
you can sort and aggregate the data, some number of seconds after the file
is closed.

rabins(), is specifically an aggregator.   It creates time bins, in your case 10 minute
bins, and it aggregates records that it receives into those 10 minute bins.
This means rabins() buffers its data  for 10 minutes and then outputs the bin
of records, all at once.  The benefits are that you get aggregation, which you can
specify on the command-line ( or you use the default), and startime sorting for
the output.   A problem is that in many situations, rabins() will use a lot
of memory, and if rabins() fails and restarts, you will lose potentially a full
10m of records.  Just a few things to consider.  

Another problem with rabins(), is that it will throw records away that do
not fall into its concept of time, so it is important to configure an input 'buffer' 
using the "-B secs" option.  The value should be equal to or greater
than the largest ARGUS_FLOW_STATUS_INTERVAL used by your probes.

Do not use rabins() if you are reading netflow records, as you will throw
many records away since the startime can be way back in historical time.

OK, I'll do something with rabins() and the output file strategies we support
for rasplit() and rastream().

Carter


On Jul 20, 2010, at 2:26 PM, Corey Smith wrote:

> I've tried it in both 3.0.2 and 3.0.3.15 without success.  Shouldn't
> this work?
> 
> rabins -S 127.0.0.1:561 -M time 10m -w t/%Y/%m/%d/argus.%H.%M.%S
> 
> # find t
> t
> t/%Y
> t/%Y/%m
> t/%Y/%m/%d
> 
> Another example:
> 
> rabins -S 127.0.0.1:561 -M time 10m -w "t/\$srcid/argus."
> 
> # find t
> t
> t/$srcid
> 
> I'm running FreeBSD i386 BTW.
> 
> -Corey Smith
> 




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100720/b8ebd065/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100720/b8ebd065/attachment.bin>