Unique, complete flows only

[Harry Bock]

Great... rabins seems to do EXACTLY what I need, thanks!

My only problem now is that rabins does not seem to process the output
prefix properly - when invoked as follows,
$ rabins -S tinderbox -M hard time 10s -B 20s -w "test.%Y%m%d-%H.%M.%S" - ip
rabins creates a file test.%Y%m%d-%H.%M.%S, and continually overwrites it -
it does not create new files with names processed with strftime,
as stated in the manual page.  Am I invoking this wrong, or is this a bug?
I'm still basing off argus-clients-3.0.2-beta8, should I try updating to
beta10?

Also, when using time splitting with rabins, does it always align to the
earliest time in that period (i.e., when you select 1 day or 1 week, does
the time period start at midnight the current day or right now, or Sunday of
this week?)? It looks like it does from your brief 10s test, which is what
I'm looking for, but I just want to make sure :)

Thanks!!
Harry

On Mon, Jul 27, 2009 at 3:02 PM, Carter Bullard <carter at qosient.com> wrote:

> Hey Harry,   racluster -r input -w output
>
> will aggregate flows using the default 6-tuple key of "srcid saddr daddr
> proto sport dport".
> The "-M norep" option is pretty much obsolete.  It caused racluster() to
> not report the AGR dsr,
> which caused problems for some client programs, like rahisto().  The option
> was "no report"
> of aggregation statistics.  If you want to ignore the agr dsr, all client
> programs now support
> input dsr filtering:
>
>    ra -r output.file -M dsr="-agr"
>
> Rmon is a completely different thing all together.  The "-M rmon" option
> indicates to racluster()
> that you want to generate IETF RMON style statistics, which reference
> single objects, rather
> than the two object stats that argus records generate.  So, if you wanted
> stats for a single IP
> address, or a single port, you would use RMON, and then choose a specific
> object for aggregation,
> such as:
>    racluster -M rmon -m smac saddr -r input -w output
>
> This would generate RMON "In" and "Out" stats for single mac/ip address
> pairs.
>
> Yes, rabins() is an important program to use, as it does all the hard work
> under the covers.
> And you can use rabins() on live feeds.  Try running this:
>
>    rabins -S argus.stream -M time 10s -m srcid matrix/16 -B 20s -s stime
> dur srcid saddr daddr spkts dpkts - ip
>
> Wait a little while ~30seconds,  and you should get, every 10 seconds, an
> aggregated matrix report for that time period.
> If you want the reported time for the flow reports to reflect the time
> period, add the "hard" mode:
>
>   rabins -S argus.stream -M hard time 10s -m srcid matrix/16 -B 20s -s
> stime dur srcid saddr daddr spkts dpkts - ip
>
> Carter
>
> On Jul 27, 2009, at 2:42 PM, Harry Bock wrote:
>
> Hi all,
>
> I was curious as to how Argus client users aggregate their traffic data
> using racluster/rabins.  What I'm looking for is essentially
> completed/timed-out flows only, aggregated so that there is only one record
> per flow.  It seems like this can be achieved for IP transactions with
> something along the lines of:
>
> racluster -r input -m saddr daddr proto sport dport -M norep
>
> What's the difference between norep and rmon in terms of aggregation?
> Judging by the description in the manual page, norep seems to be what I'm
> looking for.  For general IPv4/6 traffic, would the above aggregation
> objects be suffient for uniqueness?
>
> The program I'm writing right now does its own transaction stream
> time-splitting, but the more I look at rabins, the more it seems like it
> would make things much easier to just bin remote data with rabins() and then
> perform data processing locally on the output files.
>
> Harry
>
> --
> Harry Bock
> Software Developer, Package Maintainer
> OSHEAN, Inc.
> Email: harry at oshean.org
> PGP Key ID: 546CC353
>
>
>


-- 
Harry Bock
Software Developer, Package Maintainer
OSHEAN, Inc.
Email: harry at oshean.org
PGP Key ID: 546CC353
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090728/fd0b8f34/attachment.html>