Flow Counting

[Nick Diel]

Carter,

There seems to be a problem with rastrip and the replace mode.  The first
indication is when replace is specified then you get standard output (flow
listings), also after it runs, a simple grep on the trans column shows most
have a count greater than one.

rastrip -M replace -agr -r file
ra -r file -s trans | grep -v 1 | wc -l
>>0

Writing to a separate file or stdout, everything works as expected, no
counts greater than 1.

rastrip -M -agr -r file -w - | ra -r - -s trans | grep -v 1 | wc -l
0

Nick

On Tue, Apr 8, 2008 at 1:33 PM, Carter Bullard <carter at qosient.com> wrote:

> Hey Nick,
> racluster() does this quite well.  When racluster() merges two records,
> for whatever reason,
> it generates an aggregation statistic that keeps track of the number of
> records that are merged.
> Use this to generate all the counts you would like.  The only trick is
> that if you use racluster()
> to merge your data first, then this aggregation statistic already has
> values, and it needs to be
> cleared.   You will want to remove that aggregation structure before you
> begin.
>
> If you want to prep an entire directory of data, you can do this:
> % rastrip -R dir -M replace -agr
>
> This will remove the aggregation DSR from any record in any data file in
> the entire directory struct.
>
> If you can't, for whatever reason, delete the agr dsr, you can do it as a
> pipe.
>
> % rastrip -r data.file -M -agr -w - - ip | racluster -m matrix -s saddr
> daddr trans
>
> This will give you the number of transactions (or flows) for all the A's
> talking to B's in the file.
>
> For things like time intervals, you should use rabins().
>
> % rastrip -r data.file -M -agr -w - - ip | rabins -M soft time 10s -m
> srcid -s stime dur trans
>
> This will give you the total flow every 10s, if there are any flows to
> report.  It will
> force the startime and lasttime to coincide with the 10s time boundaries.
>
> There are an infinite number of examples, so if this doesn't help, send
> more email.
>
>
> Carter
>
>
>
> On Apr 8, 2008, at 2:23 PM, Nick Diel wrote:
>
>  I am interested in counting number of flows for things such as source
> > address, IP pairs, time intervals, etc.  This is closely related to Stéphane
> > Peters email: "Counting flows by time interval in argus."
> >
> > After using racluster to merge status flow records, you have a file
> > where a record represents a flow.  You can of course use racount with a
> > filter to tell you some of this information, but it seem quite impractical
> > to do things such as find the IP pair generating the most flows.  You could
> > also use a set of pipes or scripts (as Stéphane Peters showed us), but this
> > would require modification every time you wanted something slightly
> > different.
> >
> > Maybe there is a way using ra tools to do this already and I am missing
> > it.  If not maybe we can get racluster (when specified) or another tool to
> > "zero out"/set the trans count to 1 after we have merged status flow
> > records.  This way things such as racluster -r megredRecords.argus -m saddr
> > -s +trans will now list how many flows for each source address.
> >
> > I am curious what the group has to say about this.  Is something like
> > this already possible?  Is this information useful to other people than me?
> >  Would modifying the trans column present too much ambiguity, i.e. "under
> > these circumstances the trans column can represent number of flows else it
> > represents aggregated record count"?
> >
> >
> > Nick
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20080409/2afff8c2/attachment.html>