Flow Counting

[Carter Bullard]

Hey Nick,
racluster() does this quite well.  When racluster() merges two  
records, for whatever reason,
it generates an aggregation statistic that keeps track of the number  
of records that are merged.
Use this to generate all the counts you would like.  The only trick is  
that if you use racluster()
to merge your data first, then this aggregation statistic already has  
values, and it needs to be
cleared.   You will want to remove that aggregation structure before  
you begin.

If you want to prep an entire directory of data, you can do this:
% rastrip -R dir -M replace -agr

This will remove the aggregation DSR from any record in any data file in
the entire directory struct.

If you can't, for whatever reason, delete the agr dsr, you can do it  
as a pipe.

% rastrip -r data.file -M -agr -w - - ip | racluster -m matrix -s  
saddr daddr trans

This will give you the number of transactions (or flows) for all the  
A's talking to B's in the file.

For things like time intervals, you should use rabins().

% rastrip -r data.file -M -agr -w - - ip | rabins -M soft time 10s -m  
srcid -s stime dur trans

This will give you the total flow every 10s, if there are any flows to  
report.  It will
force the startime and lasttime to coincide with the 10s time  
boundaries.

There are an infinite number of examples, so if this doesn't help,  
send more email.


Carter


On Apr 8, 2008, at 2:23 PM, Nick Diel wrote:

> I am interested in counting number of flows for things such as  
> source address, IP pairs, time intervals, etc.  This is closely  
> related to Stéphane Peters email: "Counting flows by time interval  
> in argus."
>
> After using racluster to merge status flow records, you have a file  
> where a record represents a flow.  You can of course use racount  
> with a filter to tell you some of this information, but it seem  
> quite impractical to do things such as find the IP pair generating  
> the most flows.  You could also use a set of pipes or scripts (as  
> Stéphane Peters showed us), but this would require modification  
> every time you wanted something slightly different.
>
> Maybe there is a way using ra tools to do this already and I am  
> missing it.  If not maybe we can get racluster (when specified) or  
> another tool to "zero out"/set the trans count to 1 after we have  
> merged status flow records.  This way things such as racluster -r  
> megredRecords.argus -m saddr -s +trans will now list how many flows  
> for each source address.
>
> I am curious what the group has to say about this.  Is something  
> like this already possible?  Is this information useful to other  
> people than me?  Would modifying the trans column present too much  
> ambiguity, i.e. "under these circumstances the trans column can  
> represent number of flows else it represents aggregated record count"?
>
>
> Nick
>