Flow Counting

[Carter Bullard]

Hmmmm, the "-M replace" must not be working.  I'll double check.
carter

On Apr 9, 2008, at 1:10 PM, Nick Diel wrote:

> Carter,
>
> There seems to be a problem with rastrip and the replace mode.  The  
> first indication is when replace is specified then you get standard  
> output (flow listings), also after it runs, a simple grep on the  
> trans column shows most have a count greater than one.
>
> rastrip -M replace -agr -r file
> ra -r file -s trans | grep -v 1 | wc -l
> >>0
>
> Writing to a separate file or stdout, everything works as expected,  
> no counts greater than 1.
>
> rastrip -M -agr -r file -w - | ra -r - -s trans | grep -v 1 | wc -l
> 0
>
> Nick
>
> On Tue, Apr 8, 2008 at 1:33 PM, Carter Bullard <carter at qosient.com>  
> wrote:
> Hey Nick,
> racluster() does this quite well.  When racluster() merges two  
> records, for whatever reason,
> it generates an aggregation statistic that keeps track of the number  
> of records that are merged.
> Use this to generate all the counts you would like.  The only trick  
> is that if you use racluster()
> to merge your data first, then this aggregation statistic already  
> has values, and it needs to be
> cleared.   You will want to remove that aggregation structure before  
> you begin.
>
> If you want to prep an entire directory of data, you can do this:
> % rastrip -R dir -M replace -agr
>
> This will remove the aggregation DSR from any record in any data  
> file in
> the entire directory struct.
>
> If you can't, for whatever reason, delete the agr dsr, you can do it  
> as a pipe.
>
> % rastrip -r data.file -M -agr -w - - ip | racluster -m matrix -s  
> saddr daddr trans
>
> This will give you the number of transactions (or flows) for all the  
> A's talking to B's in the file.
>
> For things like time intervals, you should use rabins().
>
> % rastrip -r data.file -M -agr -w - - ip | rabins -M soft time 10s - 
> m srcid -s stime dur trans
>
> This will give you the total flow every 10s, if there are any flows  
> to report.  It will
> force the startime and lasttime to coincide with the 10s time  
> boundaries.
>
> There are an infinite number of examples, so if this doesn't help,  
> send more email.
>
>
> Carter
>
>
>
> On Apr 8, 2008, at 2:23 PM, Nick Diel wrote:
>
> I am interested in counting number of flows for things such as  
> source address, IP pairs, time intervals, etc.  This is closely  
> related to Stéphane Peters email: "Counting flows by time interval  
> in argus."
>
> After using racluster to merge status flow records, you have a file  
> where a record represents a flow.  You can of course use racount  
> with a filter to tell you some of this information, but it seem  
> quite impractical to do things such as find the IP pair generating  
> the most flows.  You could also use a set of pipes or scripts (as  
> Stéphane Peters showed us), but this would require modification  
> every time you wanted something slightly different.
>
> Maybe there is a way using ra tools to do this already and I am  
> missing it.  If not maybe we can get racluster (when specified) or  
> another tool to "zero out"/set the trans count to 1 after we have  
> merged status flow records.  This way things such as racluster -r  
> megredRecords.argus -m saddr -s +trans will now list how many flows  
> for each source address.
>
> I am curious what the group has to say about this.  Is something  
> like this already possible?  Is this information useful to other  
> people than me?  Would modifying the trans column present too much  
> ambiguity, i.e. "under these circumstances the trans column can  
> represent number of flows else it represents aggregated record count"?
>
>
> Nick
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20080409/86c32507/attachment.html>