What does ramon (TopN mode) actually do?

[Andrew Pollock]

Carter,

I think this going help me immensely. Many thanks.

What's this mystical -a option you refer to?

The ra manpage alludes to it but doesn't really say what it does (and 
least not clearly).

Andrew

On Thu, Jun 26, 2003 at 11:31:32AM -0400, Carter Bullard wrote:
> Hey Andrew,
>    Yes, the discrepancies and shifting of source and
> destination packet/byte counts are expected based on
> what ramon() is doing and the questions that are
> being asked and answered.
> 
>    ramon has 3 basic models that it supports.  Each
> are vastly different in how they account for packets
> and really can't be compared to each other.
> 
>    If you want in/out pkts/bytes, on an address basis,
> for IP traffic, you should use this command.
> 
>    ra -r 2003-05-* -w - - ip and host x.y.z.w | \
>       ramon -w - -M TopN - host x.y.z.w | racount -a
> 
> 
> Check these totals against the totals from
> 
>    ra -r 2003-05-* -w - - ip and host x.y.z.w |  racount -a
> 
> and
> 
>    ra -r 2003-05-* -w - - ip and host x.y.z.w | \
>       ramon -w - -M Svc  | racount -a
> 
> 
> Double check where the total discrepancies are
> by doing this:
> 
>    racount -ar 2003-05-* - host x.y.z.w
> 
> This should reveal any surprises in how the counts
> are derived.
> 
> The total_pkts and total_bytes should agree, but not
> the src and dst counters.  That is because "ramon -M topn"
> redefines the source and destination relative to the address,
> which is similar to an interface counter.  With
> "ramon -M svc", the source and destination are defined
> relative to the service, so there is no modification from
> the original argus data.  Comparing TopN data with Svc data
> from the perspective of source and destination is definitely
> an apple/orange comparison.
> 
> If you want in/out counters against a particular
> host, the "ramon -M TopN" method is the correct one,
> but you have to filter to get the data you want.
> 
> 
> Carter
> 
> 
> 
> 
> > -----Original Message-----
> > From: Andrew Pollock [mailto:andrew-argus at andrew.net.au] 
> > Sent: Thursday, June 26, 2003 4:07 AM
> > To: Carter Bullard
> > Cc: argus-info at lists.andrew.cmu.edu
> > Subject: Re: What does ramon (TopN mode) actually do?
> > 
> > 
> > Carter,
> > 
> > This isn't going to paste very well, but this is where my major 
> > discrepancy is popping up:
> > 
> > Billing for the whole month
> > 
> > ramon -w - -M TopN -r 2003-05-* - host 10.11.2.243 | racount
> > racount    records       total_pkts          in_pkts         
> > out_pkts      
> > total_bytes         in_bytes        out_bytes
> >     sum          8         50578797         28458242         
> > 22120555      
> > 29066633035      22,591,486,569       6,475,146,466
> > 
> > Breakdown for a whole month, added up
> > 
> > ra -w - -r 2003-05-* - host 10.11.2.243 | ramon -w - -M Svc 
> > -r - | racount
> > racount    records       total_pkts         src_pkts         
> > dst_pkts      
> > total_bytes        src_bytes        dst_bytes
> >     sum        313         50548167         24022400         
> > 26525767      
> > 29064795235      12,092,736,400      16,972,058,835
> > 
> > You can see that I've got a fairly close total bytes, but 
> > when I use the 
> > Svc mode, a whole heap of bytes jump from one side to the other.
> > 
> > Andrew
> > 
> > On Thu, Jun 26, 2003 at 01:14:44AM -0400, Carter Bullard wrote:
> > > Hey Andrew,
> > >    'TopN' does not imply 'tcp or udp'.  The
> > > 'TopN' mode of ramon() tracks metrics based
> > > on IP address.  As a result it will count all
> > > traffic that involves IP addresses, udp, tcp,
> > > icmp, igmp, gre, esp, multicast, ..., and
> > > arp traffic, since it also contains IP
> > > addresses.
> > > 
> > >    It's the 'Svc' (services) mode of ramon()
> > > that tracks metrics based on dst port, and as a
> > > result, 'Svc' is the mode that implies 'tcp or udp',
> > > since these are the only protocols that actually
> > > have ports.  I believe this is where the confusion
> > > comes in.
> > > 
> > >    In order for TopN to count select traffic, you
> > > have to filter the input to "ramon -M TopN" so
> > > that it only processes the traffic of interest.
> > > 
> > >    Because ramon() modifies the records so much,
> > > you can't use a filter like "tcp or udp" on the
> > > ramon() command-line.  This is because ramon()
> > > applies the command-line filter to the input and
> > > the output.  To get a better idea, try this:
> > > 
> > >       ramon -M topn -r file -w - | ra
> > > 
> > > you'll see that the output records only have
> > > the source address field intact and the protocol
> > > is ip.  So a filter like 'host x.y.z.w' works,
> > > but a filter like 'dst host x.y.z.w' will not,
> > > since there is no dst host information in the
> > > output record.
> > > 
> > > If you ran this:
> > > 
> > >       ramon -M svc -r file -w - | ra
> > > 
> > > you'd see no address information but you'd
> > > see that protocol and dst port information is the
> > > only information left in the flow key of the record.
> > > So filters like "tcp" work, but filters like
> > > "host x.y.z.w and tcp" won't.
> > > 
> > > If you ran this:
> > > 
> > >       ramon -M matrix -r file -w - | ra
> > > 
> > > you'd see that the address information for both
> > > the source and destination are intact, but no other
> > > flow information is retained.
> > > 
> > > The work around for the filter problem, is to use
> > > ra() to filter the records and pipe them into
> > > 'ramon -M TopN'.
> > > 
> > > I hope that this is helping to clear it up a bit more.
> > > Please don't hesitate to send mail if there are still
> > > questions!!!!
> > > 
> > > 
> > > Carter
> > > 
> > > 
> > > 
> > > > -----Original Message-----
> > > > From: owner-argus-info at lists.andrew.cmu.edu 
> > > > [mailto:owner-argus-info at lists.andrew.cmu.edu] On Behalf Of 
> > > > Andrew Pollock
> > > > Sent: Thursday, June 26, 2003 12:17 AM
> > > > To: argus-info at lists.andrew.cmu.edu
> > > > Subject: What does ramon (TopN mode) actually do?
> > > > 
> > > > 
> > > > Hi,
> > > > 
> > > > I'm still on my quest to get a per TCP/UDP protocol breakdown 
> > > > that matches 
> > > > my total from a TopN.
> > > > 
> > > > I'm trying to emulate what a ramon -M TopN actually does by 
> > > > playing with 
> > > > the data spat out of an ra() call.
> > > > 
> > > > If I go [1] "ramon -r argus.log -w - -M TopN - host 
> > > > 10.11.2.243 | racount"  
> > > > I get a total for in and out.
> > > > 
> > > > If I go [2] "ra -r argus.log - host 10.11.2.243" and manually 
> > > > add up the 
> > > > source bytes where the destination address is 10.11.2.243 
> > and the dst 
> > > > bytes where the source address is 10.11.2.243 I get a 
> > > > different figure for 
> > > > a total in than from [1] (slightly lower).
> > > > 
> > > > If I go [3] "ra -r argus.log -w - - tcp or udp | ramon -r - 
> > > > -w - -M TopN
> > > > - host 10.11.2.243 | racount" I get a total that matches 
> > what I've 
> > > > manually arrived at from [2].
> > > > 
> > > > So, how is [1] arriving at it's total? I've been told 
> > > > previous that a TopN 
> > > > implies a filter of "tcp or udp" along with whatever else I 
> > > > supply, but I 
> > > > can't make it add up. :-(
> > > > 
> > > > Hope I'm making sense.
> > > > 
> > > > Andrew
> > > > 
> > > 
> > 
>