What does ramon (TopN mode) actually do?

[Carter Bullard]

Hey Andrew,
   Yes, the discrepancies and shifting of source and
destination packet/byte counts are expected based on
what ramon() is doing and the questions that are
being asked and answered.

   ramon has 3 basic models that it supports.  Each
are vastly different in how they account for packets
and really can't be compared to each other.

   If you want in/out pkts/bytes, on an address basis,
for IP traffic, you should use this command.

   ra -r 2003-05-* -w - - ip and host x.y.z.w | \
      ramon -w - -M TopN - host x.y.z.w | racount -a


Check these totals against the totals from

   ra -r 2003-05-* -w - - ip and host x.y.z.w |  racount -a

and

   ra -r 2003-05-* -w - - ip and host x.y.z.w | \
      ramon -w - -M Svc  | racount -a


Double check where the total discrepancies are
by doing this:

   racount -ar 2003-05-* - host x.y.z.w

This should reveal any surprises in how the counts
are derived.

The total_pkts and total_bytes should agree, but not
the src and dst counters.  That is because "ramon -M topn"
redefines the source and destination relative to the address,
which is similar to an interface counter.  With
"ramon -M svc", the source and destination are defined
relative to the service, so there is no modification from
the original argus data.  Comparing TopN data with Svc data
from the perspective of source and destination is definitely
an apple/orange comparison.

If you want in/out counters against a particular
host, the "ramon -M TopN" method is the correct one,
but you have to filter to get the data you want.


Carter




> -----Original Message-----
> From: Andrew Pollock [mailto:andrew-argus at andrew.net.au] 
> Sent: Thursday, June 26, 2003 4:07 AM
> To: Carter Bullard
> Cc: argus-info at lists.andrew.cmu.edu
> Subject: Re: What does ramon (TopN mode) actually do?
> 
> 
> Carter,
> 
> This isn't going to paste very well, but this is where my major 
> discrepancy is popping up:
> 
> Billing for the whole month
> 
> ramon -w - -M TopN -r 2003-05-* - host 10.11.2.243 | racount
> racount    records       total_pkts          in_pkts         
> out_pkts      
> total_bytes         in_bytes        out_bytes
>     sum          8         50578797         28458242         
> 22120555      
> 29066633035      22,591,486,569       6,475,146,466
> 
> Breakdown for a whole month, added up
> 
> ra -w - -r 2003-05-* - host 10.11.2.243 | ramon -w - -M Svc 
> -r - | racount
> racount    records       total_pkts         src_pkts         
> dst_pkts      
> total_bytes        src_bytes        dst_bytes
>     sum        313         50548167         24022400         
> 26525767      
> 29064795235      12,092,736,400      16,972,058,835
> 
> You can see that I've got a fairly close total bytes, but 
> when I use the 
> Svc mode, a whole heap of bytes jump from one side to the other.
> 
> Andrew
> 
> On Thu, Jun 26, 2003 at 01:14:44AM -0400, Carter Bullard wrote:
> > Hey Andrew,
> >    'TopN' does not imply 'tcp or udp'.  The
> > 'TopN' mode of ramon() tracks metrics based
> > on IP address.  As a result it will count all
> > traffic that involves IP addresses, udp, tcp,
> > icmp, igmp, gre, esp, multicast, ..., and
> > arp traffic, since it also contains IP
> > addresses.
> > 
> >    It's the 'Svc' (services) mode of ramon()
> > that tracks metrics based on dst port, and as a
> > result, 'Svc' is the mode that implies 'tcp or udp',
> > since these are the only protocols that actually
> > have ports.  I believe this is where the confusion
> > comes in.
> > 
> >    In order for TopN to count select traffic, you
> > have to filter the input to "ramon -M TopN" so
> > that it only processes the traffic of interest.
> > 
> >    Because ramon() modifies the records so much,
> > you can't use a filter like "tcp or udp" on the
> > ramon() command-line.  This is because ramon()
> > applies the command-line filter to the input and
> > the output.  To get a better idea, try this:
> > 
> >       ramon -M topn -r file -w - | ra
> > 
> > you'll see that the output records only have
> > the source address field intact and the protocol
> > is ip.  So a filter like 'host x.y.z.w' works,
> > but a filter like 'dst host x.y.z.w' will not,
> > since there is no dst host information in the
> > output record.
> > 
> > If you ran this:
> > 
> >       ramon -M svc -r file -w - | ra
> > 
> > you'd see no address information but you'd
> > see that protocol and dst port information is the
> > only information left in the flow key of the record.
> > So filters like "tcp" work, but filters like
> > "host x.y.z.w and tcp" won't.
> > 
> > If you ran this:
> > 
> >       ramon -M matrix -r file -w - | ra
> > 
> > you'd see that the address information for both
> > the source and destination are intact, but no other
> > flow information is retained.
> > 
> > The work around for the filter problem, is to use
> > ra() to filter the records and pipe them into
> > 'ramon -M TopN'.
> > 
> > I hope that this is helping to clear it up a bit more.
> > Please don't hesitate to send mail if there are still
> > questions!!!!
> > 
> > 
> > Carter
> > 
> > 
> > 
> > > -----Original Message-----
> > > From: owner-argus-info at lists.andrew.cmu.edu 
> > > [mailto:owner-argus-info at lists.andrew.cmu.edu] On Behalf Of 
> > > Andrew Pollock
> > > Sent: Thursday, June 26, 2003 12:17 AM
> > > To: argus-info at lists.andrew.cmu.edu
> > > Subject: What does ramon (TopN mode) actually do?
> > > 
> > > 
> > > Hi,
> > > 
> > > I'm still on my quest to get a per TCP/UDP protocol breakdown 
> > > that matches 
> > > my total from a TopN.
> > > 
> > > I'm trying to emulate what a ramon -M TopN actually does by 
> > > playing with 
> > > the data spat out of an ra() call.
> > > 
> > > If I go [1] "ramon -r argus.log -w - -M TopN - host 
> > > 10.11.2.243 | racount"  
> > > I get a total for in and out.
> > > 
> > > If I go [2] "ra -r argus.log - host 10.11.2.243" and manually 
> > > add up the 
> > > source bytes where the destination address is 10.11.2.243 
> and the dst 
> > > bytes where the source address is 10.11.2.243 I get a 
> > > different figure for 
> > > a total in than from [1] (slightly lower).
> > > 
> > > If I go [3] "ra -r argus.log -w - - tcp or udp | ramon -r - 
> > > -w - -M TopN
> > > - host 10.11.2.243 | racount" I get a total that matches 
> what I've 
> > > manually arrived at from [2].
> > > 
> > > So, how is [1] arriving at it's total? I've been told 
> > > previous that a TopN 
> > > implies a filter of "tcp or udp" along with whatever else I 
> > > supply, but I 
> > > can't make it add up. :-(
> > > 
> > > Hope I'm making sense.
> > > 
> > > Andrew
> > > 
> > 
>