racount
Carter Bullard
carter at qosient.com
Mon Jul 10 13:28:07 EDT 2000
Hey Peter!
Loud and clear on the non-IP traffic counts! That is
easy to do, and will be in 2.0.
I have to admit, I know nothing of perl, so the analogy
is going to bounce off. But if its any consolation, we do
have the building blocks of your racount() description in
the vault, so it won't take long to write it (we could have
it out long before 2.0).
Now, it seems that there are many variants that could
be supported, but I have an idea that may satisfy.
Your description sounds like an aggregator and then a count
reporter. How about doing the aggregation in a separate
routine, and piping its output into racount(). That would
give us the flexibility to aggregate in a number of ways, based
on subnet address for the source and no source port number, etc ...
and then have a counter report generator work off of the
aggregation output.
Does that sound useful?
Carter
-----Original Message-----
From: owner-argus at lists.andrew.cmu.edu
[mailto:owner-argus at lists.andrew.cmu.edu]On Behalf Of Peter Van Epp
Sent: Monday, July 10, 2000 1:00 PM
To: argus at lists.andrew.cmu.edu
Subject: Re: racount
A portable long long implementation / printf package for the 32 bit
machines would be a useful addition to the arsenal (assuming there isn't
something already of course).
From my point of view an racount that acts like my perl script would
be desirable:
1) Reverse sorted by traffic volume (i.e. largest first for easy manual
reading) sorted on the IP address doing the most traffic
2) A list of address pairs and destination port number (tricky to determine
sometimes!) again reverse sorted by traffic volume.
The output would look something like this:
1.2.3.4 1,000,000,000
1.2.3.4 <-> 2.3.4.5 5180 500,000,000
4.5.6.7 <-> 1.2.3.4 5180 250,000,000
...
100 host pairs below traffic threshold (details below in this file)
This is useful for finding (for instance) napster servers. A large
traffic volume to many diverse sites on an unusual port number is very
suspicious. We probably want a configuration file like the perl script in
the
contib directory on the ftp site that allows thresholds so the output
doesn't
get overwhelming (in the above example, for instance pairs with less than
100,000,000 would appear in the "100 host pairs below threshold" summary
line
and be detailed in either a different output file or below in this one in
case
you need the detail.
Being able to filter and classify traffic as being in a class of network
i.e. in my case local on campus which is all our network and attracts no
charges at all, our remote campus which may attract usage charges on the
OC3 but not Internet transit charges, transit traffic to our IX which may
in some cases incur a transit charge at the IX, traffic to CA*net3 (free
other
than OC3 traffic) and Internet transit traffic (most expensive!) to the
commodity Internet for accounting purposes. For this you need to be able to
define address ranges as belonging to one of the address classes.
As well as a general argus goal it would be useful to be able to
at least count non IP packets flowing by (IPX, Appletalk, Decnet etc.) for
accounting purposes (even if it is only a byte count of non IP traffic).
Peter Van Epp / Operations and Technical Support
Simon Fraser University, Burnaby, B.C. Canada
>
> Hey Neil,
> Good to see you! So, great embarrassment on my
> part! I called myself a programmer once long ago,
> but no more! Hopefully, I'll have development machines
> in next week so I can fix this problem.
>
> Great opportunity to talk about racount() and how
> it can be improved. Your mods break out the counts
> by source and destination, this looks good. I used racount()
> primarily as a sanity check for debugging new versions of argus,
> so I didn't need much from it. Are there other things
> that we could add to racount.c to make it more useful?
>
> Carter
>
> -----Original Message-----
> From: Neil Long [mailto:neil.long at computing-services.oxford.ac.uk]
> Sent: Monday, July 10, 2000 12:19 PM
> To: carter at qosient.com
> Subject: Re: racount
>
>
> Hangs head in shame ..... 'twas 1.8 with mods - there was also a cherry
tree
> I wish to add to the plea bargaining...
>
> However ;-)
>
> ../racount.181 -nr argus.07.06.11:00.gz
> racount.181: totrcds 101282 rcds 101269 pkts 0
> bytes 173961940
>
>
> ../racount.mods -nr argus.07.06.11:00.gz
> racount: totrcds 101282 rcds 101269 pkts 173961940
src
> bytes 2710310111 dst bytes 2652461504 total bytes
> 5362771615
>
> This on Solaris so the printf still needs changing - the point I was
trying
> to raise was that as argus is applied to larger and larger netflows it may
> become more of a problem. Peter's comment was useful in so far as Perl
tries
> to 'do the right thing'. I would guess that the most common argus
platforms
> will be the 32-bit *nix PCs although the wealthy few may have the odd DEC
> Alphas to use as crunch tools.
>
> Cheers
> Neil
>
>
>
>
> >Hey Neil,
> > Definitely seems like an overflow problem, but the racount() in
> >argus-1.8.1, however, has its counters defined as long long, so
> >this shouldn't be a problem. Are you using 1.8.1 or 1.8 or the 1.7
> >version of racount()?
> >
> >Carter
> >
> >-----Original Message-----
> >From: owner-argus at lists.andrew.cmu.edu
> >[mailto:owner-argus at lists.andrew.cmu.edu]On Behalf Of Peter Van Epp
> >Sent: Thursday, July 06, 2000 10:05 PM
> >To: argus at lists.andrew.cmu.edu
> >Subject: Re: racount
> >
> >
> >> i.e. the total bytes for the net is larger than the total for all nets
> >>
> >> I increased the format number from 9 to 12 in racount.c just to be sure
> >> there is not a simple format error. The data types are unisigned long.
> >>
> >> Just to confirm this I simply added fields to output the
total_src_bytes,
> >> total_dst_bytes before the grand total ---
> >>
> >> ../racount -nr argus.07.06.11:00.gz
> >> racount: totrcds 101282 rcds 101269 pkts 173961940
> >> bytes 2710310111 bytes 2652461504 bytes 1067804319
> >>
> >> so that was 5.1G .....
> >
> > And if a long is 32 bits as is standard (although perhaps not on one
> >of the 64 bit machines), you just overflowed, because 32 bits is around 4
> >gigs.
> >Which would leave you about a gig and a half after the over flow as you
are
> >seeing. I get around this by feeding the raw data in to perl which
happily
> >does
> >multiprecision something under the covers and keeps on chugging up to the
> >60 to 80 gigabyte level (I ran a test once to see if it had problems
while
> >I was chasing count descrepincies, I think I was up in the 100s of gigs
> >without problem). That is perhaps one good reason for at least some
thought
> >of a perl interface from the data as Russell suggested (that and indexing
> >arrays by almost anything ...) because it protects the unwary (such as
me).
> >One of the perl scripts that I posted long ago should do verify this for
> >you if you feed it the raw ra data.
> >
> >>
> >> Anyone got a LART I can borrow?
> >
> > If I knew what a LART was I might know if I have one ...
> >>
> >> Cheers
> >> Neil
> >>
> >
> >Peter Van Epp / Operations and Technical Support
> >Simon Fraser University, Burnaby, B.C. Canada
> >
> >
> >
>
>
>
More information about the argus
mailing list