ramon question

Tue Oct 31 14:00:25 EST 2006

Merci Carter,

I ran a couple of tests and rasplit works like a charm. 

One last last question. To get top tcp/udp ports with ra 2.0.6 I was doing this:

/usr/local/bin/ra -r /var/log/argus/argus.out -w - | /usr/local/bin/ramon -u -M svc | egrep -v "(arp|llc|decr)" | head -n 10
1162317564  udp 1985             1564     0         96676        0
1162319601  tcp ssh              151      115       13798        20119
1162317570  udp ntp              39       30        3510         2700
1162317601  tcp 51404            13       13        2463         858
1162317677 icmp                  12       0         1176         0
1162319599 icmp                  9        0         1080         0
1162319599  udp netbios-ns       9        0         828          0
1162319604  udp domain           2        2         169          223

How can I get the same data using racluster ?

Thanks in advance.

Cheers,

Real.
(real.melancon at videotron.ca)

----- Message d'origine -----
De: Carter Bullard <carter at qosient.com>
Date: Lundi, Octobre 30, 2006 10:11 am
Objet: Re: Re : Re: [ARGUS] ramon question
À: real.melancon at videotron.ca
Cc: argus-info at lists.andrew.cmu.edu

> Allo Réal,
> So, you should be thinking about programs like rasplit(), which 
> can 
> effectively be
> used to build a filesystem of flow records. Currently my 
> suggestion 
> is to build a
> probe/yearly/monthly/daily/'every 5 minute' filesystem. This is 
> 
> reasonable for
> Unix filesystems, as the number of files in any given directory 
> doesn't get
> bigger than 288 files, which does pretty good (more files, the 
> slower 
> the directory
> parsing). Even for high speed links the size of the 5 minute 
> files 
> is reasonable,
> and so searching, indexing, whatever becomes bounded.
> 
> rasplit -r file -M time 5m -w 
> $srcid/%Y/%m/%d/argus.%Y.%m.%d.%H.% 
> M.%S
> or
> rasplit -S argi -M time 5m -w 
> $srcid/%Y/%m/%d/argus.%Y.%m.%d.%H.% 
> M.%S
> 
> you can run this type of program out of a cron job that runs 
> however 
> often as you
> like, or you can use rasplit() to be your actual collector, 
> connecting directly to
> probes, or to a radium(), that is connected to a large number of 
> 
> probes. If you
> want to try to provide real-time on-demand reports, then the 
> 'rasplit 
> () as a
> collector' is the best approach.
> 
> designing perl scripts to take a time range and find the 5 
> minute 
> files to search
> is pretty easy, and i have MySQL DB based programs that will 
> create time
> oriented indexes that allow you to find stuff very fast (trying 
> to 
> figure out how
> to share this software).
> 
> I would recommend that you start to think this way, and have a 
> good 
> dialog on
> the mailing list to try to design a very good archive.
> 
> Hope this is helpful.
> 
> Carter
> 
> 
> On Oct 27, 2006, at 1:13 PM, real.melancon at videotron.ca wrote:
> 
> > Thanks a lot for the informations Carter. I immediately 
> replaced my 
> > scripts and can now query Top Talkers & Listeners as in 
> version 2 
> > (among other things).
> >
> > I have one more question for you though:
> >
> > We need a long term solution (day/week/month) to collect Argus 
> 
> > data, which would then be used to define a QOS policy. I am 
> > thinking mostly of Layer 4 informations. (Right now we rotate 
> the 
> > argus.out file every hour (using argusarchive), because file 
> was 
> > growing too quickly and queries were getting longer and longer.)
> >
> > So. I would need your advices for a long-term collect of 
> Layer4 
> > informations we could later analyze (a little like MRTG/RRD)
> >
> > I first thought of hacking your Perl script (ragraph), which 
> builds 
> > on-the-fly RRD databases, but was wondering if you had 
> something 
> > more elegant.
> >
> > Thanks in advance!
> >
> >
> > ----- Message d'origine -----
> > De: carter at qosient.com
> > Date: Jeudi, Octobre 26, 2006 7:13 am
> > Objet: Re: [ARGUS] ramon question
> > À: real.melancon at videotron.ca
> >
> > > Hey Réal,
> > > To do a Layer 3 matrix using racluster:
> > > racluster -nu -m matrix -r file
> > >
> > > And if you want top 20 for bytes:
> > >
> > > racluster -m matrix -r file -w - |
> > > rasort -m bytes -w - | ra -nu -N 20
> > >
> > > The '-m matrix' option will modify each input record, flipping
> > > the addresses and metrics, to get the saddr to be the lesser of
> > > the 2 addresses, so that when the records are aggregated, you
> > > get a single record for each " a <-> b" pair, regardless of the
> > > direction of the set of flows.
> > >
> > > Using '-m saddr daddr' you will get a matrix, but it will be
> > > direction sensitive, so that you can get 2 records per address
> > > pair, " a <-> b" and " b <-> a".
> > >
> > > The '-M rmon' is not going help here at all, as it designed to
> > > convert bi-directional data to unidirectional data, so that you
> > > can get metrics per object. You would use this option to
> > > generate data for just "a", like a single ethernet address,
> > > single mac address, port, whatever. Since you want data for 2
> > > objects, " a & b ", the '-M rmon' option will just double all
> > > the metrics. Not good.
> > >
> > > Carter
> > >
> > > Carter Bullard
> > > QoSient LLC
> > > 150 E. 57th Street Suite 12D
> > > New York, New York 10022
> > > +1 212 588-9133 Phone
> > > +1 212 588-9134 Fax
> > >
> > > -----Original Message-----
> > > From: real.melancon at videotron.ca
> > > Date: Wed, 25 Oct 2006 20:42:42
> > > To:argus-info at lists.andrew.cmu.edu
> > > Subject: [ARGUS] ramon question
> > >
> > > Hello List,
> > >
> > > With Argus 2.0.6, I was using this command line to get Top
> > > Talkers & Listeners:
> > >
> > > #> ra -n -u -w - -r /var/log/argus/argus.out | ramon -n -u -
> M Matrix
> > >
> > > Which would display something like:
> > >
> > > Time SourceIP DestIP Spkts Dpkts Sbytes Dbytes (e.g.)
> > > 1161806363 10.5.192.250 10.5.29.71 214
> > > 214 19260 19260
> > > 1161806384 10.5.29.71 10.5.29.65 31
> > > 31 1302 1860
> > >
> > > Now, It has been replaced by racluster. Which is much more
> > > flexible. But I still can't figure out how to
> > > display the informations the same way. I tried this:
> > >
> > > racluster -M rmon -m saddr daddr -r /var/log/argus/argus.out 
> -w -
> > > - ip | rasort -m bytes -s ltime saddr daddr spkts dpkts sbytes
> > > dbytes | head -n 10
> > >
> > > 10-25-06 20:30:20.235473 10.6.104.192
> > > 10.6.110.73 81041 73984 32471550 11648835
> > > 10-25-06 20:30:20.235473 10.6.110.73
> > > 10.6.104.192 73984 81041 11648835 32471550
> > > 10-25-06 20:35:50.308809 10.6.104.200
> > > 10.6.110.133 12142 16005 5253886 16855338
> > > 10-25-06 20:35:50.308809 10.6.110.133
> > > 10.6.104.200 16005 12142 16855338 5253886
> > >
> > > But every line is duplicated (not exactly but display redundant
> > > informations) since racluster gives me both directions 
> flows. Is
> > > there any workaround ?
> > >
> > > Any help is welcomed.
> > >
> > > Real Melanson.
> > >
> > >
> > > ____________________________
> > > Réal Melançon
> >
> > ____________________________
> > Réal Melançon
> >
> 
> 
> 
> 
> 

____________________________
Réal Melançon