ramon question

Mon Oct 30 10:11:33 EST 2006

Allo Réal,
So, you should be thinking about programs like rasplit(), which can  
effectively be
used to build a filesystem of flow records.  Currently my suggestion  
is to build a
probe/yearly/monthly/daily/'every 5 minute' filesystem.  This is  
reasonable for
Unix filesystems, as the number of files in any given directory  
doesn't get
bigger than 288 files, which does pretty good (more files, the slower  
the directory
parsing).  Even for high speed links the size of the 5 minute files  
is reasonable,
and so searching, indexing, whatever becomes bounded.

    rasplit -r file -M time 5m -w $srcid/%Y/%m/%d/argus.%Y.%m.%d.%H.% 
M.%S
or
    rasplit -S argi -M time 5m -w $srcid/%Y/%m/%d/argus.%Y.%m.%d.%H.% 
M.%S

you can run this type of program out of a cron job that runs however  
often as you
like, or you can use rasplit() to be your actual collector,  
connecting directly to
probes, or to a radium(), that is connected to a large number of  
probes.   If you
want to try to provide real-time on-demand reports, then the 'rasplit 
() as a
collector' is the best approach.

designing perl scripts to take a time range and find the 5 minute  
files to search
is pretty easy, and i have MySQL DB based programs that will create time
oriented indexes that allow you to find stuff very fast (trying to  
figure out how
to share this software).

I would recommend that you start to think this way, and have a good  
dialog on
the mailing list to try to design a very good archive.

Hope this is helpful.

Carter

On Oct 27, 2006, at 1:13 PM, real.melancon at videotron.ca wrote:

> Thanks a lot for the informations Carter. I immediately replaced my  
> scripts and can now query Top Talkers & Listeners as in version 2  
> (among other things).
>
> I have one more question for you though:
>
> We need a long term solution (day/week/month) to collect Argus  
> data, which would then be used to define a QOS policy. I am  
> thinking mostly of Layer 4 informations. (Right now we rotate the  
> argus.out file every hour (using argusarchive), because file was  
> growing too quickly and queries were getting longer and longer.)
>
> So. I would need your advices for a long-term collect of Layer4  
> informations we could later analyze (a little like MRTG/RRD)
>
> I first thought of hacking your Perl script (ragraph), which builds  
> on-the-fly RRD databases, but was wondering if you had something  
> more elegant.
>
> Thanks in advance!
>
>
> ----- Message d'origine -----
> De: carter at qosient.com
> Date: Jeudi, Octobre 26, 2006 7:13 am
> Objet: Re: [ARGUS] ramon question
> À: real.melancon at videotron.ca
>
> > Hey Réal,
> > To do a Layer 3 matrix using racluster:
> > racluster -nu -m matrix -r file
> >
> > And if you want top 20 for bytes:
> >
> > racluster -m matrix -r file -w - |
> > rasort -m bytes -w - | ra -nu -N 20
> >
> > The '-m matrix' option will modify each input record, flipping
> > the addresses and metrics, to get the saddr to be the lesser of
> > the 2 addresses, so that when the records are aggregated, you
> > get a single record for each " a <-> b" pair, regardless of the
> > direction of the set of flows.
> >
> > Using '-m saddr daddr' you will get a matrix, but it will be
> > direction sensitive, so that you can get 2 records per address
> > pair, " a <-> b" and " b <-> a".
> >
> > The '-M rmon' is not going help here at all, as it designed to
> > convert bi-directional data to unidirectional data, so that you
> > can get metrics per object. You would use this option to
> > generate data for just "a", like a single ethernet address,
> > single mac address, port, whatever. Since you want data for 2
> > objects, " a & b ", the '-M rmon' option will just double all
> > the metrics. Not good.
> >
> > Carter
> >
> > Carter Bullard
> > QoSient LLC
> > 150 E. 57th Street Suite 12D
> > New York, New York 10022
> > +1 212 588-9133 Phone
> > +1 212 588-9134 Fax
> >
> > -----Original Message-----
> > From: real.melancon at videotron.ca
> > Date: Wed, 25 Oct 2006 20:42:42
> > To:argus-info at lists.andrew.cmu.edu
> > Subject: [ARGUS] ramon question
> >
> > Hello List,
> >
> > With Argus 2.0.6, I was using this command line to get Top
> > Talkers & Listeners:
> >
> > #> ra -n -u -w - -r /var/log/argus/argus.out | ramon -n -u -M Matrix
> >
> > Which would display something like:
> >
> > Time SourceIP DestIP Spkts Dpkts Sbytes Dbytes (e.g.)
> > 1161806363      10.5.192.250        10.5.29.71 214
> > 214       19260        19260
> > 1161806384        10.5.29.71        10.5.29.65 31
> > 31        1302         1860
> >
> > Now, It has been replaced by racluster. Which is much more
> > flexible. But I still can't figure out how to
> > display the informations the same way. I tried this:
> >
> > racluster -M rmon -m saddr daddr -r /var/log/argus/argus.out -w -
> > - ip | rasort -m bytes -s ltime saddr daddr spkts dpkts sbytes
> > dbytes | head -n 10
> >
> > 10-25-06 20:30:20.235473       10.6.104.192
> > 10.6.110.73    81041    73984     32471550     11648835
> > 10-25-06 20:30:20.235473        10.6.110.73
> > 10.6.104.192    73984    81041     11648835     32471550
> > 10-25-06 20:35:50.308809       10.6.104.200
> > 10.6.110.133    12142    16005      5253886     16855338
> > 10-25-06 20:35:50.308809       10.6.110.133
> > 10.6.104.200    16005    12142     16855338      5253886
> >
> > But every line is duplicated (not exactly but display redundant
> > informations) since racluster gives me both directions flows. Is
> > there any workaround ?
> >
> > Any help is welcomed.
> >
> > Real Melanson.
> >
> >
> > ____________________________
> > Réal Melançon
>
> ____________________________
> Réal Melançon
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20061030/e4fe1f9a/attachment.html>