Processing speed of ra utilities

[Carter Bullard]

Hey Geoff,
   Looking at your samples, ragator() can definitely
do some bulk processing for you.  With a simple
ragator.conf such as:

 Flow  100 ip  *   *   *   *    *     200  0   0
 Model 200 ip  255.255.255.0  0.0.0.0   no no no

you can generate stats for all the source class-c nets.

 Flow  100 ip  *   *   *   *    *     200  0   0
 Model 200 ip  0.0.0.0 255.255.255.0    no no no

will get you all the dst class-c net stats, in one
pass of the data.  If you want the data sorted by
network, just pipe the output through rasort(),
and with our first example ragator.conf file, you won't
be interested in the dst addr, so give this a try:

ragator -f ragator.conf -r large-file.out -w - | \
     rasort -M saddr -s -dir -s -daddr

   All the ra* programs process argus data files
sequentially, reading each record and doing whatever
processing they are designed to perform, and yes
one performance bottle neck with the simple samples
provided in the argus-clients distribution is that
they really only do one thing for each pass
of the data.

   The ra* programs are really intended as examples
and if you want to speed things up, you should
write your own ra* program to process the data in
the most efficient way.

Carter


> -----Original Message-----
> From: owner-argus-info at lists.andrew.cmu.edu
> [mailto:owner-argus-info at lists.andrew.cmu.edu] On Behalf Of
> Geoff Powell
> Sent: Monday, September 01, 2003 9:06 PM
> To: argus-info at lists.andrew.cmu.edu
> Subject: Processing speed of ra utilities
>
>
> Hi all,
>
> I'm using scripts to do a lot of similar processes on the
> same argus data
> file (which is quite large), and I'm wondering if anyone
> knows of a way I
> can speed up the process, and reduce the time it takes for ra
> utilities
> to produce results.
>
> Some examples of the commands I'm doing:
> racount -n -r large-file.out - src net c.class.ip.1/32
> racount -n -r large-file.out - dst net c.class.ip.2/32
> racount -n -r large-file.out - src net c.class.ip.3/32
> ...all the way to 254/32.
>
> After that I might look at specific ports/ip protocols for
> each IP address
> in the c class.
>
> I'm guessing racount has to process each transaction? When the argus
> data file size is 50Mb+, even though the computer doing the
> processing is
> reasonably fast (Dual Xeon 1.5Ghz with 2gb of ram), each
> racount command
> usually takes around 30sec-1min.
>
> Is there way I can speed up the process, like running
> multiple racounts,
> using ragator or another application?
>
> Thanks for any help
>
> Regards,
> Geoff (geoff at lanrex.net.au)
>