ra reads argus file very slow

[Carter Bullard]

Hey Zi,
The only time I’ve seen ra() have problems reading and writing
data, to the level you report, is when one tries to do DNS
lookups to get the names of the IP addresses, instead of
dotted decimal notation.

I can read about 2G of flow data in about 65 secs, on a
standard machine, but I can cat() that file in about
2.5 secs, so your machine may not be performing as well
as you would want.

What version of argus and clients are you using??
Do you have a .rarc file in your home directory?
What does a line of ra() output look like ?

Carter



On Oct 30, 2013, at 6:34 PM, Zi Hu <zihu at usc.edu> wrote:

> Hi, Carter,
> 
> In my application, I need a simple tool to read what it is in the argus file, then output certain fields that I am interested in ascii format, such as srcip, dstip, sport, dport. protocol, .... 
> 
> I thought the command "ra" is what I need. However, I find it is very slow to read the argus data with "ra".  I did a small experiment: dump the same argus file (about 2G) with both "ra" and "cat".
> Using the "ra" command, it took me about 87 minutes to read the file, while it took only 40 seconds to dump it with "cat".  and also I notice that the memory keeps growing when I am running "ra".
> 
> zihu at proton:~$ time cat 2013-09-01-0700/temp/20130831-223000-hWukIYC-lander4.argus > temp.dat
> 
> real    0m39.490s
> user    0m0.027s
> sys     0m4.204s
> zihu at proton:~$ time ra -r 2013-09-01-0700/temp/20130831-223000-hWukIYC-lander4.argus -u > temp.dat
> 
> real    87m40.973s
> user    86m42.397s
> sys     0m56.256s
> zihu at proton:~$ 
> 
>  
> 
> So I guess "ra" does more than just reading the argus file, formatting and outputing the result.   Does "ra" keep track of flows in memory so that the memory keeps growing ?
> 
> If "ra" is not the right choice for my application, then what's the right command for this simple application? Or if we don't have such a tool, I am thinking of writing one by myself. Could you point me where to start?  Any suggestions are welcomed. 
> 
> 
> Thanks
> -Zi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20131030/f7a099e2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20131030/f7a099e2/attachment.bin>