ra reads argus file very slow
elof2 at sentor.se
elof2 at sentor.se
Tue Nov 5 11:46:28 EST 2013
Just as a comparison, I just ran this on a 1.0 GB argus file:
# time ra -nr argus-20131103.09.log > /dev/null
I now cat a new file (not in any file cache) that is also 1.0 GB in size:
# time (sync; cat argus-20131103.10.log > /dev/null; sync)
This is on a FreeBSD 9.1 amd64 on an ordinary intel server that is pretty
loaded at the time of the test.
On Wed, 30 Oct 2013, Zi Hu wrote:
> Hi, Carter,
> In my application, I need a simple tool to read what it is in the argus
> file, then output certain fields that I am interested in ascii format, such
> as srcip, dstip, sport, dport. protocol, ....
> I thought the command "ra" is what I need. However, I find it is very slow
> to read the argus data with "ra". I did a small experiment: dump the same
> argus file (about 2G) with both "ra" and "cat".
> Using the "ra" command, it took me about 87 minutes to read the file, while
> it took only 40 seconds to dump it with "cat". and also I notice that the
> memory keeps growing when I am running "ra".
> zihu at proton:~$ time cat
> 2013-09-01-0700/temp/20130831-223000-hWukIYC-lander4.argus > temp.dat
> real 0m39.490s
> user 0m0.027s
> sys 0m4.204s
> zihu at proton:~$ time ra -r
> 2013-09-01-0700/temp/20130831-223000-hWukIYC-lander4.argus -u > temp.dat
> real 87m40.973s
> user 86m42.397s
> sys 0m56.256s
> zihu at proton:~$
> So I guess "ra" does more than just reading the argus file, formatting and
> outputing the result. Does "ra" keep track of flows in memory so that the
> memory keeps growing ?
> If "ra" is not the right choice for my application, then what's the right
> command for this simple application? Or if we don't have such a tool, I am
> thinking of writing one by myself. Could you point me where to start? Any
> suggestions are welcomed.
More information about the argus