[flow-tools] Large memory footprint of flow-stat -f11

[Mark Fullmer]

Each IP address will require 96 bytes of storage, so worst case 
each flow was a different IP this gives you about 308MB of RAM.

flow-report will only use 48 bytes if the pps and bps options are
turned off (-pps, -bps).

There is also a little bit of malloc overhead every 65536 allocations.

Buy more RAM :)  Motherboards which support 4GB of RAM are commonplace.

mark

On Sun, Jan 26, 2003 at 02:24:53PM +0100, Roland Rosenfeld wrote:
> Hi!
> 
> One of our customers seems to have got a Trojan, which scans half the
> world with 40 byte TCP packages.
> This results in a netflow file of 16MB size (15 minutes), which mainly
> contains these scans.  The file contains 3369300 flows, which isn't
> that much, but if I feed this netflow file into flow-stat -f11 (from
> flow-tools 0.62), the flow-stat process grows to 184MB.  I didn't
> already look into the code, but this process size seems to be a little
> too large to me.  Is this a memory leak or isn't it a good idea at all
> to use flow-stat -f11?
> 
> With the above 16MB file, this isn't a problem to me, but I used
> flow-cat to feed a hole day into flow-stat -f11, which resulted in
> growing flow-stat to 850MB before it began to swap, which brought the
> performance of flow-stat down (say: after 24 hours I killed the job
> because every 15 minute flow file took more than 6 hours (exponentally
> increasing) to process).
> 
> Tschoeeee
> 
>         Roland
> 
> _______________________________________________
> flow-tools@splintered.net
> http://www.splintered.net/sw/flow-tools