argus-2.0.0 tuning

[Carter Bullard]

Hey Chris,
   Hmmm, my math must be off, but with all options on
the average record size may be near 228-256 bytes, and
of course if your capturing user data, then upwards of
400-500 bytes per record is a better number.

   One of the CMU machines that we're using is in the
same performance range as yours.  240MB processes
are the norm, they are handling around 85K to 100K
simultaneous flows, and generating near max record
throughput at peak.  The tuning we've done has eliminated
the load exits that you are seeing, but the patches that
I am doing now should make this much more stable under
sustained load, which is the goal.

   I should have the patches out by Friday, after testing
on the CMU machine for a while.

   Any chance you could test on a dual-processor machine?
That would eliminate your problems, after the tuning.

Carter

Carter Bullard
QoSient, LLC
300 E. 56th Street, Suite 18K
New York, New York  10022

carter at qosient.com
Phone +1 212 588-9133
Fax   +1 212 588-9134
http://qosient.com

> -----Original Message-----
> From: Chris Newton [mailto:newton at unb.ca]
> Sent: Wednesday, April 04, 2001 12:19 PM
> To: Argus (E-mail); Carter Bullard
> Subject: RE: argus-2.0.0 tuning
> 
> 
> Hi Carter and all,
> 
>   I would say, by guessing, that most newish IDE/ATA drives 
> could do 10 MB/s 
> at _least_ for short periods of time, and sustain near that 
> for long periods 
> (given it has the disk to itself).  Also, some of the setups 
> I am going to 
> test, will have a dedicated network channel for writing argus 
> records to 
> another machine... at least 100 Mbit.  So, I would _guess_ 
> you could aim for 
> upwards of 10 MB/s... quite the jump over .128 MB/s it is 
> currently limited 
> to.
> 
>   I am monitoring a very active and agressive network, that 
> has about 12 
> universities and companies on it, and monitored from 1 single 
> point, with 1 
> argus machine (P3, 256MB ram, 600 MB swap).  This network has 
> about 2 DoS 
> attacks per week... mainly due to the residence students.  
> Argus does not, 
> currently, like these attacks very much... often over running 
> the current 
> buffer settings, and, swallowing memory in huge gobs.  I have 
> seen argus 
> processes in excess of 280 MB in size, during an attack.  
> This, I'd guess from 
> your comments, that this is because it is not expunging 
> records to disk/port 
> as fast as it could be.
> 
>   I am wondering about the calculations you have though...  
> because, during 1 
> attack I had a 140 MB log file (I move it form argus.out to 
> argus{timestamp} 
> every 30 seconds).  The records in that log file, have lots 
> of the optional 
> output functions turned on (kitter... ICMP, so on)... but, 
> still, argus 
> managed to pump out 4.66 MB/s, if you take 140 MB/30 seconds. 
>  Argus did, 
> during that attack though, die. :)  That is the most recent 
> event I had mailed 
> you about.
> 
>   I am very interested in tuning this to at least be able to 
> deal with the 
> worst that can be thrown at you on a 100 Mbit, full duplex 
> pipe (for now), 
> and, larger later :)
> 
> Chris
> 
> >===== Original Message From <carter at qosient.com> =====
> >Gentle people,
> >   Argus-2.0.0 seems to be doing OK, the only real
> >issue has been in DDOS attacks, where it can get
> >overwhelmed.  I have had some good luck with changing
> >some internal variables, and removing a syslog() call
> >in the code, and so tuning definitely has its benefits.
> >
> >   Default 2.0.0 is configured for a maximum record output
> >of 1024 records per second, and have a buffer capacity of
> >8 seconds. With an average record size of 128 bytes,
> >this is just 1Mbps (128KB) of output that Argus can
> >generate.  This seems tooooooo low.
> >
> >   We should engineer for a target max output bit rate
> >for argus.  Do numbers like 10-20Mbps seem reasonable?
> >Based on your logs, what kind of argus load are you
> >generating?   What's the best IDE throughput for writing?
> >
> >Thanks!!
> >
> >Carter
> >
> >
> >Carter Bullard
> >QoSient, LLC
> >300 E. 56th Street, Suite 18K
> >New York, New York  10022
> >
> >carter at qosient.com
> >Phone +1 212 588-9133
> >Fax   +1 212 588-9134
> >http://qosient.com
> 
> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
> 
> Chris Newton, Systems Analyst
> Computing Services, University of New Brunswick
> newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20010404/f265d078/attachment.html>