argus-clients informal survey
Chris Newton
newton at unb.ca
Thu Jun 21 13:00:25 EDT 2001
Ahh, ok, I see your direction now. No problem :)
My opinion on direction comes mostly from (in terms of high load stability)
the following. Our network is a 100 Mbit full duplex connection, that is very
abused... great for testing Argus. However, 100 Mbit is certainly not fast by
today's standards, and at these speeds, even with the fastest hardware I can
reasonably buy, Argus has troubles keeping up. This morning, Argus grew to
over 650 MB while trying to deal with an incoming DoS-style stream from two
remote IPs that lasted only a couple minutes. During this, Argus dumped out
30 MB logs files, per 30 seconds. It eventually got itself in a state where
it got pretty confused and _seemed_ to be then generating incorrect flows.
Stabilibty wise, Argus only now causes me problems during these times
(incoming high packet counts). All other stability issues are gone, it
appears. I have not reported this yet, as I was waiting to get a good copy of
the confused stream output that argus gets into (when it actually lives
through the attack, lots of times it crashes , not being able to dump out the
flow records (1 record per packet basically) fast enough). To give you an
idea of size (number of flows) ... I ran ra |wc -l, and got flowcount of
190,000 flows in 30 seconds. Basically, every packet sent by these two remote
goofs, became a flow to argus. If this one 'issue' could be resolved, I
believe Argus could survive quite well on large links.
Thanks for the comments :)
Chris
>===== Original Message From <carter at qosient.com> =====
>Hey Chris,
>Thanks for the reply. In your list of items to address
>your d) item is already being done. Clients pass filters
>to argus today, so that the filtering is all done by
>the monitor. And the e) item, selecting the feature set,
>is being done for response time data and jitter data,
>ethernet address reporting, user data capture, etc...
>Just need specifics on what to add in this area.
>
>I very much appreciate what you are saying with regard
>to the other items on your list, and if you were a paying
>customer, most of your "wish list" would be on my list
>of things to do. But, I'm not sure if your needs list
>is appropriate for the free software side of things.
>Lets discuss this, as the more the list contributes to
>where the project goes, the better things will be.
>
>I'm interested in making Argus a success, and putting
>resources into making Argus a success. Currently Argus,
>as a free piece of software, provides more basic information
>than any commercial monitor available. I'm not sure
>that the monitor is deficient, especially since there are
>more features in Argus today than are being used by the
>mailing list. But I also know that Argus isn't the be
>all and end all of monitors, so adding more features may
>be the best use of the very limited resources that I have,
>but I'm not sure.
>
>I'm pretty much of the opinion that the awareness of Argus
>and its ability to solve real problems for people is where
>the work needs to be done. Although we've got QoSient,
>Debian, and FreeBSD distributing Argus now, we don't have a
>HUGE following, like I think we could have. Its hard to
>remember that Argus-2.0 has really only been out for 3 months
>now, but Argus should be getting more attention than it has.
>
>Improving the monitor to the point where it not only
>survives DDOS attacks, but actually handles them as if
>they were traditional flows I think would be a breakthrough,
>rather than an incremental improvement on a piece of free
>software, but I could be wrong.
>
>I believe that I need to be doing what is needed to draw
>more people into Argus, to use it to solve their problems.
>I think that means more applications, rather than to
>continue to tweak the data generation itself.
>
>Any opinions?
>
>
>Carter
>
>Carter Bullard
>QoSient, LLC
>300 E. 56th Street, Suite 18K
>New York, New York 10022
>
>carter at qosient.com
>Phone +1 212 588-9133
>Fax +1 212 588-9134
>http://qosient.com
>
>> -----Original Message-----
>> From: Chris Newton [mailto:newton at unb.ca]
>> Sent: Wednesday, June 20, 2001 10:54 PM
>> To: argus; Carter Bullard
>> Subject: FWD: argus-clients informal survey
>>
>>
>> Hi Carter, here are my coments.
>>
>> First, I think you are doing great work!
>>
>> Second: I was quite happy to see the ratop addition, but I
>> believe, with
>> that, you have provided quite an array of methods for
>> retrieving information
>> from the argus log files. With a well defined interface to the argus
>> structures, possibly a perl module, and 1 client that could
>> print out anything
>> in the log file, I'd say that is very complete. It gives
>> everyone the tools
>> they need to do what ever analysis they need/want to do.
>>
>> I know your interest is in security, and others in
>> performance of the
>> monitoring capability. Having said that, _my_ vote (which is
>> obviously
>> strongly affected by my needs/interests) would be in seeing
>> more information
>> added to argus that it can track, and scaling changes to move
>> argus up so that
>> it is the hardware limits we run into, not software.
>>
>> Here are some ideas along those lines, off the top of my head:
>>
>>
>> a) the 'step down' ideas we all talked about a couple
>> months back, where
>> when argus is seeing a DoS attack (or something), it starts
>> 'doing less'. Ie:
>> says, forget the interpacket jitter information, forget the
>> super accurate
>> duration information...lets just get this stuff accounted for.
>>
>> b) when a DDoS type attack is seen, understand it, and not
>> try to generate a
>> flow record for each remote IP seen, when attacking a certain
>> host on a
>> network... tell argus to generate an aggregated record *at the argus*.
>>
>> c) when a standard DoS attack is seen, which you can know
>> by the unique
>> signature (flags/status) and sheer number of packets, do
>> aggregation at the
>> argus... dont tryand generate a flow (and, all the flow info
>> that goes along
>> with it).
>>
>> d) allow for *at the argus* filtering. IE: you connect a
>> sensor to link.
>> you then connect two remote clients to the argus server. It
>> would be nice if
>> the clients could tell argus what their filters are... and,
>> argus only
>> delivers across the network, the flows that they asked for.
>> this way, a
>> LARGE, powerful sensor handling a heavy link, could divide
>> out the analysis of
>> the flows to a number of boxes.
>>
>> e) allow for options to argus, to tell it what information
>> to try to record. For instance, maybe you want info x, y,
>> and z. bit, not a, b and c. To give
>> argus the most breathing room on larger/abused (thts me :))
>> links, we could
>> tell it up front what work it shouldnt even try to do.
>>
>> Hmm, thats all for now... If I think of anything else ... :)
>>
>> Chris
>>
>>
>> >===== Original Message From "Carter Bullard"
>> <carter at qosient.com> =====
>> Gentle people,
>> Just wanted to get some feedback on the
>> argus-clients package. If you've had any chance
>> to look at it, could you send your first impression,
>> comments? If its, "got no time" that's fine. If its
>> "I can't understand what's going on because there
>> are no man pages", that's fine. If its, "I wish
>> you would work on something else" then send that
>> ASAP. Any comments would be well received. I'm
>> trying to figure out where to apply limited
>> resources.
>>
>> Carter
>>
>> Carter Bullard
>> QoSient, LLC
>> 300 E. 56th Street, Suite 18K
>> New York, New York 10022
>>
>> carter at qosient.com
>> Phone +1 212 588-9133
>> Fax +1 212 588-9134
>> http://qosient.com
>>
>> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
>>
>> Chris Newton, Systems Analyst
>> Computing Services, University of New Brunswick
>> newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)
>>
>> "The best way to have a good idea is to have a lot of ideas."
>> Linus Pauling (1901 - 1994) US chemist
>>
>> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
>>
>> Chris Newton, Systems Analyst
>> Computing Services, University of New Brunswick
>> newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)
>>
>> "The best way to have a good idea is to have a lot of ideas."
>> Linus Pauling (1901 - 1994) US chemist
>>
>>
_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
Chris Newton, Systems Analyst
Computing Services, University of New Brunswick
newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)
"The best way to have a good idea is to have a lot of ideas."
Linus Pauling (1901 - 1994) US chemist
More information about the argus
mailing list