argus-clients informal survey

[Chris Newton]

Yes, I believe it is modeler output, but also, modeler work.  Once we hit the 
high water mark, we dont want to be calculating all that flow information for 
all those singular packets that look like a unique flows...we want to count 
them in 1 flow, and only do the flow calculations/countings/timings for that 1 
flow.  This should save _tons_ of work.  In the episode from earlier, in 30 
seconds, I saw 190,000 flows.  If the high water mark was 5000 flows to an IP 
(dont know if 5K is a reasonable # or not), then all the work done for the 
other 185,000 flows would never have occurred... since, *bang* you hit 5000 
flows, and everything else hitting that IP from then on (until it drops below 
the water mark again) lands in 1 flow record.  poof, there goes the work of 
the other 185,000 flows out the window, replaced by the time it takes to do 
that work on 1 flow.  This is what I see as the _major_ goal of this 
work/change to argus.  Since, that will allow it to survive under pressure.

Chris

>===== Original Message From <carter at qosient.com> =====
>The only real issue with per host flow tracking is
>the number of hosts that are active.  It can be pretty
>high.  I've got a few tricks up my sleeve for that, so
>that the task may not be as terrible as it seems.
>
>I believe that the real goal is to minimize modeler
>output, rather than modeler memory usage.  If this
>is true, then it may not be a major problem.
>
>
>Carter
>
>Carter Bullard
>QoSient, LLC
>300 E. 56th Street, Suite 18K
>New York, New York  10022
>
>carter at qosient.com
>Phone +1 212 588-9133
>Fax   +1 212 588-9134
>http://qosient.com
>
>> -----Original Message-----
>> From: owner-argus-info at lists.andrew.cmu.edu
>> [mailto:owner-argus-info at lists.andrew.cmu.edu] On Behalf Of
>> Chris Newton
>> Sent: Thursday, June 21, 2001 6:44 PM
>> To: argus-info; Russell Fulton
>> Subject: RE: argus-clients informal survey
>>
>>
>> Yes, this is the sort of solution I was imagining, however, I
>> have seen cases
>> where the single direction concept (drop remote side, keep
>> locals) wouldn't be
>> enough.  We have had a couple cases where a trojan got
>> installed in some unix
>> lab run by a department that is less interested in security
>> then mine is (the
>> main computing center).  That lab was on a student network,
>> with a netmask of
>> 255.255.240.0.  The intersting thing about this trojan, is
>> that it used this
>> netmask it found on the machines as a guide to determine the
>> range of possible
>> addresses it could spoof on this network, and have them all
>> be valid.  So,
>> boom, an outgoing attack from 'thousands' of machines on our
>> student network,
>> aimed at some poor soul out there in the big merky internet,
>> which our routers
>> gladly sent on their merry way, since they were all valid IP
>> addresses,
>> routes, TTLs, ...
>>
>>   So, I think the highwater mark would need to work both
>> ways...  maybe if a
>> count was added to each IP address that argus was tracking...
>>
>>   So, if 131.202.160.212 local is being attacked, each new
>> flow you create,
>> that is associated with that IP, increment the counter.
>> Reset that counter
>> each time you print a MAR record.  If in that interval of
>> time of the MAR
>> records, the IP has more then xxx records associated with
>> it... start adding
>> all info to a single flow record.  And, maybe in this record,
>> keep a count of
>> the number of unique remote IP addresses you are including in this
>> aggregation.  Do this logic with both sides (remote, local),
>> and I think argus
>> would be stable at these high packet count levels.
>>
>> Chris
>>
>>
>>
>>
>> >===== Original Message From Russell Fulton <r.fulton at auckland.ac.nz>
>> >===== On Thu, 21 Jun 2001 14:00:25 -0300 Chris Newton
>> <newton at unb.ca>
>> >wrote:
>> >
>> >> 190,000 flows in 30 seconds.  Basically, every packet sent
>> by these
>> >> two
>> remote
>> >> goofs, became a flow to argus.  If this one 'issue' could be
>> >> resolved, I believe Argus could survive quite well on large links.
>> >
>> >The way NeTraMet (to give it it's propper capitalization ;-)
>> deals with
>> >this is to have an 'fall back rule set' and settable 'high
>> water mark'.
>> >Once the number of flows passes the 'high water mark'
>> netramet changes
>> >to the fallback ruleset.  For our campus accounting this involves
>> >dumping the remote address completely so that we have just
>> one flow per
>> >local address.  Now netramet is rather a different beast to
>> argus and I
>> >am not sure if this concept can be adapted but it is worth a though.
>> >
>> >[before you ask there is also a 'low water mark' that sets the point
>> >where the meter flips back to the normal ruleset]
>> >
>> >The question is what information could argus discard that
>> would reduce
>> >the numbre of flows in these sorts of circumstances?  In your
>> >particular attacks aggregation the destination address and
>> ports would
>> >do the trick.
>> >
>> >Carter is the flow aggregation working in the server now?  I
>> know you
>> >planned to do it.  If so the netramet model would work
>> without change,
>> >when the meter gets stressed you swap to an alternative flow model.
>> >What would be really neat would be able to say "For source
>> IP addresses
>> >with over X active flows aggregate their destination peer and port
>> >addresses.
>> >
>> >Russell Fulton, Computer and Network Security Officer
>> >The University of Auckland,  New Zealand
>>
>> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
>>
>> Chris Newton, Systems Analyst
>> Computing Services, University of New Brunswick
>> newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)
>>
>> "The best way to have a good idea is to have a lot of ideas."
>> Linus Pauling (1901 - 1994) US chemist
>>
>>

_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/

Chris Newton, Systems Analyst
Computing Services, University of New Brunswick
newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)

"The best way to have a good idea is to have a lot of ideas."
Linus Pauling (1901 - 1994) US chemist