Ok, really, the last one.....ragator question

[Carter Bullard]

Hey Wozz,
   I sent a description of the timer strategy for Argus
in a previous mail, but ragator() and argus() both have
timeout behaviors that are different.  The idea is that
ragator() should have a longer view of a flow than argus(),
so that it can merge multiple reports for the same flow
together.

   Merging records together is a two pass problem.  In
order to do direction correction on activity for flows
that have been timed out earlier by argus, all the fields
have to be intact.  And you don't know how long you need
to hold the original flow descriptors, because you don't
know when the error record is going to show up.  Say for
Microsoft TCP stacks, which seems to send stray FINs for
as long as 1-2 hours after the flows have expired, it's a
hard value to nail down.

   When you aggregate using a modified flow model, we
actually change the flow descriptions in the argus records
themselves, so you can only do this step after you're done
with the previous pass.

   Now ragator() could support its own idle timers, which
would allow you to pass on some records while reading in
others, that would be more efficient allowing you to
pipeline the task a bit better.  I could add that.

   But, with the given implementations, your incantation is
the only way.

Carter

Carter Bullard
QoSient, LLC
300 E. 56th Street, Suite 18K
New York, New York  10022

carter at qosient.com
Phone +1 212 588-9133
Fax   +1 212 588-9134
http://qosient.com

> -----Original Message-----
> From: Wozz [mailto:wozz+argus at wookie.net] 
> Sent: Wednesday, November 14, 2001 8:57 PM
> To: Carter Bullard
> Cc: argus-info at lists.andrew.cmu.edu
> Subject: Re: Ok, really, the last one.....ragator question
> 
> 
> On Wed, Nov 14, 2001 at 07:22:32PM -0500, Carter Bullard wrote:
> > 
> > Hey Wozz,
> >    The issue is the '?' in the direction indicator.
> > Because argus is absolutely stateful, and it is saying
> > that it doesn't know precisely who the source or the destination is 
> > (because it didn't see a SYN or a SYN_ACK before it saw the FIN).  
> > With the '?' its saying that the src and dst assignments may not be 
> > reliable.
> > 
> >    What probably happened is that argus timed the original 
> flow out, 
> > before the stray FIN came in, and because there is no flow 
> cache, it 
> > treats the lone FIN without any context.
> > 
> >    There is a solution.  First pass your traffic through 
> ragator with 
> > no configuration.  It will correct the '?'. If there was a 
> flow that 
> > this FIN belongs to, and it gets loaded into ragator before the FIN 
> > record is loaded, then it will discover the correct direction and 
> > merge the FIN report into the parent flow.
> > 
> 
> Ah ha!
> 
> I get it now.  It appears to work correctly now without all the extra
> flows.   What determines when a flow gets timed out?  Is that 
> in argus or
> in my ragator config?  The command line I'm using now is:
> 
> ragator -r * -w - host a.b.c.d |ragator -w - -f fmodel.conf 
> -r - | rasort -s startime -n -r -
> 
> Is there any way to shorten this?  Is there some way to have 
> make ragator do its default aggregation first, then the 
> defined flows, without running ragator twice?
> 
> 
>