Tracking source/destination application bytes

Tue Apr 17 13:49:55 EDT 2007

Carter,

I truly agree with this :) -

Well the biggest problem with argus is the lack of documentation, howeverthe
lack of documentation is really the result of the large problem set that
argus can fill.  You can't write documentation to solve all the worlds
problems, if the world doesn't know that there are problems to solve

Yeah, I'm trying to swallow your explanation here as it's midnight here ;)

I think both ratio will definitely help, we can easily identify either
producer and consumer with it but need some tinkering when working on
different kind of protocols, as long as it is adjustable that would be good
-

sd appbytes:total appbytes
sappbytes:dappbytes

Also a possible candidate is the src and dst appbyte to total byte ratio,
since attacks/problems generally impact the efficiency of transport, and
this ratio is a direct measure of efficiency.

Yes, not only that but also -

sd bytes:total bytes

I think the ratio can actually helps to understand the connections in the
flow easily especially with the help of the connection state.

On 4/17/07, Carter Bullard <carter at qosient.com> wrote:
>
> Well the biggest problem with argus is the lack of documentation, howeverthe
> lack of documentation is really the result of the large problem set that
> argus can fill.  You can't write documentation to solve all the worlds
> problems, if the world doesn't know that there are problems to solve ;o)
>
> So, here is a question that should come up.  Is the "src appbyte ratio"
> a better metric than "src dst appbyte ratio" when trying to describe the
> producer/consumer ?  Of course that begs for a definiition:
>
>    src appbyte ratio =  sappbytes / total appbytes;
>        appbyte ratio =  sappbytes / dappbytes;
>
>
> I like the simple appbyte ratio so that "> 1.0" is a producer
> and "< 1.0" is a consumer, because you get a steep transition
> even with simple rpc style program models.  The approach
> does need some refinement as most protocols do have overhead.
> Depending on the type of service, and the type of negotiation that is
> required at service establishment, say authentication, or authorization,
> or option negotiation or whatever, you may need to adjust the byte values
> to actually know what the real service bytes are, but the idea should be
> that the amount of service data should be higher than the service
> establishment load, you would think.
>
> This metric is also very sensitive to detecting service failure.
>
> The src appbyte ratio gives you some of the same notions,  "> 0.5"
> is a producer and "< 0.5" is a consumer, but my experience is that
> the deviation from 0.5 is not dramatic as you move from single rpc
> style programs that have little input and little output, to true push
> and pull data program models.  Sensitivity is of course a double
> edge sword, so to speak.
>
> Also a possible candidate is the src and dst appbyte to total byte ratio,
> since attacks/problems generally impact the efficiency of transport, and
> this ratio is a direct measure of efficiency.
>
>
> Carter
>
>
>
> On Apr 17, 2007, at 10:57 AM, CS Lee wrote:
>
> Carter,
>
> That's exactly what i'm trying to look at, and the more I use argus the
> more i feel adventurous.
>
> I will try the example you have shown me. However I believe most people
> who uses argus has this feeling, sometimes we really don't know which tools
> for which job, I'm now looking into all the man page to get the exact idea(I
> think i'm not fully utilize argus yet for its power features). And that's
> why I raise a lot of questions in mailing lists :)
>
> Excellent job, Carter!
>
> On 4/17/07, Carter Bullard <carter at qosient.com> wrote:
> >
> > Well don't stop there!!!!  You're just starting to get interested inapplication
> > behavior and baselining.   What does it really take
> > to behaviorally describe an application?  You're going after
> > push vs pull, or rather producer vs consumer.   This is a great
> > starting point.
> >
> > To start you will need, IMHO, a list of servers, which implies
> > a list of services.  I have found it reasonable to use rasplit()
> > for this function:
> >
> >    rasplit -R data -M time 1d -w servers/\$daddr/argus.%Y.%m.%d.%H.%M.%S
> > \
> >            - \( tcp and syn and synack \) or \( not tcp and con \)
> >
> > This will generate a set of data where all the servers are in a single
> > directory, broken down by day.  The ( tcp and ... ) filter will give
> > you 'correct' tcp flows, so you can start building your client/server
> > data.  The ( not tcp ... ) filter will give you all the other protocols,
> > udp, arp, whatever, where there is a response, which is very
> > important to understanding if there really is a service.  You can
> > be much more clever and have additional filters for multicast
> > data, etc...., but this is a good starting point.
> >
> > Once you have your data set, you'll probably want to racluster()
> > the data, to make sure that all the flows are consolidated:
> >
> >    racluster -M replace -R servers -V
> >
> > Then you should have a decent set of data as a starting point.  If you
> > find
> > that packet ratios and application byte ratios are good for your
> > analysis, I can add these as metrics to print.
> >
> > Carter
> >
> >
> > On Apr 17, 2007, at 1:14 AM, CS Lee wrote:
> >
> > Carter,
> >
> > Thanks, it seems I can do this -
> >
> > ragraph sappbytes dappbytes dport -M 1s -r file.arg -
> >
> > The reason why I want to do this is that I can easily spot people
> > whether he is doing http normal surfing or he is downloading file via http,
> > as if the normal surfing used to have bigger src appbytes comparing to the
> > http download src appbytes. It's more of for fun but it does the work very
> > well for me now. Thanks!
> >
> > For protocol breakdown, I think it is more of giving overview or big
> > picture of the transaction in the network instead. Cheers.
> >
> >
> > On 4/17/07, Carter Bullard <carter at qosient.com> wrote:
> > >
> > >  Hmmmmm, this is pretty easy to do, but what are you really asking
> > > for?Do you want something to "know" what the src/dst byte ratio for
> > > each
> > > application client/host pair is suppose to be and notify you if
> > > somehow
> > > it changes?  Again this is very easy to do, in say perl, but what/how
> > > would
> > > you want such a program to be structured?
> > >
> > > Hmmmm, graphing based on protocol is very easy:
> > >
> > >    ragraph sbytes dbytes proto -M time 5m -r daily.files
> > >
> > > Of course if you want something more detailed:
> > >
> > >    ragraph sbytes dbytes proto -M time 5s -r daily.files
> > >
> > > Carter
> > >
> > >
> > > On Apr 16, 2007, at 11:47 AM, CS Lee wrote:
> > >
> > > Hey people,
> > >
> > > I'm looking for a way to track the changes of source and destination
> > > application bytes over time from the flow(for example I can easily look at
> > > whether people are either doing http surfing or http file transfer. I have
> > > tried few combinations and has no success so I guess it would be good to
> > > turn into mailing lists. Is that possible to do it in argus way?
> > >
> > > Another thing is that I think a lot of people want a Protocol
> > > Breakdown in daily basis, I know racluster can do it when combining with
> > > rasort. But is that possible to generate the graph for all the Protocol
> > > breakdown by using ragraph, or our alternative is we still have to use it
> > > with excel to do it(that's what i do for now).
> > >
> > > Thanks.
> > >
> > > --
> > > Best Regards,
> > >
> > > CS Lee<geekooL[at]gmail.com>
> > >
> > >
> > >
> >
>
>
> --
> Best Regards,
>
> CS Lee<geekooL[at]gmail.com>
>
>
> Carter Bullard
> CEO/President
> QoSient, LLC
> 150 E. 57th Street Suite 12D
> New York, New York 10022
>
> +1 212 588-9133 Phone
> +1 212 588-9134 Fax
>
>
>

-- 
Best Regards,

CS Lee<geekooL[at]gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20070418/1c600501/attachment.html>