Tracking source/destination application bytes

Tue Apr 17 11:35:21 EDT 2007

Well the biggest problem with argus is the lack of documentation,  
however
the lack of documentation is really the result of the large problem  
set that
argus can fill.  You can't write documentation to solve all the worlds
problems, if the world doesn't know that there are problems to solve ;o)

So, here is a question that should come up.  Is the "src appbyte ratio"
a better metric than "src dst appbyte ratio" when trying to describe the
producer/consumer ?  Of course that begs for a definiition:

    src appbyte ratio =  sappbytes / total appbytes;
        appbyte ratio =  sappbytes / dappbytes;

I like the simple appbyte ratio so that "> 1.0" is a producer
and "< 1.0" is a consumer, because you get a steep transition
even with simple rpc style program models.  The approach
does need some refinement as most protocols do have overhead.
Depending on the type of service, and the type of negotiation that is
required at service establishment, say authentication, or authorization,
or option negotiation or whatever, you may need to adjust the byte  
values
to actually know what the real service bytes are, but the idea should be
that the amount of service data should be higher than the service
establishment load, you would think.

This metric is also very sensitive to detecting service failure.

The src appbyte ratio gives you some of the same notions,  "> 0.5"
is a producer and "< 0.5" is a consumer, but my experience is that
the deviation from 0.5 is not dramatic as you move from single rpc
style programs that have little input and little output, to true push
and pull data program models.  Sensitivity is of course a double
edge sword, so to speak.

Also a possible candidate is the src and dst appbyte to total byte  
ratio,
since attacks/problems generally impact the efficiency of transport, and
this ratio is a direct measure of efficiency.

Carter

On Apr 17, 2007, at 10:57 AM, CS Lee wrote:

> Carter,
>
> That's exactly what i'm trying to look at, and the more I use argus  
> the more i feel adventurous.
>
> I will try the example you have shown me. However I believe most  
> people who uses argus has this feeling, sometimes we really don't  
> know which tools for which job, I'm now looking into all the man  
> page to get the exact idea(I think i'm not fully utilize argus yet  
> for its power features). And that's why I raise a lot of questions  
> in mailing lists :)
>
> Excellent job, Carter!
>
> On 4/17/07, Carter Bullard <carter at qosient.com> wrote:
> Well don't stop there!!!!  You're just starting to get interested in
> application behavior and baselining.   What does it really take
> to behaviorally describe an application?  You're going after
> push vs pull, or rather producer vs consumer.   This is a great
> starting point.
>
> To start you will need, IMHO, a list of servers, which implies
> a list of services.  I have found it reasonable to use rasplit()
> for this function:
>
>    rasplit -R data -M time 1d -w servers/\$daddr/argus.%Y.%m.%d.%H.% 
> M.%S \
>            - \( tcp and syn and synack \) or \( not tcp and con \)
>
> This will generate a set of data where all the servers are in a single
> directory, broken down by day.  The ( tcp and ... ) filter will give
> you 'correct' tcp flows, so you can start building your client/server
> data.  The ( not tcp ... ) filter will give you all the other  
> protocols,
> udp, arp, whatever, where there is a response, which is very
> important to understanding if there really is a service.  You can
> be much more clever and have additional filters for multicast
> data, etc...., but this is a good starting point.
>
> Once you have your data set, you'll probably want to racluster()
> the data, to make sure that all the flows are consolidated:
>
>    racluster -M replace -R servers -V
>
> Then you should have a decent set of data as a starting point.  If  
> you find
> that packet ratios and application byte ratios are good for your
> analysis, I can add these as metrics to print.
>
> Carter
>
>
> On Apr 17, 2007, at 1:14 AM, CS Lee wrote:
>
>> Carter,
>>
>> Thanks, it seems I can do this -
>>
>> ragraph sappbytes dappbytes dport -M 1s -r file.arg -
>>
>> The reason why I want to do this is that I can easily spot people  
>> whether he is doing http normal surfing or he is downloading file  
>> via http, as if the normal surfing used to have bigger src  
>> appbytes comparing to the http download src appbytes. It's more of  
>> for fun but it does the work very well for me now. Thanks!
>>
>> For protocol breakdown, I think it is more of giving overview or  
>> big picture of the transaction in the network instead. Cheers.
>>
>>
>> On 4/17/07, Carter Bullard <carter at qosient.com> wrote:
>> Hmmmmm, this is pretty easy to do, but what are you really asking  
>> for?
>> Do you want something to "know" what the src/dst byte ratio for each
>> application client/host pair is suppose to be and notify you if  
>> somehow
>> it changes?  Again this is very easy to do, in say perl, but what/ 
>> how would
>> you want such a program to be structured?
>>
>> Hmmmm, graphing based on protocol is very easy:
>>
>>    ragraph sbytes dbytes proto -M time 5m -r daily.files
>>
>> Of course if you want something more detailed:
>>
>>    ragraph sbytes dbytes proto -M time 5s -r daily.files
>>
>> Carter
>>
>>
>> On Apr 16, 2007, at 11:47 AM, CS Lee wrote:
>>
>>> Hey people,
>>>
>>> I'm looking for a way to track the changes of source and  
>>> destination application bytes over time from the flow(for example  
>>> I can easily look at whether people are either doing http surfing  
>>> or http file transfer. I have tried few combinations and has no  
>>> success so I guess it would be good to turn into mailing lists.  
>>> Is that possible to do it in argus way?
>>>
>>> Another thing is that I think a lot of people want a Protocol  
>>> Breakdown in daily basis, I know racluster can do it when  
>>> combining with rasort. But is that possible to generate the graph  
>>> for all the Protocol breakdown by using ragraph, or our  
>>> alternative is we still have to use it with excel to do it(that's  
>>> what i do for now).
>>>
>>> Thanks.
>>>
>>> -- 
>>> Best Regards,
>>>
>>> CS Lee<geekooL[at]gmail.com>
>>
>
>
>
>
> -- 
> Best Regards,
>
> CS Lee<geekooL[at]gmail.com>

Carter Bullard
CEO/President
QoSient, LLC
150 E. 57th Street Suite 12D
New York, New York 10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20070417/659b058d/attachment.html>