Filter type 'appbytes' behavior unexpected

[Carter Bullard]

Hey Jesse,
John was on the right track.  There is a difference between filters
and printable fields, and it really is about the limitation of the
filters more than a design goal.

So for most metrics, we have src and dst metrics, and we use the [s|d]
prefix to the metrics to specify the src or dst for operations.  We
have provided a convenience strategy where we provide the sum of the
src and dst metrics, for only a few of these, pkts, bytes and appbytes.

Now, we've provided these metrics like they were actual numbers, so
you can sort on them, do analytics on them and print them. But filtering
on them has a snag, because of two basic issues.

One issue is that the filters are simple comparisons of values, or value
ranges, against flow record content.  We know what the offsets are
for every metric in an argus record, and we can build filters that
are just load and compare programs.  Load this value from this offset
and compare with the literal provided in the filter statement.  Use the
"-b" option, and you'll see how this works.

The problem is that the sum of the src and dst metrics are not in the
flow records themselves.  I need to implement some fetch and basic math
operators in the filters to make some of these comparisons work, in
the general sense.

Now, we do provide printing, sorting AND filtering on metrics like
"rate" or "load", which are not in the flow records.  These are
derived metrics.  we're loading up the end of the " in memory "
flow record with some of these derived values.  I could do that with
pkts, bytes and appbytes, but,…, if you look at the support for "rate",
you'll see that we've been consistent with these derived values and
the other values.  When you print on "rate", its the combined rate,
when you  filter, its not on the combined value.  This comes from
the second issue that we have to deal with….

The second has to do more with tcpdump's filter strategy.
Tcpdump() has a general rule, that if in the filter, you leave off
the direction indicators, the filter assumes that you mean to
query against either src or dst.  So not providing a direction
indicator, causes the filter to use a "src or dst" query logic.

The filter descriptions for argus in all the man pages do spell
this out.

Now this behavior is historical.  The argus compiler was derived
from the tcpdump/libpcap compiler, and it inherited its benefits
but also its limitations.

So what to do???  I'm always for improving stuff.  The filters
could use the ability to fetch fields and do operations on them, like:

    ra -r file - (src host/24 = dst host/24)   // same subnet
    ra -r file - ((src appbytes + dst appbytes) / (src bytes + dst bytes)) < 0.5 // efficiency


If its important, then I can do the work, unless of course, you want
to do it…

So what are the real queries you want to do ??????


Carter


On Oct 6, 2013, at 11:16 PM, Jesse Bowling <jessebowling at gmail.com> wrote:

> Hello,
> 
> I noticed while looking at some data that using the filter type
> 'appbytes' appears to filter on either sappbytes or dappbytes, but not
> on appbytes, which is unexpected to me. From the man page:
> 
>       [src | dst] appbytes [gt | gte | lt | lte | eq] number
>              True if the application byte count in the Argus record
> (default) equals number.
> 
> For instance, I would expect this invocation:
> 
> ra -r test.argus -s sappbytes dappbytes appbytes - appbytes eq 2760
> 
> To display output for only flows whose total application bytes
> (appbytes) equals 2760. Instead the behavior I see is that I get flows
> which have 2760 bytes in either sappbytes or dappbytes.
> 
>   SAppBytes    DAppBytes TotAppByte
>         151         2760       2911
>         169         2760       2929
>        2760          151       2911
>         151         2760       2911
> 
> I can see the utility of this behavior, but based on the documentation I
> would not expect this result. I'm seeing the same results from 3.0.7.9
> -> 3.0.7.15.
> 
> Is the current behavior the 'correct' behavior or a bug?
> 
> Thanks,
> 
> Jesse
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20131007/8876d6c7/attachment.bin>