Flow aggregation..

[Carter Bullard]

Hey Rick,
Application specific configuration files use the "-f conf.file" option.
The "-F ra.rc.like.conf" option is used for the general ra* client
configuration. So for racluster(), you should use the "-f conf.file"  
option.

    Ok, so I need to fix a few things including my brain.   The  
filter syntax
for networks and masks that I suggested was way off.   For the filter,
the correct syntax is:

    "net x.y.z.w mask x.y.z.a"

    For the aggregator, it does support masking, but the address based
mask support seems to be out (must have ripped it out during the culling
of the code).  I'll put that back in.  The syntax is:

    saddr/[digit | ipv4 | ipv6]

so examples could be:

    saddr/18
    saddr/67
    saddr/255.255.0.255
    saddr/ffff::ffff

The saddrlen and daddrlen variables are for the address mask length,
and so an 'int' can handle it no problem (actually I've changed it to  
a short
since you pointed it out).

I'll have a new set of code up on monday that implements this.

Carter



On Oct 6, 2006, at 3:51 AM, Denton, Rick wrote:

> sorry, This email is rated 'W' for way too long..
>
>> Hey Rick,
>> Glad to see your mail/tome!!!  racluster() is suppose to be
>
> yea my apologies it was a bit waffly :( as is this one..
>
>> better, so if you can't do something you want to do, we'll fix it!!
>> I changed the name because it is a different beast, but we
>> can use the ragator name again, very difficult to kill a
>> dragon you know!!!!
>
> :) the name was good and harsh sounding.. good for scaring small
> children into doing what you tell them.. :)
> "pick up all your toys or big bad ragator will come and aggregate them
> all with an all 0s mask and then you'll be sorry! >:)"
>
>> You should be able to do your bidding using the 'net
>> x.y.z.w/a.b.c.d' filter syntax.
>
> <snip>
>
>> Filter="src net 192.168.0.0/16" model="saddr/24 proto dport"
>> Filter="dst net 192.168.0.0/16" model="daddr/24 proto dport"
>>
>> You're suppose to be able to provide a non-contiguous mask by
>> putting a real mask instead of a digit, does this work?
>>
>> Filter="src net 192.168.0.0/192.168.0.255 and dst port 53"
>> model="saddr/255.255.0.255 proto dport"
>>
>
> excellent.. :) i guess i should have read the code and not the man
> page.. teasingly 3.0.0-rc30 has a ragator man page in it still ;) yes
> the ability to arbitrarily filter with a pcap stle filter per  
> aggregate
> is indeed more flexible... and i didn't realise you could apply a
> netmask to -m's saddr/daddr :\
>
> ... time passes ...
>
> ok.. just tested.. addresses altered to protect the innocent..
>
> foo.conf:
>
> filter="net 192.168.0.0/16" model="daddr/255.255.0.255 proto dport"
>
> then:
>
> $ racluster -r input -F foo.conf - "host 192.168.0.10 or host
> 192.168.0.11"
>
> the filter to help things along a bit.. input is large.. and i  
> failed to
> add a catchall filter in foo.conf :)
> this seems to fail to aggregate anything..  in fact produces the same
> output as without the -F :( this seems remarkably odd i suspect  
> this is
> me..
>
> so:
>
> $ racluster -r input -M net 192.168.0.0/16 -m daddr/255.255.0.255 -
> "host 192.168.0.10 or host 192.168.0.11"
>
> fails to aggregate as expected but interestingly the least significant
> bit of the daddr has been zeroed :) so i now have aggregates  
> instead of
> 1 but they both have the same daddr cos the lsb has been zeroed..
> interesting :/
>
> $ racluster -r input -M net 192.168.0.0/16 -m daddr/16 - "host
> 192.168.0.10 or host 192.168.0.11"
>
> returns one aggregate as it should..
>
> i apologise in advance for using this awful mua i have no choice..  
> so i
> apologise for any font width issues that may occur from stupid font
> usage which it shouldn't as i have apparently told it to use text
> format.. so i also apologise for any ^M :(
>
> so i have a quick look at the code.. semi-randomly guessing this is
> processed in
> argus-clients-3.0.0.rc.30/common/argus_client.c
>
>
> <snip>
>
> if ((mode = modelist) != NULL) {
>     while (mode) {
>         ...
>
>         if ((ptr = strchr(mode->mode, '/')) != NULL) {
>             *ptr++ = '\0';
>             if ((value = strtol(ptr, endptr, 10)) == 0)
>                 if (*endptr == ptr)
>                     usage();
>         }
>
> <snip>
>
>
> This is interesting.. it is hunting for a /<decimal_value>  
> regardless of
> the mode.. ok.....
> then:
> retn->saddrlen = value;
> retn->daddrlen = value;
>
> in each if/else mode instead of just once when parsed?
>
> This shows a couple of things:
>
> - a dotted quad wont work as a mask
> - there is a serious lack of error checking on the strtol()
>   ie a dotted quad will cause endptr to not be where it should be and
> thus
>   *endptr should be tested for '\0' also (hopefully ptr is null
> terminated ;)).
>   The check you have there will only check if none of the string at  
> ptr
> is valid :(
> - ipv6 has no hope ;) in particular as 'retn' the 'struct
> ArgusAggregatorStruct' has
>   'saddrlen' and 'daddrlen' as an 'int'..
>
> either this hasn't been converted with the ipv6 bits or it was  
> intended
> to hold a prefixlen/cidr instead :)
>
> i don't know enough about the code to warrant my rambling so please  
> tell
> me to bugger off if i'm horribly off track or just being annoying :)
> (like looking at the wrong chunk of code).. i will have to rectify  
> this
> a bit and read more..
>
> to put a real mask in there is going to require using whatever you are
> using to hold a v6 address in there the  
> ArgusAggregatorStruct :/.......
> i need to look at the code a bit more to grok what is being done to  
> hold
> v6 addresses in here..
>
> i can produce a patch to check for and parse real mask aswell as a
> decimal int  (prolly better to use a stroul() vs strtol()?)... but i
> fear this will mess with v6.. this is obvious for v4 :).. more  
> annoying
> for v6.. i guess a quick check and a call to inet_pton() to convert  
> the
> mask back.. that'll be reasonable sane for v4 and v6 addresses?
>
> this will require changes to the struct ArgusAggregatorStruct perhaps
> throwing a sockaddr_storage or something in there :(
>
> you prolly have a way to fix this that complies with the rest of your
> coding style / structures so i'll stop guessing :)
>
> i have a fair bit of rant to say regarding SQL support also.. i am  
> happy
> to help write sql support for argus as i have a few opinions on it
> having being dealing with it for the last few years (argus<->sql) but
> i'll have to subscribe and post from my home email as i about to leave
> work..
>
>> Well have no sadness!!!!!
>>
>
> :)
>

Carter Bullard
CEO/President
QoSient, LLC
150 E. 57th Street Suite 12D
New York, New York 10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20061006/a0f48566/attachment.html>