Flow aggregation..

[Denton, Rick]

sorry, This email is rated 'W' for way too long..

> Hey Rick,
> Glad to see your mail/tome!!!  racluster() is suppose to be 

yea my apologies it was a bit waffly :( as is this one..

> better, so if you can't do something you want to do, we'll fix it!!
> I changed the name because it is a different beast, but we 
> can use the ragator name again, very difficult to kill a 
> dragon you know!!!!

:) the name was good and harsh sounding.. good for scaring small
children into doing what you tell them.. :)
"pick up all your toys or big bad ragator will come and aggregate them
all with an all 0s mask and then you'll be sorry! >:)"

> You should be able to do your bidding using the 'net 
> x.y.z.w/a.b.c.d' filter syntax.

<snip>

> Filter="src net 192.168.0.0/16" model="saddr/24 proto dport"
> Filter="dst net 192.168.0.0/16" model="daddr/24 proto dport"
> 
> You're suppose to be able to provide a non-contiguous mask by 
> putting a real mask instead of a digit, does this work?
> 
> Filter="src net 192.168.0.0/192.168.0.255 and dst port 53" 
> model="saddr/255.255.0.255 proto dport"
> 

excellent.. :) i guess i should have read the code and not the man
page.. teasingly 3.0.0-rc30 has a ragator man page in it still ;) yes
the ability to arbitrarily filter with a pcap stle filter per aggregate
is indeed more flexible... and i didn't realise you could apply a
netmask to -m's saddr/daddr :\

... time passes ...

ok.. just tested.. addresses altered to protect the innocent..

foo.conf:

filter="net 192.168.0.0/16" model="daddr/255.255.0.255 proto dport"

then:

$ racluster -r input -F foo.conf - "host 192.168.0.10 or host
192.168.0.11"

the filter to help things along a bit.. input is large.. and i failed to
add a catchall filter in foo.conf :)
this seems to fail to aggregate anything..  in fact produces the same
output as without the -F :( this seems remarkably odd i suspect this is
me..

so:

$ racluster -r input -M net 192.168.0.0/16 -m daddr/255.255.0.255 -
"host 192.168.0.10 or host 192.168.0.11"

fails to aggregate as expected but interestingly the least significant
bit of the daddr has been zeroed :) so i now have aggregates instead of
1 but they both have the same daddr cos the lsb has been zeroed..
interesting :/

$ racluster -r input -M net 192.168.0.0/16 -m daddr/16 - "host
192.168.0.10 or host 192.168.0.11"

returns one aggregate as it should..

i apologise in advance for using this awful mua i have no choice.. so i
apologise for any font width issues that may occur from stupid font
usage which it shouldn't as i have apparently told it to use text
format.. so i also apologise for any ^M :(

so i have a quick look at the code.. semi-randomly guessing this is
processed in
argus-clients-3.0.0.rc.30/common/argus_client.c


<snip>

if ((mode = modelist) != NULL) {
    while (mode) {
        ...
        
        if ((ptr = strchr(mode->mode, '/')) != NULL) {
            *ptr++ = '\0';
            if ((value = strtol(ptr, endptr, 10)) == 0)
                if (*endptr == ptr)
                    usage();
        }

<snip>


This is interesting.. it is hunting for a /<decimal_value> regardless of
the mode.. ok.....
then:
retn->saddrlen = value;
retn->daddrlen = value;

in each if/else mode instead of just once when parsed?

This shows a couple of things:

- a dotted quad wont work as a mask
- there is a serious lack of error checking on the strtol()
  ie a dotted quad will cause endptr to not be where it should be and
thus
  *endptr should be tested for '\0' also (hopefully ptr is null
terminated ;)).
  The check you have there will only check if none of the string at ptr
is valid :(
- ipv6 has no hope ;) in particular as 'retn' the 'struct
ArgusAggregatorStruct' has
  'saddrlen' and 'daddrlen' as an 'int'..

either this hasn't been converted with the ipv6 bits or it was intended
to hold a prefixlen/cidr instead :)

i don't know enough about the code to warrant my rambling so please tell
me to bugger off if i'm horribly off track or just being annoying :)
(like looking at the wrong chunk of code).. i will have to rectify this
a bit and read more..

to put a real mask in there is going to require using whatever you are
using to hold a v6 address in there the ArgusAggregatorStruct :/.......
i need to look at the code a bit more to grok what is being done to hold
v6 addresses in here..

i can produce a patch to check for and parse real mask aswell as a
decimal int  (prolly better to use a stroul() vs strtol()?)... but i
fear this will mess with v6.. this is obvious for v4 :).. more annoying
for v6.. i guess a quick check and a call to inet_pton() to convert the
mask back.. that'll be reasonable sane for v4 and v6 addresses?

this will require changes to the struct ArgusAggregatorStruct perhaps
throwing a sockaddr_storage or something in there :(

you prolly have a way to fix this that complies with the rest of your
coding style / structures so i'll stop guessing :)

i have a fair bit of rant to say regarding SQL support also.. i am happy
to help write sql support for argus as i have a few opinions on it
having being dealing with it for the last few years (argus<->sql) but
i'll have to subscribe and post from my home email as i about to leave
work..

> Well have no sadness!!!!!
> 

:)