Collecting multiple types of information at once

[Carter Bullard]

OK, so I've been thinking about this, and it seems that you're thinking that you
want to label flows when they are processed by a specific line in the racluster.conf file?
It would be trivial to add a labeling step in the racluster/rabins processing path.

You can do a preliminary pipeline to do this by using ralabel() piping into rabins().
The ralabel.conf and the racluster.conf strategies are the same, fall through filter
statements.  You can have the same filter logic that labels the flows as a first
stage in a pipeline, and then your racluster() would get flows that are pre-labeled,
and the aggregation will preserve the labels so that all the output flows can be
identified using the " -s +label" field.

Once you look into it, you should see that its pretty simple to test.

What I can do, if you want the resulting aggregations to have a unique label so you
can tell what aggregation was used for the output flow, we can add a " label='string'"
field to the racluster.conf configuration file entry.  Does that sound like what you're
thinking ?

Carter



On Aug 30, 2012, at 8:55 AM, Carter Bullard <carter at qosient.com> wrote:

> Dense message.  Use rabins() instead of rasplit(), without the racluster() call.  rabins() is specifically designed to " bin " the aggregation scope, so it is racluster() within each " bin ".
> 
> So run this :
> 
>   rabins -M time 5m -r argus.2012.08.29.15.00.07.gz -f /tmp/filtertest -s stime saddr  dir daddr pkts bytes -nn | less
> 
> The " cont " support is definately in argus-clients-3.0.6, but may also be in 3.0.4. (not near the code right now).
> 
> Yes, the problem is how to know that one aggregation context is over and the next has begun.  We should put MAN records in between the output, so if they aren't there I'll investigate.
> 
> With respect to field widths, you can specify in the .rarc RA_FIELD_WIDTH='variable' and you won't get field truncation.
> 
> So with regard to labeling, not sure what you're looking for.  ralabel() and radium() are labelers, all ra* programs can print the labels, " -s +label " and grep patterns in the label with the " -e regex " option.  So what labels do you want to add ?
> 
> Carter
> 
> On Aug 30, 2012, at 6:06 AM, Martijn van Oosterhout <kleptog at gmail.com> wrote:
> 
>> On 30 August 2012 09:11, Martijn van Oosterhout <kleptog at gmail.com> wrote:
>>> On 30 August 2012 01:39, Carter Bullard <carter at qosient.com> wrote:
>>>> Hey Martijn,
>>>> One thing to consider is the " cont " directive that you can use in racluster.conf.  It may do some of what you are interested in.
>>> 
>>> Now that is interesting. I don't see "cont" mentioned in the man pages
>>> I found online but I will have to check if that works. Do you know
>>> from which version it is available?
>> 
>> Ok, so I've played with this and "cont" seems to work as advertised.
>> However, other bits not so much. For example, with rasplit I get an
>> error:
>> 
>> rasplit -M time 5m -r argus.2012.08.29.15.00.07.gz -w - | racluster
>> '-c|' -f /tmp/filtertest -s stime saddr  dir daddr pkts bytes -nn
>> |less
>> 
>> Gives the error:
>> rasplit[1644]: 30 Aug 12 11:54:28
>> ArgusWriteNewLogfile(-.2012.08.29.14.05.00, 0xfcebb00c) fwrite error
>> Bad file descriptor
>> 
>> What I would have expected to happen is the clustering to break on
>> 5minute intervals. From the documentation this is what I expect to
>> happen.
>> 
>> (By the way, the -c option seems to fix all problems with respect to
>> field widths).
>> 
>> Also, the documentation on the web site, specifically the man pages,
>> seem to be missing stuff. The racluster.conf has a paragraph that ends
>> with "These filters are" and then nothing.
>> 
>> Anyway, there doesn't appear to be a way to label the output flows, or
>> at least not in a way you can print. And it's a bit odd to try to
>> detect it like "if dest is 0.0.0.0 then is was the saddr flow".
>> Ralabel looks promising, but doesn't seem quite right.
>> 
>> Any more tips? It's seems almost within reach now.
>> 
>> Thanks in advance,
>> -- 
>> Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/
>> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2589 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120830/682d02c6/attachment.bin>