Collecting multiple types of information at once

[Martijn van Oosterhout]

On 30 August 2012 22:21, Carter Bullard <carter at qosient.com> wrote:
> OK, so I've been thinking about this, and it seems that you're thinking that you
> want to label flows when they are processed by a specific line in the racluster.conf file?
> It would be trivial to add a labeling step in the racluster/rabins processing path.

Basically yes, that would be the right way of describing it.

> You can do a preliminary pipeline to do this by using ralabel() piping into rabins().
> The ralabel.conf and the racluster.conf strategies are the same, fall through filter
> statements.  You can have the same filter logic that labels the flows as a first
> stage in a pipeline, and then your racluster() would get flows that are pre-labeled,
> and the aggregation will preserve the labels so that all the output flows can be
> identified using the " -s +label" field.

Ah, I didn't see you could use ralabel that way. But ofcourse this is
incompatible with "cont", since that would assign the all labels to
all records, which is not what you want. Thanks for the tip "-s
+label", I was looking in the man pages for the field name but
couldn't find it.

> Once you look into it, you should see that its pretty simple to test.

Will do. Although rabins is acting up for me, I need to work out what
I'm doing wrong.

> What I can do, if you want the resulting aggregations to have a unique label so you
> can tell what aggregation was used for the output flow, we can add a " label='string'"
> field to the racluster.conf configuration file entry.  Does that sound like what you're
> thinking ?

Yes! That I think would do exactly what I want.

Perhaps a better description of what I would like would be:

For a given argus file, for each 5 minute interval, produce a list of
the top ten src ips, dst ip, src ports, dest ports and total
bandwidth.

Now, the racluster with the "cont" gives you most of this, and with
rasplit/rabins it should be able to do the 5 minute splitting. The
last bits I think are specialised enough that we can code that
ourselves, though the labelling is needed to reliably work out which
aggregates are which. The most important thing is that we only want to
read the datafiles exactly once, and racluster gives us that, so I'm
happy. The labelling in racluster would make it perfect.

Thanks again!

Have a nice day,
-- 
Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/