Argus with PF_RING DNA clusters

[Chris Wakelin]

On 05/10/12 15:55, Carter Bullard wrote:
> Hey Chris,
> Sorry for the delay. 
> 
> Argus is already structured for multiple threads to eat packets, but there
> is work that needs to be done to enable many cores to get into 
> the packet processing business for a single interface.  We can do that
> kind of effort on the list if that is of interest.   Not hard, but will need some
> re-architecting for argus to support multiple cores at this level.
> 
> There are two strategies possible here, that I see.  
> 
> First involves routing packets to the appropriate cores for processing.  Argus
> has been ported to multi-core chips, like Tilera, and they use this technique,
> where you run a separate image of argus on each core.  One core reads
> packets, and processes the headers to the point where it can decide which 
> argus context should process the packets, and then queues the packets for
> the independent argi to process.  This is pretty good, as it avoids any form
> of locking, but does have the problem of asymmetric work loads.

That was more or less what I had in mind. PF_RING DNA will split the
traffic by flow into up to 32 queues bound to virtual ethernet
interfaces (though in my case I might be limited to 10). Unlike RSS in
the native ixgbe driver, the flows should be symmetrically balanced so
that packets from X->Y go to the same queue as those from Y->X. The flow
binding is via a user-customisable function, so it may be possible to
arrange things to get a similar number of packets in each core.

I'm hoping to have my three applications each get their own view of each
of the queues, but each bound to the same CPU for a particular queue.
This should avoid cache-misses on each core, which is one of the
problems, I believe, in multithreaded processing.

At the same time, it would be nice for all the flow records to end up in
one Argus database, so running 10 independent copies of Argus could be a
problem. I'm hoping that starting Argus with multiple virtual
interfaces, and then setting the CPU affinities for the individual
threads manually should avoid Argus having to do the flow balancing
itself, but have one database.

> 
> The other is where you have multiple cores doing the packet header processing,
> getting to the point where you have established the flow cache keys, and
> calculated the hash values, then a single core does the look ups, and updates
> the metrics.  This type of processing has some advantages for hardware
> implementations, but does have some issues in software scheduling, as the
> amount of header processing can be different for each packet.

That would be fine if it was just ARGUS. I'm hoping to have PF_RING do
the flow balancing, and then have each application process the packets
in the appropriate queues and save their results (flow database, IDS
alerts etc.) in one place each.

> If you're interested, and can help me with the PF_RING part, I can do
> a lot of the multicore parts.
> 
> Carter

I'll do my best to help with PF_RING, but I'm not really a programmer!

Best Wishes,
Chris

-- 
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
Christopher Wakelin,                           c.d.wakelin at reading.ac.uk
IT Services Centre, The University of Reading,  Tel: +44 (0)118 378 2908
Whiteknights, Reading, RG6 6AF, UK              Fax: +44 (0)118 975 3094