hardware for argus with 10GB link

[Carter Bullard]

Hey Michael,
I have argus's that have been tuned for specific processors that get up to 1.2M pps (Endace
cards in Mac Pro's with dual core G5s), and the trick is, as Dave and Luca have pointed out,
is cache management.  The good performance was done with early versions of argus-3.0,
but before a lot of new parsing features were added.  As we add more and more features to
argus, and add all the new multi-core architectures which have differing cache strategies,
the cache performance can be good or terrible.  Sometimes all it takes is moving a prefetch
of memory a few cycles before a read to make or break great performance at these speeds.

Argus has been ported to Bivio, and Tilera multicore architectures, and performs well in
those environments.  Napatech has an implementation that uses Tilera chips to provide
multiple cores in the capture card, but I have not worked with these cards at all.  I see
around 2-3 Gbps from Endace Ninja probes out of the box.  To get higher, you need to turn
stuff off, like all the newer deep header parsing, jitter,  packet size distribution reporting, tcp
performance, etc.....

At the same time, I've seen slight variations in say a Dell multicore PC really kill performance
on packet processing, and I've seen variations among the exact same devices, so to grab
a box, put say some Endace cards in them, and start processing packets, isn't a guarantee
that it will perform great.

I'd love to work with anyone to make things go faster.  It would be really cool if someone had
some bucks to do it.

Carter

On Apr 23, 2010, at 3:14 PM, Michael Sanderson wrote:

> We've recently had our campus network connection upgraded to 10Gb and are now looking at our options for tapping that connection and getting argus collecting data again.  We'll be tapping our fibre links with most likely with NetOptics taps, but we're looking for suggestions of appropriate 10Gb NICs for our argus "sensor" box.  We definitely won't be pushing the 10Gb link initially, but I expect that we'll see periods where we approach 2Gb sustained.
> 
> I recall discussion on the list regarding interrupt coalescing on the network cards generating potential issues due to the way that time stamps are assigned to packets.  I know that Endance's DAG cards solve this particular issue by timestamping every packet, but they are expensive and eat a big portion of my hardware budget.
> 
> Does anyone recall the specific issues with interrupt coalescing and argus?  There will definitely be potential for out of order TCP connection startup/shutdown.  What is the impact on the flow reporting?
> 
> Are there other options to Endance's cards that solve the timestamp problem, either other vendors or system/kernel configuration?
> 
> Thanks,
> 
>    Michael Sanderson
>    Manager, Computing Facilities
>    Department of Computer Science
>    University of British Columbia
>