hardware for argus with 10GB link

[Guillaume FORTAINE]

If we consider 10GbE packet captures cards with FPGA, there are :

-3 OEMS (with Fixed Firmware)

a) NTT Advanced Technology PRESTA 10G (FPGA : Altera Stratix II,
Firmware : Fixed, price : 3000 USD)

http://docs.google.com/viewer?url=http://www.internet2.edu/presentations/jt2010feb/20100201-sebayashi.pdf


a) Napatech NT20E (OEM, FPGA : Xilinx Virtex-4, Firmware : Fixed,
price : 10 000 USD)

http://www.napatech.com/products/capture_adapters/2x10g_pcie_nt20e.html


b) Endace DAG 9.2X2 (OEM, FPGA : ?, Firmware : Fixed, price : ?)

http://www.endace.com/dag-9.2x2-packet-capture-card.html



-3 ODMs (with Programmable Firmware)

a) INVEA-TECH COMBO (FPGA : Virtex-5, Firmware : Programmable/NetCOPE,
price : 18995 USD)

http://www.invea-tech.com/fpga-solutions/fpga-solutions-overview


b) Algo-Logic NetFPGA 10G  (FPGA : Virtex-5, Firmware :
Programmable/NetFPGA, price : 7995 USD)

http://docs.google.com/viewer?url=http://algo-logic.com/pdfs/NetFPGA10g_BlockDiagram.pdf


c) CESNET MTPP10-V5 (FPGA : Virtex-5, Firmware : Programmable / MTPP,
price : ?)

http://docs.google.com/viewer?url=http://www.ces.net/project/qosip/hw/mtpp10-v5.pdf



On 04/23/2010 11:35 PM, Peter Van Epp wrote:
> On Fri, Apr 23, 2010 at 12:14:02PM -0700, Michael Sanderson wrote:
>    
>> We've recently had our campus network connection upgraded to 10Gb and
>> are now looking at our options for tapping that connection and getting
>> argus collecting data again.  We'll be tapping our fibre links with most
>> likely with NetOptics taps, but we're looking for suggestions of
>> appropriate 10Gb NICs for our argus "sensor" box.  We definitely won't
>> be pushing the 10Gb link initially, but I expect that we'll see periods
>> where we approach 2Gb sustained.
>>
>> I recall discussion on the list regarding interrupt coalescing on the
>> network cards generating potential issues due to the way that time
>> stamps are assigned to packets.  I know that Endance's DAG cards solve
>> this particular issue by timestamping every packet, but they are
>> expensive and eat a big portion of my hardware budget.
>>      
> 	Unfortunatly you only get what you pay for :-). The biggest win on DAGs
> is a large (4 meg on gig cards) internal SRAM buffer. That (against 64 kbytes
> on a typical Gig NIC) gives you a small amount of breathing room for bus and
> interrupt latency in to the CPU before packet loss occurs. My comments on
> a bunch of this (including interrupt coalescing) are available at
>
> http://www.qosient.com/argus/sensorPerformance.shtml
>
> 	As Mr Plonka noted the www.ntop.org site is a good place to go to learn
> more about the issues of high speed capture as well (you really want a zero
> copy interface such as pf-ring for instance, the memory copy is expensive).
>
>    
>> Does anyone recall the specific issues with interrupt coalescing and
>> argus?  There will definitely be potential for out of order TCP
>> connection startup/shutdown.  What is the impact on the flow reporting?
>>      
> 	Mostly timing inaccuracies as multiple packets appear at the same
> apparant time (when they didn't in reality). Not much else happens as Carter
> put in some changes for me when I was running 2 NICs on an FDX link that
> (as long as the individual time order is correct which it should be) allowed
> for things like a syn-ack to arrive before the syn was sent due to interrupt
> scheduling. As far as I know thats still there and works.
>
>    
>> Are there other options to Endance's cards that solve the timestamp
>> problem, either other vendors or system/kernel configuration?
>>
>>      
> 	I haven't heard of any, but since I retired I haven't been paying much
> attention either :-). There is an Intel 10 Gig NIC that has internal time
> stamping for the HPC guys (I came across it looking at ntp implementations
> since thats one of the things you need for a DAG clone) that may be a good bet,
> but it is brand new and perhaps not very available yet and drivers will likely
> be an issue. The problem is emulating a DAG is low volume and non standard (I'm
> poking at doing so more cheaply at gig with FPGAs) in that you need to tap in
> to the Ethernet MAC to get the start of frame signal, you need an ntp type
> time source on card and you want large amounts of SRAM buffer which are
> expensive. I'll note one other thing to consider on a DAG is that it has an
> internal CPU and thus could do the network to host order header translation
> on card (perhaps ...) which sould save you a very valuable %20 or so memory
> bandwidth on the sensor side (but you would also need to lie to argus about
> machine endian). This assumes that your sensor is little endian (i.e. Intel),
> if you have a big endian machine you are better off (except probably in price
> :-).
> 	BCnet's network research group has a 10 Gig capture applience (an
> Endace Ninja) fed from a Netoptics Network Director (more to aggregate multiple
> 1 Gig segments on to the 10 gig for capture than the other way but ether will
> work, but it was the same price as a 10 gig DAG) although I don't know how its
> working as I retired just as it was being installed (I did a spec and run :-)),
> so it would probably be a good bet to have a chat with the NMC folks your way
> about what its doing.
> 	Its also worth noting that argus runs perfectly well half duplex, so
> (budget allowing) two sensors are a good bet one for each half of the FDX
> connection. Both sensors feed out via ethernet to a collection box where argus
> merges the streams and writes to disk (you don't want to be doing disk I/O on
> your sensor box). This halves the required performance on each sensor (and
> DAGs at 10gig are HDX only or used to be anyway). You also want to take care
> to get the fastest RAM you can afford (preferably the fastest DDR3) in a
> high performance CPU (your HPC folks are a good bet for advice here) for the
> sensors as memory bandwidth is the most usual limiting factor at high link
> speeds.
>
> Peter Van Epp
>
>
>