'packet engine' discussion.

[Chris Newton]

I'm hoping the subject line will attract attention :)

  I would like to get a discussion going about how to build the best platform 
for monitoring the worst case scenarios in the life of a network, usually DoS 
attacks.

  I have an article at home (I'll post the URL tonight), that talks about how 
a guy got the Intel Etherexpress Pro 1000 (one variant, anyways), with a 
modified Linux driver to be able to receive (he had stats on send too), 60 
byte packets at a rate of 680,000/second.  Now, this is only the card... and I 
assume the machine was very likely consumed, being able to d o no real work on 
these packets it was receiving... but, that is indeed a very good number (I 
think).

  I have heard that most Intel network cards will generate 1 interrupt per 
packet, and that the CPU will start having difficulties at around 20,000 
interrupts per second.  The Etherexpress pro 1000, and other cards, batch up 
packets and send them, DMA, to memory on one interrupt, saving lots of CPU 
over head.  Other cards do some of the TCP/ip header work, on the card.  
Others do... (fill in cool performance feature here), and so on.  Question:  
Which card, OS, drivers, features, and setup are best??

  Also, some of the places we will be monitoring, are full duplex.  Putting 
taps in (like the ones at www.shomiti.com), feed us two 100, or 1000 Mbit 
wires.  These will have to go into two cards in a server, and have Argus read 
from both cards, and merge records.  Question:  Whats best, two cards, or 1 
cards with dual ports??  If two cards, which is best, single, dual, or quad 
CPU?  Do we tie interrupts from each card to a unique CPU?

  Dealing with the packets you get is another issue... i.e.: memory bandwidth. 
Now, there are new memory technologies coming (and current ones that maybe 
have no data as to how they affect an application like Argus, ie: Rambus).  
Which of these is most promising?

  With regard to operating system technologies, Linux has some new zero-copy 
networking patches, that tries to avoid moving network traffic stuff around in 
memory very much, and there may be others.  Question: does this help us, and 
if so, what technology like this is the most promising for high speed network 
monitoring?

  The CPUs: P3, P4, Itanium (its coming)...

  The PCI bus...  obviously a bottleneck.  PCI-X is coming, Infiband right 
after that.  32 bit, or 64 bit current PCI?  We need to move data across the 
bus to memory/cpu... Question:  Is the optimal machine PCI-X, Infinband, a Sun 
server?

  The general thing here I'm trying to weed out is this:  Money no object 
(lets be realistic though)... what is the best hardware/software/network 
card/bus/memory/CPUs(dual/quad/single)(P3,p4, Itanium), and _configuration_  
combination to be able to deal with the stormiest network events... ie: as 
many tiny packets, or other crap, thrown at your network.  Obviously there is 
a real hard limit out there... but, how do we get as close as possible to it?

Thanks,

Chris

_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/

Chris Newton, Systems Analyst
Computing Services, University of New Brunswick
newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)