real time flow classification

[Peter Van Epp]

On Sat, Oct 06, 2012 at 09:22:55PM +0300, Oguz Yarimtepe wrote:
> Hi,
> 
> I am a Ph.D student from Turkey. My subject is real time security visualization. Now a days i am searching about real time traffic classification. My first aim is to collect real time flow attributes on a database. It seems Argus can save to MySQL but i may be using a non-sql database. I am also planning to use TNAPI and PF_RING with an Intel card on a PREEMPTIVE_KERNEL. So i have some questions. It would be great if you readirect me to some sources.
> 
> * Did you test Argus with TNAPI + PF_RING? Will it need any modifications for working with PREEMPTIVE_KERNEL and these drivers or how can i use it to sniff 1Gbit campus network? My Intel card is not Endace card, it is Intel Corporation 82572EI Gigabit Ethernet Controller. So with my commodity hardware (that is a dual core pentium 4 with 4 GB RAM, an old IBM ThinkCenter Workstation) i will be trying to collect the Gbit traffic.
> 

	While DAGs are the best solution, Intel Server pro cards and PFring 
work fine with sufficient horsepower even at gig. I used to work for a 
university (and have been using argus for 15 years or more) and one of the 
things I did was (along with our HPC guys who had a clear channel gig link
light path on the Canadian research net) was capture a netperf run saturating
the link (don't try this on a production network though!). While the machine
running argus was an IBM P5 power PC with linux and pfring (which is probably
more powerful that a P4), that worked fine with minimal packet loss. The reason
for the PPC machine is that it is big-endian (or little-endian, I never can 
remember which :-)) unlike the Intel which is the other. Thus the htons macros
are no op. This appears to be about a %10 or %20 speed increase for capture 
by not having to swap bytes in the captured packets. Thus if you can find an
old Apple Power PC machine around (which is no obsolete) that can run linux and
take the Intel card (i.e. it needs PCIX 64/66 slots) that may be a better
choice. As long as PFring is happy with your card and OS I suspect Argus will
be fine (although note the late comments on the list about pfring select 
ignoring timeouts!). I'm retired these days and so no longer have fast links
to play with so my knowledge is getting old, but I should be perfect for the 
older hardware you are running :-). It also depends on how busy the link you 
are monitoring is how much packet loss you will see. The argus man records will
report how much loss pcap reports (because pfring doesn't do the kernel to 
user space copy, that may not apply in this case though). Its really best to 
have some external way (RMON on the network switches, or a network management
system that can independently of argus report packet and/or byte counts) to 
compare what argus sees with what the network thinks is on the wire. This isn't
at all easy however. I'll let someone that knows about database issues answer
your other questions. Good luck and have fun!

Peter Van Epp