Heavy Traffic Conditions

[Carter Bullard]

Hey Jeff and Kevin,
There was a time where argus didn’t realize that the PF_RING interface did not support polling with a timeout, so we just hammered the interface looking for a packet.  That was/should be fixed, and a lot of people talk about using PF_RING and argus, but with mixed results.
We worked quite a bit trying to deal with timestamp issues when using PF_RING, and there are a lot of e-mails on the list that refer to “timestamps wayyy out of order…” error messages.

Many have turned to netmap as the packet processing throughput is pretty impressive there.

I think you should be able to use PF_RING now with good results up to a performance point.
Jeff’s desire to hang multiple argi against PF_RINGs dispatcher should work well,
but if there are problems, send to mailing list and we’ll figure it out.

Carter


On Feb 18, 2014, at 2:46 PM, Reynolds, Jeffrey <JReynolds at utdallas.edu> wrote:

> Kevin,
> 
> I’ve actually had to sit down and address some other things here recently, but hopefully most of this will be cleared up soon.  I’m actually going to try to implement the configuration discussed below, and see how it works out for us.  I understand quite a few people run with this kind of configuration, so I’m hoping it won’t be too difficult to instantiate, but I’ll let the list (or you) know how it’s running once we get something up and running in the test lab.
> 
> Jeff
> 
> From: The Branches <branchbunch at gmail.com<mailto:branchbunch at gmail.com>>
> Date: Tuesday, February 18, 2014 at 1:19 PM
> To: "argus-info at lists.andrew.cmu.edu<mailto:argus-info at lists.andrew.cmu.edu>" <argus-info at lists.andrew.cmu.edu<mailto:argus-info at lists.andrew.cmu.edu>>
> Subject: Re: [ARGUS] Heavy Traffic Conditions
> 
> I've used PF_RING on CentOS and Ubuntu NIDS sensor boxes for years, but each time I've linked argus against PF_RING, invariably I found my argus daemon suddenly consuming all my CPU cycles even when traffic wasn't going crazy, so I just use PF_RING with Daemonlogger and Snort/Suricata.  I haven't tried this lately, and would be interested if anyone else has had a positive experience with argus using PF_RING.
> 
> Kevin Branch
> 
> 
> On 2/3/2014 4:37 PM, Carter Bullard wrote:
> 
> Hey Jeffrey,
> This is how the Tilera chip port works.  Each argus generates flow records
> based on context of the packet stream that it gets, so if the packet capture
> facility forwards packets intelligently, you’ll get some good parallelism /
> concurrency.  Breaks down when the packet forwarder is doing something
> stupid, and you end up with the same flow being tracked by multiple argi.
> That is not a disaster, just some of the metrics don’t work as well as
> you would like, such as jitter.
> 
> You then use radium() to collect from all your argus instances, to generate
> a single stream of argus records that represent the observation domain.  As
> long as the timestamps in the packets are good, then this works very well.
> 
> You can have all the argi use the same source id, although radium() may not
> be completely happy with that, but it should work.  Any ra* aggregator,
> such as racluster(), ratop(), rabins() etc….can be used to merge records
> together that should be merged because the packet forwarding layer was
> broken, or if you want to try to get bi-directional flows from unidirectional
> flow generators.
> 
> I think a significant number of people are doing this.
> 
> Carter
> 
> On Feb 3, 2014, at 2:48 PM, Reynolds, Jeffrey <JReynolds at utdallas.edu><mailto:JReynolds at utdallas.edu> wrote:
> 
> 
> 
>> Hello,
>>> I’m curious to know if it’s possible to having multiple instances of Argus running on one machine, and have traffic load balanced across these instances utilizing PF_RING inside of CentOS.  From my understanding, PF_RING has this capability and it is used with applications such as Snort to utilize parallel processing of high bandwidth links, but I haven’t seen any documentation on how one might accomplish this with Argus.  Any information on the topic would be much appreciated.  Thanks!
>>> Jeff

Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York  10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140218/e3caf989/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140218/e3caf989/attachment.sig>