Argus Filter Question

Tue May 17 19:40:01 EDT 2011

Hey Stephen,
Yes, argus can report nanoSec resolution, and all the tools know what to do, so definitely can use the improved res.

I did go down the hardware filter path with argus, so that may take a few brain cells to bypass the pcap compiler in the current code, but you guys provide a compiler and loader for your filters?

I will put the dag support back in, and will include the infiniband support that was in gargoyle.

May take a few weeks,
Thanks, and hope all is most excellent,

Carter

On May 17, 2011, at 6:07 PM, Stephen Donnelly <Stephen.Donnelly at endace.com> wrote:

> It sounds like the timestamp problem is not DAG related, or dag->pcap related.
> 
> There is some overhead in converting DAG native ERF records into pcap records. The overhead is hard to measure, but is roughly twice as much for pcap compared to using the DAG APIs directly (but still very low compared to pcap on NICs).
> 
> Apart from the overhead I do  not know if Argus would make use of the additional timestamp resolution or metadata (Port Id etc) available in the ERF records, since pcap is the lowest common denominator.
> 
> If you are using DAG-enabled libpcap, then any pcap filters you specify will be executed in userspace inside the libpcap library. They should work the same way as for regular NICs. It sounds like John spotted a bug in your filter, so hopefully that is solved.
> 
> If you are using either DAG-enabled libpcap OR the DAG APIs natively you can load hardware filters onto the DAG cards as well (if supported). As this filtering/load balancing happens before the packets leave the card they operate transparently to libpcap.
> 
> Carter, if you decide to put native support back in I can look at updating it to support DAG Streams with the newer APIs.
> 
> Regards,
> Stephen.
> 
> -----Original Message-----
> From: argus-info-bounces+stephen.donnelly=endace.com at lists.andrew.cmu.edu [mailto:argus-info-bounces+stephen.donnelly=endace.com at lists.andrew.cmu.edu] On Behalf Of Carter Bullard
> Sent: Wednesday, 18 May 2011 5:31 a.m.
> To: Leif Tishendorf
> Cc: argus-info at lists.andrew.cmu.edu
> Subject: Re: [ARGUS] Argus Filter Question
> 
> The timestamp problem is a bug I've been working for a while, so I don't think it's the DAG, but you never know.
> I think there is a benefit, but it is hard to say how much.
> Stephen Donnelly from Endace lurks on the mailing list a bit, possibly he could chime in.  If not, I'll send him some mail directly asking what is the best approach today.
> 
> Carter
> 
> On May 17, 2011, at 1:16 PM, Leif Tishendorf <ltishend at gmail.com> wrote:
> 
>> I wonder if there would be some overhead reduction if Argus read the ERF directly rather then having to have libpcap read the ERF translate and then hand to Argus.  If so, that would be a bonus.  Also I have some weird timestamping issues (on occasion Argus gets packets it seems to think have hit a worm hole and have arrived from 1964(they must have hit 88MPH)) and I'm wondering if getting the timestamp from the ERF timestamp header would correct that.
>> 
>> Instead of filtering at the libpcap level I've moved it back to the rasplit so I can also filter out a few other things that libpcap doesn't understand.
>> 
>> Adding the Dag option back in certainly isn't a priority, or if you don't think it has any value.
>> 
>> -Leif
>> 
>> On 05/17/2011 10:00 AM, Carter Bullard wrote:
>>> So sorry.  My bad, I forgot that I took it out for 3.0.4.  If important I can put it back in for 3.0.5.
>>> Argus has supported dag's and ERF packet formats for many, many 
>>> years. Probably took it out when I cleaned up all the legacy packet formats, that no one uses any longer.  snoop, moat, etc....
>>> 
>>> No thing to put back in.  If you looked in the code you would see 
>>> ArgusErfRead(), ArgusDagPacket(), Arguslookup_dag_callback(), etc....
>>> 
>>> Would this be important.  Don't remember that you get the filter 
>>> support when you use native DAG drivers?
>>> 
>>> Carter
>>> 
>>> 
>>> On May 17, 2011, at 12:52 PM, Leif Tishendorf wrote:
>>> 
>>>> With Argus 3.0.5.2 I don't see a dag option in the configuration help. So I went ahead and tried to feed it the option anyway and I get:
>>>> 
>>>> root@:~/argus-3.0.5.2# ./configure --with-dag=/usr/local/lib/dag
>>>> configure: WARNING: unrecognized options: --with-dag
>>>> 
>>>> --Leif
>>>> 
>>>> 
>>>> On 05/16/2011 05:20 PM, Carter Bullard wrote:
>>>>> Not at compile time, but at ./configure time.  Try:
>>>>>   ./configure --help
>>>>> 
>>>>> You should see with-dag=dir
>>>>> 
>>>>> Provide the path to the dag code distribution.
>>>>> Carter
>>>>> 
>>>>> 
>>>>> On May 16, 2011, at 6:52 PM, Leif Tishendorf<ltishend at gmail.com>   wrote:
>>>>> 
>>>>>>> Also, not sure if I remember that you're using dags?  If so are 
>>>>>>> you using the native dag driver or the libpcap interface to the dags?
>>>>>> 
>>>>>> I'm not sure on that one. Can Argus read the Endance ERF format?  I have Argus complied against the Dag enabled libpcap, but if they can understand ERF that would cut down on some overhead.  I don't see an option in the Argus configuration script to point it at the Dag drivers for compile time.
>>>>>> 
>>>>>> -Leif
>>>>>> 
>>>>>> On 05/16/2011 03:24 PM, Carter Bullard wrote:
>>>>>>> Hey Leif,
>>>>>>> Argus passes this filter down to libpcap, so the filter needs to be formulated as if you were using it with tcpdump.
>>>>>>> Play with tcpdump() to figure out the right filter.  "not proto 
>>>>>>> gre" is much different than "not ip proto gre", so not sure if 
>>>>>>> your filter works or not.  Also, not sure if I remember that you're using dags?  If so are you using the native dag driver or the libpcap interface to the dags?
>>>>>>> 
>>>>>>> Carter
>>>>>>> 
>>>>>>> On May 16, 2011, at 5:40 PM, Leif Tishendorf wrote:
>>>>>>> 
>>>>>>>> Hey Carter or anyone else really,
>>>>>>>> 
>>>>>>>> I have a question about usage of "ARGUS_FILTER" in argus.conf.  We have a significant amount of GRE traffic on the network that I don't care about and I'm trying to filter it out using "ARGUS_FILTER="not proto gre"", but I'm still seeing it in the Argus records.  Not sure if I'm doing it right.  Any help is much appreciated.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> 
>>>>>>>> --Leif
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> --Leif
>>>>>> 
>>>> 
>>>> --
>>>> --Leif
>>>> 
>>> 
>> 
>> --
>> --Leif
>> 
> 