libpcap/DAG build issues

Mon Feb 28 17:07:54 EST 2011

Hi Carter,

Looks pretty good. The only issue I see is that if the user runs 
configure with --with-libpcap then pcap-config is not used.

I am attaching a patch for acsite.m4 which I believe would add this, can 
you take a look?

./configure --with-libpcap=/home/local/ENDACE/sfd/github/libpcap
...
checking for specified library... 
/home/local/ENDACE/sfd/github/libpcap/libpcap.a  -ldag -lpthread
checking for specified pcap.h... found
checking for pcap_list_datalinks... yes
checking for pcap_set_datalink... yes
checking for pcap_datalink_name_to_val... yes
checking for pcap_datalink_val_to_description... yes
checking for pcap_get_selectable_fd... yes
checking for pcap_next_ex... yes
checking for pcap_dump_ftell... yes
...

Thanks,
Stephen.

On 01/03/11 03:48, Carter Bullard wrote:
> Hey Stephen,
> I uploaded a new argus this weekend, that has the changes in it for DAG.
> Could you give this a glance?
>
>     http://qosient.com/argus/dev/argus-latest-tar.gz
>
> Thanks for all the help !!!!
>
> Carter
>
>
> On Feb 22, 2011, at 4:49 PM, Stephen Donnelly wrote:
>
>> Hi Carter,
>>
>> Thanks, I'll take a look at the new version when I see it.
>>
>> I'm not familiar with the DAG support in Gargoyle. Argus appears to use older deprecated DAG APIs which pre-date multiple streams etc.
>>
>> Libpcap can be used to capture from DAG cards as an alternative since libpcap supports multiple DAG card streams as separate libpcap 'interfaces'. This may mean that native DAG support could be dropped in Argus, however there are some drawbacks.
>>
>> The software overhead of packet capture is increased when using libpcap. Although the packet payload is still available directly memory mapped, the pcap header must be constructed by software including the timestamp conversion.
>>
>> The timestamp resolution in libpcap of 1 microsecond is degraded from the DAG native format of ~233ps. Most DAG cards have a timestamp precision of 7.5ns. Converting ERF records to 'traditional' libpcap DLTs also loses metadata such as the capture interface port, error indications, and in-band packet loss counts.
>>
>> If Argus does not make use of the additional ERF metadata or the timestamp precision then this may be acceptable?
>>
>> Alternatively you can set libpcap to 'DLT_ERF' which delivers complete ERF records wrapped in libpcap frames. This delivers all the metadata and higher resolution timestamp, but the libpcap overheads are still incurred so native APIs are better.
>>
>> When capturing InfiniBand x4 SDR (DAG 8.4I) or x4 DDR (DAG 8.5I) you can use the native APIs to receive ERF_TYPE_INFINIBAND records, or set libpcap to DLT_ERF to receive these records wrapped in libpcap records. There is no native libpcap DLT for InfiniBand. If you are capturing 'InfiniBand over Ethernet' E.g. RoCEE using a 10G Ethernet DAG card (E.g. DAG 9.2X2) the frames would be captured as regular Ethernet records via DAG APIs or libpcap.
>>
>> We may be able to assist with updating the Argus DAG code if desired, and if you do not already have this available in Gargoyle. Let me know what you want to do.
>>
>> Regards,
>> Stephen.
>>
>> -----Original Message-----
>> From: Carter Bullard [mailto:carter at qosient.com]
>> Sent: Wednesday, 23 February 2011 9:00 a.m.
>> To: Stephen Donnelly
>> Cc: argus-info at lists.andrew.cmu.edu
>> Subject: Re: [ARGUS] libpcap/DAG build issues
>>
>> Hey Stephen,
>> I decided to port the pcap-config sections to the AC_QOSIENT_LIBPCAP, rather than go AC_LBL_LIBPCAP.
>> I'm uploading argus-3.0.3.23 later today, so please give it a try if you have the time.
>>
>> We have native dag driver support in gargoyle, but for argus we're going libpcap based dag support.
>> Is this good for you guys?  I'll put native in argus-3.0.6 if there is a need.  I'm putting the Infiniband support
>> from gargoyle into argus in 3.0.6, do I need to use native drivers to get Infiniband packets?
>>
>> Carter
>>
>> On Feb 16, 2011, at 3:40 PM, Stephen Donnelly wrote:
>>
>>> Hi Carter,
>>>
>>> The tcpdump 4.1.1 version from tcpdump.org should be fine. It appears to be backwards compatible for building against older libpcap versions, pcap-config is optional.
>>>
>>> The Argus specific modifications in AC_QOSIENT_LIBPCAP seem to be mostly around preferring libpcap variants such as lpcap or wpcap if available? If so it may be cleaner to make those separate tests, and call AC_LBL_LIBPCAP if they fail.
>>>
>>> Regards,
>>> Stephen.
>>>
>>> -----Original Message-----
>>> From: Carter Bullard [mailto:carter at qosient.com]
>>> Sent: Thursday, 17 February 2011 12:28 a.m.
>>> To: Stephen Donnelly
>>> Cc: argus-info at lists.andrew.cmu.edu
>>> Subject: Re: [ARGUS] libpcap/DAG build issues
>>>
>>> Hey Stephen,
>>> I suspect AC_LBL_LIBPCAP will be fine.  Which version should I grab?  From the latest tcpdump?  Is it backward compatible?
>>>
>>> Carter
>>>
>>>
>>> On Feb 15, 2011, at 9:48 PM, Stephen Donnelly<Stephen.Donnelly at endace.com>  wrote:
>>>
>>>> I have received some reports of issues compiling the current Argus release against recent versions of libpcap built against the current Endace DAG software release.
>>>>
>>>> There seem to be two issues. Firstly static builds of libpcap do not currently work correctly when using the latest DAG software releases. I will submit patches upstream to resolve these issues for static libpcap builds. This will resolve issues building applications that link against static libpcap libraries, provided they use the 'pcap-config' mechanism to pick up additional library dependencies. It will take time any changes to be incorporated into a stable libpcap release version.
>>>>
>>>> Dynamic (shared object) builds of libpcap are currently working correctly with the latest DAG software releases and applications dynamically linking against libpcap are not seeing any issues.
>>>>
>>>> Secondly the Argus aclocal.m4/acsite.m4 defines AC_QOSIENT_LIBPCAP rather than using the upstream AC_LBL_LIBPCAP. This appears to be based on an older version of AC_LBL_LIBPCAP and does not support the 'pcap-config' method. This will not work with the libpcap DAG updates and may not be working for other static libpcap builds with external dependencies.
>>>>
>>>> AC_QOSIENT_LIBPCAP also may not find local installs of libpcap built from git as the egrep regex has not been updated for the git version name scheme (-PRE-GIT).
>>>>
>>>> The simplest solution for these issues may be to pull AC_LBL_LIBPCAP from tcpdump upstream. Argus could call this from inside AC_QOSIENT_LIBPCAP either before or after looking for libpcap variants such as lpcap, wpcap etc? Alternatively at minimum AC_QOSIENT_LIBPCAP needs to be updated to support 'pcap-config'.
>>>>
>>>> Let me know if you want me to submit a patch.
>>>>
>>>> Regards,
>>>> Stephen.
>>>> -- 
>>>> Dr Stephen Donnelly
>>>> Core Software Manager
>>>> +64 7 959 2640
>>>> stephen.donnelly at endace.com
>>>> Level 9, 85 Alexandra St,
>>>> Hamilton 3204, New Zealand.
>>>> www.endace.com, follow us on Twitter
>>>>
>>>>
>>>>
>>

-- 

endace

*Stephen Donnelly
Core Software Manager*

+64 7 959 2640 x740
stephen at endace.com <mailto:stephen at endace.com>
linkedin <http://www.linkedin.com/pub/stephen-donnelly/1/129/278>, skype 
<skype:stephen.donnelly?add>
Level 9, 85 Alexandra St,
Hamilton 3204, New Zealand.

www.endace.com <http://www.endace.com>, follow us on Twitter 
<http://twitter.com/endacecorp>
power to see all

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110301/2d642cee/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: logo.gif
Type: image/gif
Size: 990 bytes
Desc: http://www.endace.com/
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110301/2d642cee/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: power-to-see-all.jpg
Type: image/jpeg
Size: 4700 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110301/2d642cee/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: argus-3.0.3.23-sfd.patch
Type: text/x-patch
Size: 1054 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110301/2d642cee/attachment.bin>