Argus and PF_RING ZC drivers
Craig Merchant
craig.merchant at oracle.com
Tue Oct 6 20:11:44 EDT 2015
I’ve looked through the Makefile and I honestly have no idea how to add those libraries to it. Can you point me in the right direction?
Thx.
C
From: Carter Bullard [mailto:carter at qosient.com]
Sent: Tuesday, October 06, 2015 3:46 PM
To: Craig Merchant
Cc: argus-info at lists.andrew.cmu.edu
Subject: Re: [ARGUS] Argus and PF_RING ZC drivers
Hey Craig,
Just edit the Makefile in ./argus to add the library. Better to do it by hand that to try to get automaker to figure it out, at least in the short term !!
You are in uncharted waters for me ... but I'll make any changes needed !!!
Carter
http://qosient.com/
HYPERLINK "mailto:carter at qosient.com"Carter Bullard • CTO
150 E 57th Street Suite 12D
New York, New York 10022-2795
Phone +1.212.588.9133 • Mobile +1.917.497.9494
On Oct 6, 2015, at 6:30 PM, Craig Merchant <HYPERLINK "mailto:craig.merchant at oracle.com"craig.merchant at oracle.com> wrote:
I did some more digging around… If I specify /opt/PF_RING/userland/libpcap/libpcap.a instead of just the directory, Argus finds what it needs and will complete the “make” process successfully.
Luca Deri from NTOP says I need to “ add /opt/PF_RING/userland/lib/libpfring.a after libpcap.a”. I tried the following:
./configure –with-libpcap=/opt/PF_RING/userland/libpcap –with-pfring=/opt/PF_RING/userland/lib/libpfring.a
But that gives me a warning at the end:
configure: WARNING: unrecognized options: --with-pfring
How can I add the pfring files that the developer says are needed?
Thx.
C
From: Craig Merchant
Sent: Tuesday, October 06, 2015 2:35 PM
To: HYPERLINK "mailto:argus-info at lists.andrew.cmu.edu"argus-info at lists.andrew.cmu.edu
Subject: Re: [ARGUS] Argus and PF_RING ZC drivers
I tried compiling the Argus network flow tool against the libpcap libraries from pf_ring:
./configure –with-libpcap=/opt/PF_RING/userland/libpcap
The configure script doesn’t throw any errors, but it doesn’t seem to find everything it wants:
checking for specified library... /opt/PF_RING/userland/libpcap/libpcap.a
checking for specified pcap.h... found
checking for pcap_list_datalinks... no
checking for pcap_set_datalink... no
checking for pcap_datalink_name_to_val... no
checking for pcap_set_buffer_size... no
checking for pcap_fopen_offline... no
checking for pcap_get_selectable_fd... no
checking for pcap_next_ex... no
checking for pcap_dump_ftell... no
checking for pcap_dump_flush... no
Running make, however, throws a ton of errors:
make[1]: Entering directory `/home/craig.merchant/argus-3.0.8.2.rc.2/argus'
gcc -O -I. -I/opt/PF_RING/userland/libpcap -I./../include -DHAVE_CONFIG_H -o ../bin/argus argus.o ArgusModeler.o ArgusSource.o ArgusUtil.o ArgusOutput.o ArgusUdp.o ArgusTcp.o ArgusIcmp.o ArgusIgmp.o ArgusEsp.o ArgusArp.o ArgusFrag.o ArgusUdt.o ArgusLcp.o ArgusIsis.o ArgusAuth.o Argus802.11.o ArgusApp.o ArgusEvents.o ArgusNetflow.o ArgusSflow.o /opt/PF_RING/userland/libpcap/libpcap.a -lpthread -lm ../lib/argus_common.a -lm
/opt/PF_RING/userland/libpcap/libpcap.a(pcap.o): In function `pcap_breakloop':
/opt/PF_RING/userland/libpcap/./pcap.c:882: undefined reference to `pfring_breakloop'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_read_packet':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1628: undefined reference to `pfring_recv'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_stats_linux':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:2071: undefined reference to `pfring_stats'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_cleanup_linux':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1088: undefined reference to `pfring_close'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_setfilter_linux_common':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:2630: undefined reference to `pfring_get_bound_device_ifindex'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_activate_linux':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1484: undefined reference to `pfring_enable_ring'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1485: undefined reference to `pfring_get_selectable_fd'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1316: undefined reference to `pfring_open'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1320: undefined reference to `pfring_set_socket_mode'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1347: undefined reference to `pfring_set_poll_watermark'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1345: undefined reference to `pfring_enable_rss_rehash'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1341: undefined reference to `pfring_set_application_name'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1325: undefined reference to `pfring_set_cluster'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1327: undefined reference to `pfring_set_cluster'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1329: undefined reference to `pfring_set_cluster'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1331: undefined reference to `pfring_set_cluster'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1333: undefined reference to `pfring_set_cluster'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o):/opt/PF_RING/userland/libpcap/./pcap-linux.c:1335: more undefined references to `pfring_set_cluster' follow
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_get_pfring_id':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:6377: undefined reference to `pfring_get_ring_id'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_set_watermark':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:6404: undefined reference to `pfring_set_poll_watermark'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_setdirection_linux':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:2733: undefined reference to `pfring_set_direction'
/opt/PF_RING/userland/libpcap/./pcap-linux.c:2733: undefined reference to `pfring_set_direction'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_inject_linux':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:2018: undefined reference to `pfring_send'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_set_appl_name_linux':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1502: undefined reference to `pfring_set_application_name'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_set_cluster':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:1508: undefined reference to `pfring_set_cluster'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_set_master_id':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:6383: undefined reference to `pfring_set_master_id'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_set_master':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:6389: undefined reference to `pfring_set_master'
/opt/PF_RING/userland/libpcap/libpcap.a(pcap-linux.o): In function `pcap_set_application_name':
/opt/PF_RING/userland/libpcap/./pcap-linux.c:6395: undefined reference to `pfring_set_application_name'
collect2: error: ld returned 1 exit status
make[1]: *** [../bin/argus] Error 1
make[1]: Leaving directory `/home/craig.merchant/argus-3.0.8.2.rc.2/argus'
### Done with /home/craig.merchant/argus-3.0.8.2.rc.2/argus
### Making in /home/craig.merchant/argus-3.0.8.2.rc.2/events
make[1]: Entering directory `/home/craig.merchant/argus-3.0.8.2.rc.2/events'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/craig.merchant/argus-3.0.8.2.rc.2/events'
### Done with /home/craig.merchant/argus-3.0.8.2.rc.2/events
Any ideas what’s happening here?
Thx.
C
From: Alfredo Cardigliano [mailto:cardigliano at ntop.org]
Sent: Tuesday, October 06, 2015 1:46 PM
To: HYPERLINK "mailto:ntop-misc at listgateway.unipi.it"ntop-misc at listgateway.unipi.it
Subject: Re: [Ntop-misc] How to make an application "PF_RING aware"
Hi Craig
in order to run a pcap-based application on top of ZC, you need to link it to our pf_ring-aware libpcap,
that’s how the tcpdump included in pf_ring works.
Alfredo
On 06 Oct 2015, at 21:23, Craig Merchant <HYPERLINK "mailto:craig.merchant at oracle.com"craig.merchant at oracle.com> wrote:
I’ve installed the ZC drivers on an OEL 7 server. I’ve tried getting both Splunk Stream and Argus to read the ZC interfaces, but neither of them see traffic. The tcpdump included with the OS can’t see traffic on those interfaces, but the one included with ZC can.
What does an application developer have to do to make an application see traffic on a ZC interface? I’m not a developer, so feel free to use small words. I’m just curious what the process is and how much work is required to add that functionality.
Thanks.
C
_______________________________________________
Ntop-misc mailing list
HYPERLINK "mailto:Ntop-misc at listgateway.unipi.it"Ntop-misc at listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
-----Original Message-----
From: Jesse Bowling [mailto:jessebowling at gmail.com]
Sent: Tuesday, October 06, 2015 9:31 AM
To: Craig Merchant
Cc: HYPERLINK "mailto:argus-info at lists.andrew.cmu.edu"argus-info at lists.andrew.cmu.edu
Subject: Re: [ARGUS] Argus and PF_RING ZC drivers
I'm experiencing a similar issue (at least it also applies to ZC drivers)...In my case I've worked around by having the included tcpdump read the interface, and write output to a FIFO pipe on the filesystem; I then have argus "read" that FIFO and generate data...i.e.:
mknod -p /tmp/argus
tcpdump -nn -i zc:99 at 0 -s 1600 -w /tmp/argus argus -F /etc/argus.conf -f -r /tmp/argus
It works, but I can't speak to the additional load that creates by using a FIFO...
Craig, you might try quoting your interface command line and using at least argus-3.0.8.2.rc.2...
Specifically, when I try to specify a ZC interface I I don't get any packets in:
# argus -D 4 -F /etc/argus.conf -i 'zc:99 at 0'
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041453 ArgusNewModeler() returning 0x7fb5c4603010
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041569 ArgusNewSource(0x7fb5c4603010) returning 0x7fb5c30dd010
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041586 ArgusNewQueue () returning 0x1a384a0
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041596 ArgusNewList () returning 0x1a38540
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041606 ArgusNewList () returning 0x1a385e0
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041613 ArgusNewOutput() returning retn 0x1a37a20
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041633 setArgusMarReportInterval(60) returning
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041715 setArgusPortNum(561) returning
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041731 ArgusNewList () returning 0x1a388c0
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041741 ArgusParseResourceFile: ArgusBindAddr "(null)"
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041780 setArgusMarReportInterval(60) returning
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041847 ArgusParseResourceFile (/etc/argus.conf) returning
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041861 clearArgusDevice(0x7fb5c30dd010) returning
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041872 ArgusNewList () returning 0x1a38680
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041882 setArgusDevice(zc:99 at 0 ) returning
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.041890 setArgusInterfaceStatus(0x7fb5c30dd010, 1)
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.055180 ArgusGenerateInitialMar() returning
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.055246 ArgusEstablishListen(0x1a37a20, 0x7ffeb3610200) binding: 127.0.0.1:561 family: 2
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.055276 ArgusEstablishListen(0x1a37a20, 0x7ffeb3610200) returning 3
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.055322 ArgusInitOutput() done
ArgusAlert: 06 Oct 15 11:35:15.055346 started
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.055363 ArgusNewList () returning 0x1ae7de0
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.055376 ArgusCloneSource(0x7fb5c30dd010) returning 0x7fb5c22c5010
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.055385 clearArgusDevice(0x7fb5c22c5010) returning
argus[6265.0007edc2b57f0000]: 06 Oct 15 11:35:15.055433 ArgusOutputProcess(0x1a37a20) starting
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.064100 Arguslookup_pcap_callback(1) returning 0x417b0a
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.064125 ArgusOpenInterface(0x7fb5c22c5010, 'zc:99 at 0') returning 1
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.064829 ArgusNewHashTable (65536) returning 0x1af8d20
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.064852 ArgusNewQueue () returning 0x1af8de0
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.064862 ArgusNewQueue () returning 0x1af8e80
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.064895 ArgusInitModeler(0x7fb5c44f1010) done
argus[6265.404768c4b57f0000]: 06 Oct 15 11:35:15.064904 ArgusInitSource(0x7fb5c22c5010) returning 1
argus[6265.00d783c1b57f0000]: 06 Oct 15 11:35:15.064951 ArgusGetPackets (0x7fb5c22c5010) starting
argus[6265.00d783c1b57f0000]: 06 Oct 15 11:35:15.064986 setArgusInterfaceStatus(0x7fb5c22c5010, 1)
argus[6265.00d783c1b57f0000]: 06 Oct 15 11:35:15.065003 ArgusGetPackets: interface is selectable
argus[6265.00d783c1b57f0000]: 06 Oct 15 11:35:15.065012 setArgusInterfaceStatus(0x7fb5c22c5010, 1)
argus[6265.00d783c1b57f0000]: 06 Oct 15 11:35:15.265347 setArgusInterfaceStatus(0x7fb5c22c5010, 1)
argus[6265.00d783c1b57f0000]: 06 Oct 15 11:35:15.765989 setArgusInterfaceStatus(0x7fb5c22c5010, 1)
argus[6265.00d783c1b57f0000]: 06 Oct 15 11:35:16.266622 setArgusInterfaceStatus(0x7fb5c22c5010, 1)
argus[6265.00d783c1b57f0000]: 06 Oct 15 11:35:16.767294 setArgusInterfaceStatus(0x7fb5c22c5010, 1) <snip; last line just repeats indefinitely>
Cheers,
Jesse
> On 2015/10/6, at 11:07, Craig Merchant <HYPERLINK "mailto:craig.merchant at oracle.com"craig.merchant at oracle.com> wrote:
>
> Hey, Carter…
>
> I’m trying to get Argus to recognize my PF_RING ZC interfaces, but it says it can’t find them.
>
> I see the following at line 4436 of ArgusSource.c:
>
> if ((strstr(device->name, "dag")) || (strstr(device->name, "napa")) ||
> (strstr(device->name, "dna")) || (strstr(device->name, "zc")) ||
> ((strstr(device->name, "eth")) && (strstr(device->name, "@"))))
> {
>
> It looks to me like you’ve compiled support for both ZC and the old DNA/libzero interfaces into Argus. I’m running the following ZC client to fan out my network traffic:
>
> zbalance_ipc -i enp48s0f0,enp48s0f1 -c 10 -n 4,1 -m 1 –d
>
> That means my interfaces are zc:0, zc:1, zc:2, and zc:3 for the load balanced traffic and zc:4 for the second full copy of the traffic. Argus doesn’t recognize any of them as valid interfaces:
>
> ArgusWarning: 05 Oct 15 23:07:12.848794 ArgusOpenInterface zc:10 at 4:
> SIOCGIFHWADDR: No such device
>
> The tcpdump that ships with OEL 7 can’t see them either, but the pf_ring aware version that comes with the ZC drivers sees traffic on those interfaces.
>
> Is there something I can do to make Argus aware of the pf_ring ZC interfaces?
>
> Thanks!
>
> C
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20151006/b46773ce/attachment.html>
More information about the argus
mailing list