Argus on Bivio 7500
Jason Carr
jcarr at andrew.cmu.edu
Wed Aug 5 16:34:39 EDT 2009
So a few things that I've noticed as I'm testing:
For our setup we will be using monitoring three different networks.
For that I believe we'll be using the argus source ID to figure out
where the flow came from.
Inside of the radium config there is a list of specific nodes that
have argus running. I'm not sure how often those nodes would change
but surely in a failover situation the nodes would change. I'm not
sure how to handle that properly. I had originally planned on listing
every node in our stack in the radium config, but that's somewhat messy.
Is there a way for radium to only supply a specific source ID to a
network connection? That way we can provide border argus feeds to
researchers without giving them other networks.
The other thing I noticed is that there is no way to start argus
listening on the interface called "default". According to the Bivio
manual, it's a pseudo-interface name that allows capture on all
interfaces that might be bound to that inspection group that the
Bivio's customized pcap handles internally. For example, we have two
interfaces at 10G each from a network that we are monitoring. We'd
want to monitor both interfaces at the same time. It's a lot easier
to monitor "default" instead of monitoring s0.e0 and s1.e0. Right
now, argus just quits with no error message.
Specifying multiple interfaces on the command line does not work
either, for example:
[Bivio] root at CPU-3c0 ~$ /usr/local/sbin/argus -X -U 128 -i s1.e0 -i
s2.e0 -P 561 -e 1 - ip
argus[26239]: 05 Aug 09 15:17:52.400570 ArgusOpenInterface:
pcap_open_live zcopy_open: Can't MMAP to kernel (errno 12)
Thanks Eric and Carter!
- Jason
On Aug 5, 2009, at 12:03 PM, Jason Carr wrote:
> Depending on what the method is for traffic splitting you can get
> random packets or proper flows so this answer could be yes and no at
> the same time.
>
> For example, this is a load-balance by port in a traffic
> classification configuration which will load balance using a five
> tuple (sip, dip, sp, dp, proto) which should make one flow stick
> with one processor:
>
> <CIG:Action type="load-balance-by-port">core-ig</CIG:Action>
>
>
> On Aug 4, 2009, at 10:52 PM, Eric Gustafson wrote:
>
>> Hey Carter,
>> I posed that very question regarding the packet dispatch logic to a
>> Bivio engineer last year. He seemed not too sure of himself, but
>> claimed that it should separate things on a "per-connection"
>> basis. Now that I can see the flows though, I'll verify this in
>> the next day or so.
>>
>> My bosses are super excited to have this working, as they love the
>> data argus gives them, and now have it in a 10G-capable box!
>> Cheers,
>> Eric
>>
>> Carter Bullard wrote:
>>> Hey Eric,
>>> Excellent!!!! I'm working on a hardware area on the web site with
>>> Peter Van Epp, and
>>> I'll put something up for Bivio (and then of course we'll need
>>> something for Endace, and
>>> Tilera, and ....). It will be interesting to see if the packet
>>> dispatch logic in the Bivio is
>>> flow sensitive, such that a single CPU see's all the packets for
>>> any given flow. If not,
>>> we'll need to insert a rabins() into the pipe to merge the records
>>> together, so that the
>>> device looks like a single monitor.
>>>
>>> Thanks again, and lets keep sharing notes!!!
>>>
>>> Carter
>>>
>>>
>>> On Aug 4, 2009, at 5:53 PM, Eric Gustafson wrote:
>>>
>>>> Hey Carter,
>>>> Good news!! It works!
>>>> I've attached the Bivio-style init scripts and NRSP profiles I
>>>> created. Inside the tarball is a readme too, for those not
>>>> familiar with Bivio's examples (which I used to create these)
>>>> When started with NRSP, a separate argus process will be spawned
>>>> for each loadshared CPU installed, and will listen for connections.
>>>> When started with NRSP (and the included radium.conf), radium
>>>> will combine all the CPUs' data together into one feed, which can
>>>> be accessed locally on CPU-X or the mgt0 management interface.
>>>> Of course, our Bivio boxes only have 2x 2-core CPUs, so if you
>>>> have the expansion module, you'll need to change radium.conf or
>>>> you'll be missing some traffic.
>>>>
>>>> Feel free to include these in the support directory if you want.
>>>>
>>>> Thanks again for your help!
>>>> Eric
>>>>
>>>> On Fri, Jul 31, 2009 at 11:24 AM, Eric Gustafson
>>>> <subwire at gmail.com <mailto:subwire at gmail.com>> wrote:
>>>>
>>>> Hey Carter,
>>>> I'm happy to test those Bivio-related mods for you any
>>>> time. Just let me know.
>>>>
>>>> rasplit is holding up fine with our script that restarts it every
>>>> time the argus daemon notices it disconnect.
>>>> I'll check out those fixes and get back to you on that though!
>>>>
>>>> Thanks!
>>>> - Eric
>>>>
>>>>
>>>> On Mon, Jul 27, 2009 at 7:17 AM, Carter Bullard
>>>> <carter at qosient.com <mailto:carter at qosient.com>> wrote:
>>>>
>>>> Hey Guys,
>>>> I'd like to work on specific Bivio mods for argus-3.0.2 this
>>>> week. I've got most of the code
>>>> structured to conditionally not use pcap_next_ex now, is
>>>> there a chance we can test
>>>> it out sometime this week or next?
>>>>
>>>> Eric, how has your rasplit() held up? There were very
>>>> specific rasplit() mods in some of
>>>> the recent updates, did you get a chance to grab new code and
>>>> try them out?
>>>>
>>>> Thanks for all the help!!!
>>>>
>>>> Carter
>>>>
>>>>
>>>> On Jun 22, 2009, at 9:39 PM, Jason Carr wrote:
>>>>
>>>> Hi Eric,
>>>>
>>>> We also had the same problem compiling the 3.x series on
>>>> our Bivio units. Bivio ships (even with the newest OS
>>>> 5.0.5) with an older libpcap. We were told that the new
>>>> libpcap that implements the pcap_get_selectable_fd method
>>>> is in beta and should be released with the next OS
>>>> release.
>>>>
>>>> Right now we're running argus 2.x and running rastream
>>>> 3.x on a non-Bivio machine. The 2.x series compiles just
>>>> fine (but no IPv6).
>>>>
>>>> This was before Carter implemented any sort of Bivio
>>>> changes, so I have not tested those.
>>>>
>>>> Let me know if you have any questions. I'm also
>>>> interested in what else you might be using your Bivio
>>>> for.
>>>>
>>>> - Jason
>>>>
>>>>
>>>> On Jun 22, 2009, at 4:49 PM, Eric Gustafson wrote:
>>>>
>>>> Hi Carter et al,
>>>> I'm trying to compile the latest test argus (3.0.2
>>>> beta8) on one of our Bivio 7500s, and am running into
>>>> linking trouble.
>>>>
>>>> gcc -O3 -I. -I./../include -DPACKAGE_NAME=\"\"
>>>> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
>>>> -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\"
>>>> -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1
>>>> -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1
>>>> -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1
>>>> -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1
>>>> -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_STRING_H=1
>>>> -DHAVE_FCNTL_H=1 -DHAVE_SYS_FILE_H=1
>>>> -DHAVE_SYSLOG_H=1 -DHAVE_SYS_VFS_H=1
>>>> -DHAVE_VFPRINTF=1 -DHAVE_STRCASECMP=1 -DHAVE_STRDUP=1
>>>> -DHAVE_STRFTIME=1 -DHAVE_SETLINEBUF=1 -DHAVE_ALARM=1
>>>> -DHAVE_STRERROR=1 -DHAVE_STRTOF=1
>>>> -DHAVE_SYS_BITYPES_H=1 -DHAVE_INTTYPES_H=1
>>>> -DHAVE_VSNPRINTF=1 -DHAVE_SNPRINTF=1
>>>> -DHAVE_GETADDRINFO=1 -DHAVE_ETHER_HOSTTON=1
>>>> -DHAVE_NETINET_ETHER_H=1
>>>> -DNETINET_ETHER_H_DECLARES_ETHER_HOSTTON=
>>>> -DHAVE_DECL_ETHER_HOSTTON=1 -D_FILE_OFFSET_BITS=64
>>>> -DHAVE_TCP_WRAPPER=1 -DLBL_ALIGN=1 -DSTDC_HEADERS=1
>>>> -DARGUS_SYSLOG=1 -o ../bin/argus argus.o
>>>> ArgusModeler.o ArgusSource.o ArgusUtil.o
>>>> ArgusOutput.o ArgusUdp.o ArgusTcp.o ArgusIcmp.o
>>>> ArgusIgmp.o ArgusEsp.o ArgusArp.o ArgusFrag.o
>>>> ArgusAuth.o ArgusApp.o ../lib/libpcap.a -lwrap -lnsl
>>>> ../lib/argus_common.a -lm
>>>> ArgusSource.o: In function
>>>> `ArgusGetPackets':ArgusSource.c:(.text+0x2cf8):
>>>> undefined reference to `pcap_get_selectable_fd'
>>>> :ArgusSource.c:(.text+0x2d90): undefined reference to
>>>> `pcap_next_ex'
>>>> :ArgusSource.c:(.text+0x2dcc): undefined reference to
>>>> `pcap_next_ex'
>>>> :ArgusSource.c:(.text+0x2e08): undefined reference to
>>>> `pcap_next_ex'
>>>> :ArgusSource.c:(.text+0x2e44): undefined reference to
>>>> `pcap_next_ex'
>>>> :ArgusSource.c:(.text+0x2eac): undefined reference to
>>>> `pcap_next_ex'
>>>> ArgusSource.o:ArgusSource.c:(.text+0x2ec8): more
>>>> undefined references to `pcap_next_ex' follow
>>>> collect2: ld returned 1 exit status
>>>> make[1]: *** [../bin/argus] Error 1
>>>> make[1]: Leaving directory
>>>> `/bivio/shared/root/argus-3.0.1.beta.3/argus'
>>>> ### Done with /root/argus-3.0.1.beta.3/argus
>>>>
>>>> I configured with --with-libpcap=/usr/lib/zcp/, which
>>>> is where Bivio stashes its special version of
>>>> libpcap.
>>>> I noticed your mention of "changes to support Bivio
>>>> hardware" for this release, but I didn't see any
>>>> instructions regarding extra steps to get it to work.
>>>> Any ideas?
>>>>
>>>> Thanks so much,
>>>> Eric
>>>>
>>>>
>>>>
>>>>
>>>> Carter Bullard
>>>> CEO/President
>>>> QoSient, LLC
>>>> 150 E 57th Street Suite 12D
>>>> New York, New York 10022
>>>>
>>>> +1 212 588-9133 Phone
>>>> +1 212 588-9134 Fax
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> <argus-bivio-support-files.tar.gz>
>>>
>>
>>
>
>
More information about the argus
mailing list