Argus & IPFIX?

Drew Dixon dwdixon at umich.edu
Tue Oct 24 17:47:01 EDT 2017 

Thanks Carter, I re-compiled as you directed and now have debug
functionality running argus-clients-3.0.8.2...still seeing IPFIX data
coming in via 192.xx.xx.xx:9995 (UDP) in a steady stream via tcpdump.

I ran radium again with debug set to 6 and using the config file parameters
I mentioned earlier in the thread.  I keep getting repeated
"ArgusClientTimeout()" after radium binds to the 192.xx.xx.xx:9995 network
interface on the same server to listen for/collect the IPFIX data.  I also
tried running ra as you suggested, with ipfix:// ra just exits immediately,
with cisco:// ra also repeatedly gives me "ArgusClientTimeout()" after
binding to the server NIC's IP on udp/9995...all debug log results are
below (level 6).

Not certain but guessing this means it's not receiving/recognizing any of
the incoming IPFIX data or something else?


***********************************************************************
Oct 24 17:20:54 argus1 systemd: Started radium Service.
Oct 24 17:20:54 argus1 systemd: Starting radium Service...
Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582224 ArgusCalloc (1, 560) returning 0x3113e80
Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582308 ArgusCalloc (1, 112) returning 0x31137c0
Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582319 ArgusCalloc (1, 80) returning 0x3113840
Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582327 ArgusNewQueue () returning 0x3113840
Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582336 ArgusCalloc (1, 56) returning 0x31138a0
Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582349 ArgusCalloc (65536, 8) returning 0xe0bd8010
Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582357 ArgusNewHashTable (65536) returning 0x31138a0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582566 setArgusMarReportInterval(15) returning
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582854 ArgusCalloc (1, 461728) returning 0xdfdf5010
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582894 ArgusAddHostList (0xe0c59010, 192.xx.xx.xx:9995, 16, 17)
returning 1
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582929 RadiumParseResourceFile: ArgusBindAddr "127.0.0.1"
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582959 ArgusCalloc (1, 144) returning 0x3114390
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.582990 ArgusNewList () returning 0x3114390
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.583032 ArgusCalloc (1, 296) returning 0x3114460
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.583060 ArgusPushFrontList (0x3114390, 0x3114460, 1) returning
0x3114970
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.583538 RadiumParseResourceFile (/etc/radium.conf) returning 0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.583589 ArgusEstablishListen(0xe0c59010, 561, 127.0.0.1) binding:
127.0.0.1:561
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584032 ArgusEstablishListen(0xe0c59010, 561, 127.0.0.1) returning 3
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584073 ArgusCalloc (1, 392) returning 0x31140c0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584102 ArgusCalloc (1, 80) returning 0x3114b40
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584129 ArgusNewQueue () returning 0x3114b40
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584153 ArgusNewOutput() returning retn 0x31140c0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584178 ArgusCalloc (1, 144) returning 0x3114250
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584202 ArgusNewList () returning 0x3114250
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584227 ArgusCalloc (1, 128) returning 0x3115da0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584251 ArgusGenerateInitialMar() returning
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584282 ArgusCalloc (1, 168) returning 0x3115e30
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584355 ArgusCalloc (1, 65792) returning 0x3115ee0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584388 ArgusCalloc (1, 144) returning 0x3125ff0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584412 ArgusNewList () returning 0x3125ff0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584436 ArgusNewSocket (4) returning 0x3115ee0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584479 ArgusPushBackList (0x3114390, 0x3114460, 1) returning 1
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.584965 ArgusInitOutput() done
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.585011 main: reading files completed
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.585040 ArgusCalloc (1, 80) returning 0x31262d0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.585065 ArgusNewQueue () returning 0x31262d0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.585121 ArgusFree (0x31262d0)
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.585149 ArgusDeleteQueue (0x31262d0) returning
Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:54.585174 ArgusReadStream(0x7fc7e0c59010) starting
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.585224 ArgusConnectRemote(0x7fc7dfdf5010) starting
Oct 24 17:20:54 argus1 radium: radium[41360]: 17:20:54.585349 Binding
192.xx.xx.xx:9995 Expecting Netflow records
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.585433 receiving
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.585464 ArgusGetServerSocket (0x7fc7dfdf5010) returning 5
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.585513 ArgusCalloc (1, 4194304) returning 0xce7de010
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.586086 ArgusCalloc (1, 262144) returning 0xce79d010
Oct 24 17:20:54 argus1 radium[41360]: 17:20:54.585349 Binding
192.xx.xx.xx:9995 Expecting Netflow records
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.602162 ArgusInitAddrtoname (0x7fc7e0c59010, 0x0, 0x0)
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.602202 ArgusParseInit(0x7fc7e0c59010 0x7fc7dfdf5010
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.602223 ArgusReadConnection(0xdfdf5010, 2) reading cisco wire format
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.602245 ArgusReadConnection(0xdfdf5010, 2) returning 0
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.602265 ArgusConnectRemote(0x7fc7dfdf5010) connected to 192.xx.xx.xx
Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]:
17:20:54.602285 ArgusConnectRemote() done!
Oct 24 17:20:55 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:55.636383 ArgusClientTimeout()
Oct 24 17:20:56 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:56.637568 ArgusClientTimeout()
Oct 24 17:20:57 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:57.638727 ArgusClientTimeout()
Oct 24 17:20:58 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:58.639887 ArgusClientTimeout()
Oct 24 17:20:59 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:20:59.641141 ArgusClientTimeout()
Oct 24 17:21:00 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:21:00.642402 ArgusClientTimeout()
Oct 24 17:21:01 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]:
17:21:01.643565 ArgusClientTimeout()
Oct 24 17:21:01 argus1 systemd: Stopping radium Service...
***********************************************************************


ra using ipfix://
***********************************************************************
# ra -D6 -S ipfix://192.xx.xx.xx:9995

ra[41491.4037f00ea37f0000]: 17:32:00.930991 ArgusCalloc (1, 461728)
returning 0xed4e010
ra[41491.4037f00ea37f0000]: 17:32:00.931065 ArgusAddHostList (0xedbf010,
ipfix://192.xx.xx.xx:9995, 64, 6) returning 1
ra[41491.4037f00ea37f0000]: 17:32:00.931091 main: reading files completed
ra[41491.4037f00ea37f0000]: 17:32:00.931136 ArgusCalloc (1, 80) returning
0x1545840
ra[41491.4037f00ea37f0000]: 17:32:00.931157 ArgusNewQueue () returning
0x1545840
ra[41491.4037f00ea37f0000]: 17:32:00.931185 ArgusGetServerSocket
(0x7fa30ed4e010) returning -1
ra[41491.4037f00ea37f0000]: 17:32:00.931217 ArgusFree (0x1545840)
ra[41491.4037f00ea37f0000]: 17:32:00.931237 ArgusDeleteQueue (0x1545840)
returning
ra[41491.4037f00ea37f0000]: 17:32:00.931269 ArgusShutDown (0)
ra[41491.4037f00ea37f0000]: 17:32:00.931298 ArgusFree (0x7fa30ed4e010)
ra[41491.4037f00ea37f0000]: 17:32:00.931319 ArgusFree (0x1545190)
ra[41491.4037f00ea37f0000]: 17:32:00.931336 ArgusDeleteQueue (0x1545190)
returning
ra[41491.4037f00ea37f0000]: 17:32:00.931355 ArgusFree (0x15451f0)
ra[41491.4037f00ea37f0000]: 17:32:00.931371 ArgusDeleteQueue (0x15451f0)
returning
ra[41491.4037f00ea37f0000]: 17:32:00.931389 RaParseComplete(caught signal 0)
ra[41491.4037f00ea37f0000]: 17:32:00.931404 ArgusWindowClose () returning
ra[41491.4037f00ea37f0000]: 17:32:00.931420 RaParseComplete(caught signal 0)
ra[41491.4037f00ea37f0000]: 17:32:00.931439 RaParseComplete(caught signal 0)
ra[41491.4037f00ea37f0000]: 17:32:00.931457 ArgusShutDown (0)
ra[41491.4037f00ea37f0000]: 17:32:00.931789 ArgusFree (0x1545050)
ra[41491.4037f00ea37f0000]: 17:32:00.931808 ArgusDeleteList (0x1545050, 4)
returning
ra[41491.4037f00ea37f0000]: 17:32:00.931829 ArgusFree (0x15450f0)
ra[41491.4037f00ea37f0000]: 17:32:00.931842 ArgusDeleteList (0x15450f0, 4)
returning
***********************************************************************

ra using cisco://
***********************************************************************
# ra -D6 -S cisco://192.xx.xx.xx:9995
ra[41498.407743631c7f0000]: 17:33:07.958512 ArgusCalloc (1, 461728)
returning 0x63282010
ra[41498.407743631c7f0000]: 17:33:07.958588 ArgusAddHostList (0x632f3010,
cisco://192.xx.xx.xx:9995, 16, 17) returning 1
ra[41498.407743631c7f0000]: 17:33:07.958612 main: reading files completed
ra[41498.407743631c7f0000]: 17:33:07.958633 ArgusCalloc (1, 80) returning
0x18dc840
ra[41498.407743631c7f0000]: 17:33:07.958664 ArgusNewQueue () returning
0x18dc840
ra[41498]: 17:33:07.958737 Binding 192.xx.xx.xx:9995 Expecting Netflow
records
ra[41498.407743631c7f0000]: 17:33:07.959292 receiving
ra[41498.407743631c7f0000]: 17:33:07.959312 ArgusGetServerSocket
(0x7f1c63282010) returning 3
ra[41498.407743631c7f0000]: 17:33:07.959354 ArgusCalloc (1, 4194304)
returning 0x620ff010
ra[41498.407743631c7f0000]: 17:33:07.959384 ArgusCalloc (1, 262144)
returning 0x620be010
ra[41498.407743631c7f0000]: 17:33:07.977130 ArgusInitAddrtoname
(0x7f1c632f3010, 0x0, 0x0)
ra[41498.407743631c7f0000]: 17:33:07.977161 ArgusParseInit(0x7f1c632f3010
0x7f1c63282010
ra[41498.407743631c7f0000]: 17:33:07.977174 ArgusReadConnection(0x63282010,
2) reading cisco wire format
ra[41498.407743631c7f0000]: 17:33:07.977186 ArgusReadConnection(0x63282010,
2) returning 0
ra[41498.407743631c7f0000]: 17:33:07.977209 ArgusHandleRecord
(0x7f1c63282228, 0x7f1c63414808) returning -1
ra[41498.407743631c7f0000]: 17:33:07.977227 ArgusFree (0x18dc840)
ra[41498.407743631c7f0000]: 17:33:07.977240 ArgusDeleteQueue (0x18dc840)
returning
ra[41498.407743631c7f0000]: 17:33:07.977253 ArgusReadStream(0x7f1c632f3010)
starting
ra[41498.407743631c7f0000]: 17:33:09.228565 ArgusClientTimeout()
ra[41498.407743631c7f0000]: 17:33:10.229714 ArgusClientTimeout()
ra[41498.407743631c7f0000]: 17:33:11.230838 ArgusClientTimeout()
***********************************************************************

Thank you,

-Drew

On Tue, Oct 24, 2017 at 3:45 PM, Carter Bullard <carter at qosient.com> wrote:

> When you run the ./configure program, like many open source distros, you
> can create files, like .debug or .devel,  that ./configure looks for, and
> if they exist, it will configure the package to add specific
> functionality.  If there is a ./.debug file, the ./conifgure script will
> compile in support for printing debug information on the command line ….
> Try this …
>
>    % cd /full/path/to/your/argus/clients/distribution
>    % touch .debug
>    % ./configure
>    % make clean; make
>
> Now all your ra* programs will be able to print detail debugging
> information, using the “-D debuglevel” option on the command line.
> You can also turn on this type of support like this …
>
>    % ./configure —enable-debug=yes
>
> I prefer the .debug file, as you can set it and forget that you need the
> switch on the configure line.
> Please keep all email on the mailing list … Thanks...
>
> Carter
>
>
>
> On Oct 24, 2017, at 3:33 PM, Drew Dixon <dwdixon at umich.edu> wrote:
>
> Whoops...apologies with that- I noticed it said on the downloads page that
> 3.0.8.1 was the current stable version and never thought I had a need to
> look on the main page.
>
> Please forgive my ignorance but I'm not 100% sure what you mean by "with
> the .debug tag" I've listed out my options using ./configure --help but
> don't see anything related to enabling debugging.  Only thing that I can
> see is  --enable-FEATURE[=ARG]  include FEATURE [ARG=yes] is this what
> you're referring to?
>
> --enable-debug=yes ?
>
> Or somehow with referencing the files below?
>
> /argus-clients-3.0.8.2/common/._argus_debug.h
> /argus-clients-3.0.8.2/common/argus_debug.h
> /argus-clients-3.0.8.2/include/._argus_debug.h
> /argus-clients-3.0.8.2/include/argus_debug.h
>
> I'm not certain on how to compile to enable debugging properly in the
> fashion you've described.
>
> I very much appreciate your help.
>
> Sincerely,
>
> -Drew
>
> On Tue, Oct 24, 2017 at 3:09 PM, Carter Bullard <carter at qosient.com>
> wrote:
>
>> How about the link to argus-clients-3.0.8.2 from the main page …
>>
>> http://qosient.com/argus/
>>
>> You should ./configure and compile with the .debug tag in the home
>> directory, and then run your ra command with the -D4 or -D5 option to see
>> if you are receiving any packets.
>> Carter
>>
>> [image: QoSient] <http://qosient.com/>
>> Carter Bullard <carter at qosient.com> • CTO
>> 150 E 57th Street, Suite 12D
>> <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York&entry=gmail&source=g>
>> New York, New York
>> <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York&entry=gmail&source=g>
>> 10022-2795
>> Phone +1.212.588.9133 • Mobile +1.917.497.9494
>>
>>
>>
>> On Oct 24, 2017, at 2:15 PM, Drew Dixon <dwdixon at umich.edu> wrote:
>>
>> I'm using the latest stable version (argus-clients-3.0.8) downloaded from
>> the website (http://www.qosient.com/argus/downloads.shtml).
>>
>> That command doesn't appear to be working upon running it, it appears to
>> just immediately exit and throw me back into a command prompt.
>>
>> In comparison if I run the following:
>>
>> ra -S cisco://192.xx.xx.xx:9995
>>
>> It appears to stay running and sits on a new line with a flashing cursor
>> waiting to print something possibly?  But nothing is printed to stdout.
>>
>> Thank you,
>>
>> -Drew
>>
>> On Tue, Oct 24, 2017 at 1:51 PM, Carter Bullard <carter at qosient.com>
>> wrote:
>>
>>> Not sure what version you are using, but you should be able to read
>>> IPFIX data from an ra* program using the same strategy as radium.
>>> Does this work ???
>>>
>>>    ra -S ipfix://192.xx.xx.xx:9995
>>>
>>> Carter
>>>
>>>
>>> On Oct 24, 2017, at 1:25 PM, Drew Dixon <dwdixon at umich.edu> wrote:
>>>
>>> For us near real-time processing isn't really an issue- so at least for
>>> now I'll be aiming to leverage the second approach you've described which
>>> also sounds to be the most common with the scenario we are in at the
>>> moment.  I've been reading/studying the radium/rastream/racluster
>>> documentation/man pages etc. but still have questions related to
>>> configuration where there are gaps.
>>>
>>> Being that this approach will be collecting IPFIX data directed to my
>>> server which will be running radium|rastream|racluster etc.  I'm hoping
>>> someone can help me better understand/confirm the best configuration
>>> options combination available via radium in order to do so?  I may be
>>> overthinking/over-complicating this a bit but my main confusion is
>>> surrounding how to tell radium to listen for that IPFIX data properly.  I'm
>>> getting incoming data on udp/9995 which is confirmed via tcpdump...
>>>
>>> I'm currently telling radium to listen for what the docs refer to as
>>> Cisco Netflow (cmdline -C) but I am using radium.conf and the current
>>> settings I'm using in the config file are as follows:
>>>
>>> RADIUM_DAEMON=yes
>>> RADIUM_DEBUG_LEVEL=0
>>> RADIUM_MAR_STATUS_INTERVAL=60
>>> RADIUM_CISCONETFLOW_PORT=192.xx.xx.xx:9995
>>> RADIUM_ACCESS_PORT=561
>>> RADIUM_BIND_IP=127.0.0.1
>>> RADIUM_OUTPUT_FILE=/flowdata/radium/radium.out
>>> RADIUM_SETUSER_ID=argus
>>> RADIUM_SETGROUP_ID=argus
>>>
>>> Invoked with.... radium -f /etc/radium.conf
>>>
>>> Does radium log somewhere other than /var/log/messages?  Is it required
>>> to manually specify the ability to generate debug logs when compiling for
>>> that to be available? I'm not seeing anything mentioning that specifically
>>> via ./configure --help for argus-clients.
>>>
>>> I'm trying to test that radium is processing the data I've confirmed is
>>> coming via udp/9995 using rastream like:
>>>
>>> rastream -S 127.0.0.01:561 <http://127.0.0.1:561/> -w -
>>>
>>> When I invoke rastream in this fashion (as root from a root shell)
>>> should I not see the converted argus formatted data being spewed out to
>>> stdout in unidirectional argus format?
>>>
>>> I do see the log entries from radium confirming the connection from
>>> rastream, does this look normal?:
>>>
>>> Oct 24 12:56:16  radium: radium[14196]: 12:56:16.271560 connect from
>>> localhost[127.0.0.1]
>>> Oct 24 12:56:16  radium[14196]: 12:56:16.271560 connect from
>>> localhost[127.0.0.1]
>>> Oct 24 12:56:30  radium: radium[14196]: 12:56:30.781854
>>> ArgusCheckClientMessage: client localhost[127.0.0.1] sent DONE
>>> Oct 24 12:56:30  radium[14196]: 12:56:30.781854 ArgusCheckClientMessage:
>>> client localhost[127.0.0.1] sent DONE
>>>
>>> However, I'm not seeing anything being printed out to stdout though,
>>> also the output file I've configured via radium doesn't seem to be doing
>>> much (as far as I can tell at least), it's incrementing in size but not
>>> nearly to the degree I would expect but then again it does appear to be a
>>> DBase 3 data file which appears to be a database that is showing a lot of
>>> records already according to the output of the file command:
>>>
>>> $ file /flowdata/radium/radium.out
>>> /flowdata/radium/radium.out: DBase 3 data file with memo(s) (2097152
>>> records)
>>>
>>> Should I be using another instance of radium to test the incoming data
>>> is being processed rather than trying to use rastream for that?  Does this
>>> make sense and/or is that doable?
>>>
>>> Still very much learning all these tools- help is greatly appreciated,
>>> many thanks in advance.
>>>
>>> -Drew
>>>
>>> ---------- Forwarded message ----------
>>> From: Carter Bullard <carter at qosient.com>
>>> Date: Tue, Oct 17, 2017 at 10:10 AM
>>> Subject: Re: [ARGUS] Argus & IPFIX?
>>> To: Drew Dixon <dwdixon at umich.edu>
>>>
>>>
>>> Hey Drew,
>>>
>>> The open source clients have all the basic functionality to build a
>>> system that can handle complex asymmetric monitoring problems, but they not
>>> designed specifically to do this, nor are they designed to go very fast...
>>>
>>> If you have any open source argus questions, please send them to the
>>> public mailing list.
>>>
>>> If you want to merge asymmetric flows from multiple sources, the best
>>> approach is going to be dependent on how quickly you want to know about
>>> issues on the wire.  If time is important, you will want to use radium() to
>>> collect real-time streaming data, and then use programs like rabins() to
>>> aggregate the uni-directional streams into bi-directional streams, and keep
>>> the processing pipeline going.  There are time issues to work out, but the
>>> concepts in radium and rabins are what are needed to provide near-realtime
>>> multi-source aggregation.
>>>
>>> If near real-time processing is not an issue, then having radium collect
>>> the IPFIX data, covert it to argus data and use a program like rastream()
>>> to break the stream up into files, and then post process the files with
>>> racluster() to merge the records to form the bi-directional flow data.
>>> That is the most common way of doing what you’ve described.  The open
>>> source racluster() is a bit of a memory hog, so you’ll want give your
>>> system a bit of RAM, and then tune the process to handle the number of
>>> flows that you are processing.  So far in our experience, 100G doesn’t
>>> significantly increase the numbers of simultaneous flows, but rather
>>> supports bigger elephants co-existing with the traditional mix of data
>>> (research university networks).  You may find that in your specific
>>> network, that the flow data processing requirements aren’t much different
>>> than 10G, but …, then again you may be more like a wireless cellular
>>> network, where 100G really means 10x the flows seen at 10G … just all
>>> depends on your network design and customer base.
>>>
>>> Good luck, and keep us posted ….
>>> Carter
>>>
>>> [image: QoSient]        <http://qosient.com/>
>>> Carter Bullard  <carter at qosient.com>• CTO
>>> 150 E 57th Street, Suite 12D
>>> <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York+10022&entry=gmail&source=g>
>>> New York, New York 10022
>>> <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York+10022&entry=gmail&source=g>
>>> -2795
>>> Phone +1.212.588.9133 • Mobile +1.917.497.9494
>>>
>>>
>>> On Mon, Oct 16, 2017 at 1:52 PM, Carter Bullard <carter at qosient.com>
>>> wrote:
>>>
>>>> Hey Drew,
>>>> Argus should be able to read most/any IPFIX TCP/UDP data source, at
>>>> least that is the goal.  To that end, if you have some IPFIX data that the
>>>> ra* programs can’t read, I’ll spend some time making it work.  So if your
>>>> using Juniper, have it export UDP IPFIX, and we should be able to read
>>>> them, as the router advertises the templates in a reasonable timeframe, as
>>>> we need to see the templates before we can decode the records (really
>>>> terrible design flaw).
>>>>
>>>> We, of course recommend that you generate your own flow records rather
>>>> than read from integrated IPFIX, especially if you’re network is going
>>>> particularly fast.  QoSient has 1g, 10g, 40g and 100g argus sensor
>>>> appliances for sale, so if you’re looking to do the do for real, think
>>>> about generating your own data.
>>>>
>>>> Hope all is most excellent,
>>>> Carter
>>>>
>>>> [image: QoSient] <http://qosient.com/>
>>>> Carter Bullard <carter at qosient.com> • CTO
>>>> 150 E 57th Street, Suite 12D
>>>> <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York+10022&entry=gmail&source=g>
>>>> New York, New York 10022
>>>> <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York+10022&entry=gmail&source=g>
>>>> -2795
>>>> Phone +1.212.588.9133 • Mobile +1.917.497.9494
>>>>
>>>>
>>>>
>>>> On Oct 16, 2017, at 11:18 AM, Drew Dixon <dwdixon at umich.edu> wrote:
>>>>
>>>> Hello,
>>>>
>>>> I'm wondering what the current status of Argus' support of reading
>>>> IPFIX and if there might be any relevant information/updates on that front
>>>> which someone could share?
>>>>
>>>> I did some quick searching online and see mention of IPFIX in relation
>>>> to Argus but nothing really stating that it's officially supported at this
>>>> time etc.
>>>>
>>>> Thank you!
>>>>
>>>> -Drew
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20171024/4daa9f99/attachment.html>

More information about the argus mailing list