Argus & IPFIX?

Tue Oct 24 23:02:58 EDT 2017

The timeout is a 1 second timeout routine that is called every second, regardless of what is going on …
Try running ra.1 instead of radium to see if you can’t get the clients to read your packets, and you can up the -D option higher (9) to see if you’re getting any input.
If you can use tcpdump/wireshark to capture the packets you expect on the interface that ra* is opening, that would help in diagnosis.  Reacmember, you’re not capturing from the remote host, you’re opening a specific interface on your sensor, where IPFIX packets are being sent.

Carter

> On Oct 24, 2017, at 5:47 PM, Drew Dixon <dwdixon at umich.edu> wrote:
> 
> Thanks Carter, I re-compiled as you directed and now have debug functionality running argus-clients-3.0.8.2...still seeing IPFIX data coming in via 192.xx.xx.xx:9995 (UDP) in a steady stream via tcpdump.  
> 
> I ran radium again with debug set to 6 and using the config file parameters I mentioned earlier in the thread.  I keep getting repeated "ArgusClientTimeout()" after radium binds to the 192.xx.xx.xx:9995 network interface on the same server to listen for/collect the IPFIX data.  I also tried running ra as you suggested, with ipfix:// ra just exits immediately, with cisco:// ra also repeatedly gives me "ArgusClientTimeout()" after binding to the server NIC's IP on udp/9995...all debug log results are below (level 6).
> 
> Not certain but guessing this means it's not receiving/recognizing any of the incoming IPFIX data or something else?
> 
> 
> ***********************************************************************
> Oct 24 17:20:54 argus1 systemd: Started radium Service.
> Oct 24 17:20:54 argus1 systemd: Starting radium Service...
> Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582224 ArgusCalloc (1, 560) returning 0x3113e80
> Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582308 ArgusCalloc (1, 112) returning 0x31137c0
> Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582319 ArgusCalloc (1, 80) returning 0x3113840
> Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582327 ArgusNewQueue () returning 0x3113840
> Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582336 ArgusCalloc (1, 56) returning 0x31138a0
> Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582349 ArgusCalloc (65536, 8) returning 0xe0bd8010
> Oct 24 17:20:54 argus1 radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582357 ArgusNewHashTable (65536) returning 0x31138a0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582566 setArgusMarReportInterval(15) returning
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582854 ArgusCalloc (1, 461728) returning 0xdfdf5010
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582894 ArgusAddHostList (0xe0c59010, 192.xx.xx.xx:9995, 16, 17) returning 1
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582929 RadiumParseResourceFile: ArgusBindAddr "127.0.0.1"
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582959 ArgusCalloc (1, 144) returning 0x3114390
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.582990 ArgusNewList () returning 0x3114390
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.583032 ArgusCalloc (1, 296) returning 0x3114460
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.583060 ArgusPushFrontList (0x3114390, 0x3114460, 1) returning 0x3114970
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.583538 RadiumParseResourceFile (/etc/radium.conf) returning 0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.583589 ArgusEstablishListen(0xe0c59010, 561, 127.0.0.1) binding: 127.0.0.1:561 <http://127.0.0.1:561/>
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584032 ArgusEstablishListen(0xe0c59010, 561, 127.0.0.1) returning 3
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584073 ArgusCalloc (1, 392) returning 0x31140c0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584102 ArgusCalloc (1, 80) returning 0x3114b40
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584129 ArgusNewQueue () returning 0x3114b40
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584153 ArgusNewOutput() returning retn 0x31140c0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584178 ArgusCalloc (1, 144) returning 0x3114250
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584202 ArgusNewList () returning 0x3114250
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584227 ArgusCalloc (1, 128) returning 0x3115da0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584251 ArgusGenerateInitialMar() returning
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584282 ArgusCalloc (1, 168) returning 0x3115e30
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584355 ArgusCalloc (1, 65792) returning 0x3115ee0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584388 ArgusCalloc (1, 144) returning 0x3125ff0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584412 ArgusNewList () returning 0x3125ff0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584436 ArgusNewSocket (4) returning 0x3115ee0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584479 ArgusPushBackList (0x3114390, 0x3114460, 1) returning 1
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.584965 ArgusInitOutput() done
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.585011 main: reading files completed
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.585040 ArgusCalloc (1, 80) returning 0x31262d0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.585065 ArgusNewQueue () returning 0x31262d0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.585121 ArgusFree (0x31262d0)
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.585149 ArgusDeleteQueue (0x31262d0) returning
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:54.585174 ArgusReadStream(0x7fc7e0c59010) starting
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.585224 ArgusConnectRemote(0x7fc7dfdf5010) starting
> Oct 24 17:20:54 argus1 radium: radium[41360]: 17:20:54.585349 Binding 192.xx.xx.xx:9995 Expecting Netflow records
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.585433 receiving
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.585464 ArgusGetServerSocket (0x7fc7dfdf5010) returning 5
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.585513 ArgusCalloc (1, 4194304) returning 0xce7de010
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.586086 ArgusCalloc (1, 262144) returning 0xce79d010
> Oct 24 17:20:54 argus1 radium[41360]: 17:20:54.585349 Binding 192.xx.xx.xx:9995 Expecting Netflow records
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.602162 ArgusInitAddrtoname (0x7fc7e0c59010, 0x0, 0x0)
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.602202 ArgusParseInit(0x7fc7e0c59010 0x7fc7dfdf5010
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.602223 ArgusReadConnection(0xdfdf5010, 2) reading cisco wire format
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.602245 ArgusReadConnection(0xdfdf5010, 2) returning 0
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.602265 ArgusConnectRemote(0x7fc7dfdf5010) connected to 192.xx.xx.xx
> Oct 24 17:20:54 argus1 radium: radium: radium[41360.00f73dcfc77f0000]: 17:20:54.602285 ArgusConnectRemote() done!
> Oct 24 17:20:55 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:55.636383 ArgusClientTimeout()
> Oct 24 17:20:56 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:56.637568 ArgusClientTimeout()
> Oct 24 17:20:57 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:57.638727 ArgusClientTimeout()
> Oct 24 17:20:58 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:58.639887 ArgusClientTimeout()
> Oct 24 17:20:59 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:20:59.641141 ArgusClientTimeout()
> Oct 24 17:21:00 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:21:00.642402 ArgusClientTimeout()
> Oct 24 17:21:01 argus1 radium: radium: radium[41360.40d7d9e0c77f0000]: 17:21:01.643565 ArgusClientTimeout()
> Oct 24 17:21:01 argus1 systemd: Stopping radium Service...
> ***********************************************************************
> 
> 
> ra using ipfix:// 
> ***********************************************************************
> # ra -D6 -S ipfix://192.xx.xx.xx:9995
> 
> ra[41491.4037f00ea37f0000]: 17:32:00.930991 ArgusCalloc (1, 461728) returning 0xed4e010
> ra[41491.4037f00ea37f0000]: 17:32:00.931065 ArgusAddHostList (0xedbf010, ipfix://192.xx.xx.xx:9995, 64, 6) returning 1
> ra[41491.4037f00ea37f0000]: 17:32:00.931091 main: reading files completed
> ra[41491.4037f00ea37f0000]: 17:32:00.931136 ArgusCalloc (1, 80) returning 0x1545840
> ra[41491.4037f00ea37f0000]: 17:32:00.931157 ArgusNewQueue () returning 0x1545840
> ra[41491.4037f00ea37f0000]: 17:32:00.931185 ArgusGetServerSocket (0x7fa30ed4e010) returning -1
> ra[41491.4037f00ea37f0000]: 17:32:00.931217 ArgusFree (0x1545840)
> ra[41491.4037f00ea37f0000]: 17:32:00.931237 ArgusDeleteQueue (0x1545840) returning
> ra[41491.4037f00ea37f0000]: 17:32:00.931269 ArgusShutDown (0)
> ra[41491.4037f00ea37f0000]: 17:32:00.931298 ArgusFree (0x7fa30ed4e010)
> ra[41491.4037f00ea37f0000]: 17:32:00.931319 ArgusFree (0x1545190)
> ra[41491.4037f00ea37f0000]: 17:32:00.931336 ArgusDeleteQueue (0x1545190) returning
> ra[41491.4037f00ea37f0000]: 17:32:00.931355 ArgusFree (0x15451f0)
> ra[41491.4037f00ea37f0000]: 17:32:00.931371 ArgusDeleteQueue (0x15451f0) returning
> ra[41491.4037f00ea37f0000]: 17:32:00.931389 RaParseComplete(caught signal 0)
> ra[41491.4037f00ea37f0000]: 17:32:00.931404 ArgusWindowClose () returning
> ra[41491.4037f00ea37f0000]: 17:32:00.931420 RaParseComplete(caught signal 0)
> ra[41491.4037f00ea37f0000]: 17:32:00.931439 RaParseComplete(caught signal 0)
> ra[41491.4037f00ea37f0000]: 17:32:00.931457 ArgusShutDown (0)
> ra[41491.4037f00ea37f0000]: 17:32:00.931789 ArgusFree (0x1545050)
> ra[41491.4037f00ea37f0000]: 17:32:00.931808 ArgusDeleteList (0x1545050, 4) returning
> ra[41491.4037f00ea37f0000]: 17:32:00.931829 ArgusFree (0x15450f0)
> ra[41491.4037f00ea37f0000]: 17:32:00.931842 ArgusDeleteList (0x15450f0, 4) returning
> ***********************************************************************
> 
> ra using cisco://
> ***********************************************************************
> # ra -D6 -S cisco://192.xx.xx.xx:9995
> ra[41498.407743631c7f0000]: 17:33:07.958512 ArgusCalloc (1, 461728) returning 0x63282010
> ra[41498.407743631c7f0000]: 17:33:07.958588 ArgusAddHostList (0x632f3010, cisco://192.xx.xx.xx:9995, 16, 17) returning 1
> ra[41498.407743631c7f0000]: 17:33:07.958612 main: reading files completed
> ra[41498.407743631c7f0000]: 17:33:07.958633 ArgusCalloc (1, 80) returning 0x18dc840
> ra[41498.407743631c7f0000]: 17:33:07.958664 ArgusNewQueue () returning 0x18dc840
> ra[41498]: 17:33:07.958737 Binding 192.xx.xx.xx:9995 Expecting Netflow records
> ra[41498.407743631c7f0000]: 17:33:07.959292 receiving
> ra[41498.407743631c7f0000]: 17:33:07.959312 ArgusGetServerSocket (0x7f1c63282010) returning 3
> ra[41498.407743631c7f0000]: 17:33:07.959354 ArgusCalloc (1, 4194304) returning 0x620ff010
> ra[41498.407743631c7f0000]: 17:33:07.959384 ArgusCalloc (1, 262144) returning 0x620be010
> ra[41498.407743631c7f0000]: 17:33:07.977130 ArgusInitAddrtoname (0x7f1c632f3010, 0x0, 0x0)
> ra[41498.407743631c7f0000]: 17:33:07.977161 ArgusParseInit(0x7f1c632f3010 0x7f1c63282010
> ra[41498.407743631c7f0000]: 17:33:07.977174 ArgusReadConnection(0x63282010, 2) reading cisco wire format
> ra[41498.407743631c7f0000]: 17:33:07.977186 ArgusReadConnection(0x63282010, 2) returning 0
> ra[41498.407743631c7f0000]: 17:33:07.977209 ArgusHandleRecord (0x7f1c63282228, 0x7f1c63414808) returning -1
> ra[41498.407743631c7f0000]: 17:33:07.977227 ArgusFree (0x18dc840)
> ra[41498.407743631c7f0000]: 17:33:07.977240 ArgusDeleteQueue (0x18dc840) returning
> ra[41498.407743631c7f0000]: 17:33:07.977253 ArgusReadStream(0x7f1c632f3010) starting
> ra[41498.407743631c7f0000]: 17:33:09.228565 ArgusClientTimeout()
> ra[41498.407743631c7f0000]: 17:33:10.229714 ArgusClientTimeout()
> ra[41498.407743631c7f0000]: 17:33:11.230838 ArgusClientTimeout()
> ***********************************************************************
> 
> Thank you,
> 
> -Drew
> 
> On Tue, Oct 24, 2017 at 3:45 PM, Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>> wrote:
> When you run the ./configure program, like many open source distros, you can create files, like .debug or .devel,  that ./configure looks for, and if they exist, it will configure the package to add specific functionality.  If there is a ./.debug file, the ./conifgure script will compile in support for printing debug information on the command line …. Try this …
> 
>    % cd /full/path/to/your/argus/clients/distribution
>    % touch .debug
>    % ./configure
>    % make clean; make
> 
> Now all your ra* programs will be able to print detail debugging information, using the “-D debuglevel” option on the command line.
> You can also turn on this type of support like this …
> 
>    % ./configure —enable-debug=yes
> 
> I prefer the .debug file, as you can set it and forget that you need the switch on the configure line.
> Please keep all email on the mailing list … Thanks...
> 
> Carter
> 
> 
> 
>> On Oct 24, 2017, at 3:33 PM, Drew Dixon <dwdixon at umich.edu <mailto:dwdixon at umich.edu>> wrote:
>> 
>> Whoops...apologies with that- I noticed it said on the downloads page that 3.0.8.1 was the current stable version and never thought I had a need to look on the main page.
>> 
>> Please forgive my ignorance but I'm not 100% sure what you mean by "with the .debug tag" I've listed out my options using ./configure --help but don't see anything related to enabling debugging.  Only thing that I can see is  --enable-FEATURE[=ARG]  include FEATURE [ARG=yes] is this what you're referring to?
>> 
>> --enable-debug=yes ?
>> 
>> Or somehow with referencing the files below?
>> 
>> /argus-clients-3.0.8.2/common/._argus_debug.h
>> /argus-clients-3.0.8.2/common/argus_debug.h
>> /argus-clients-3.0.8.2/include/._argus_debug.h
>> /argus-clients-3.0.8.2/include/argus_debug.h
>> 
>> I'm not certain on how to compile to enable debugging properly in the fashion you've described.  
>> 
>> I very much appreciate your help.
>> 
>> Sincerely,
>> 
>> -Drew
>> 
>> On Tue, Oct 24, 2017 at 3:09 PM, Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>> wrote:
>> How about the link to argus-clients-3.0.8.2 from the main page …
>> 
>> http://qosient.com/argus/ <http://qosient.com/argus/>
>> 
>> You should ./configure and compile with the .debug tag in the home directory, and then run your ra command with the -D4 or -D5 option to see if you are receiving any packets.
>> Carter
>> 
>>          <http://qosient.com/>     	 	
>> Carter Bullard  <mailto:carter at qosient.com>• CTO
>> 150 E 57th Street, Suite 12D <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York&entry=gmail&source=g>
>> New York, New York <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York&entry=gmail&source=g> 10022-2795
>> Phone +1.212.588.9133 • Mobile +1.917.497.9494
>> 
>>  
>> 
>>> On Oct 24, 2017, at 2:15 PM, Drew Dixon <dwdixon at umich.edu <mailto:dwdixon at umich.edu>> wrote:
>>> 
>>> I'm using the latest stable version (argus-clients-3.0.8) downloaded from the website (http://www.qosient.com/argus/downloads.shtml <http://www.qosient.com/argus/downloads.shtml>).
>>> 
>>> That command doesn't appear to be working upon running it, it appears to just immediately exit and throw me back into a command prompt.  
>>> 
>>> In comparison if I run the following:
>>> 
>>> ra -S cisco://192.xx.xx.xx:9995 <>
>>> 
>>>  <>
>>> It appears to stay running and sits on a new line with a flashing cursor waiting to print something possibly?  But nothing is printed to stdout. <>
>>> 
>>>  <>
>>> Thank you, <>
>>> 
>>>  <>
>>> -Drew <>
>>> 
>>> On Tue, Oct 24, 2017 at 1:51 PM, Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>> wrote:
>>> Not sure what version you are using, but you should be able to read IPFIX data from an ra* program using the same strategy as radium.
>>> Does this work ???
>>> 
>>>    ra -S ipfix://192.xx.xx.xx:9995 <>
>>> 
>>> Carter
>>> 
>>> 
>>>> On Oct 24, 2017, at 1:25 PM, Drew Dixon <dwdixon at umich.edu <mailto:dwdixon at umich.edu>> wrote:
>>>> 
>>>> For us near real-time processing isn't really an issue- so at least for now I'll be aiming to leverage the second approach you've described which also sounds to be the most common with the scenario we are in at the moment.  I've been reading/studying the radium/rastream/racluster documentation/man pages etc. but still have questions related to configuration where there are gaps.
>>>> 
>>>> Being that this approach will be collecting IPFIX data directed to my server which will be running radium|rastream|racluster etc.  I'm hoping someone can help me better understand/confirm the best configuration options combination available via radium in order to do so?  I may be overthinking/over-complicating this a bit but my main confusion is surrounding how to tell radium to listen for that IPFIX data properly.  I'm getting incoming data on udp/9995 which is confirmed via tcpdump...
>>>> 
>>>> I'm currently telling radium to listen for what the docs refer to as Cisco Netflow (cmdline -C) but I am using radium.conf and the current settings I'm using in the config file are as follows:
>>>> 
>>>> RADIUM_DAEMON=yes
>>>> RADIUM_DEBUG_LEVEL=0
>>>> RADIUM_MAR_STATUS_INTERVAL=60
>>>> RADIUM_CISCONETFLOW_PORT=192.xx.xx.xx:9995
>>>> RADIUM_ACCESS_PORT=561
>>>> RADIUM_BIND_IP=127.0.0.1
>>>> RADIUM_OUTPUT_FILE=/flowdata/radium/radium.out
>>>> RADIUM_SETUSER_ID=argus
>>>> RADIUM_SETGROUP_ID=argus
>>>> 
>>>> Invoked with.... radium -f /etc/radium.conf
>>>> 
>>>> Does radium log somewhere other than /var/log/messages?  Is it required to manually specify the ability to generate debug logs when compiling for that to be available? I'm not seeing anything mentioning that specifically via ./configure --help for argus-clients.
>>>> 
>>>> I'm trying to test that radium is processing the data I've confirmed is coming via udp/9995 using rastream like:
>>>> 
>>>> rastream -S 127.0.0.01:561 <http://127.0.0.1:561/> -w -
>>>> 
>>>> When I invoke rastream in this fashion (as root from a root shell) should I not see the converted argus formatted data being spewed out to stdout in unidirectional argus format?  
>>>> 
>>>> I do see the log entries from radium confirming the connection from rastream, does this look normal?:
>>>> 
>>>> Oct 24 12:56:16  radium: radium[14196]: 12:56:16.271560 connect from localhost[127.0.0.1]
>>>> Oct 24 12:56:16  radium[14196]: 12:56:16.271560 connect from localhost[127.0.0.1]
>>>> Oct 24 12:56:30  radium: radium[14196]: 12:56:30.781854 ArgusCheckClientMessage: client localhost[127.0.0.1] sent DONE
>>>> Oct 24 12:56:30  radium[14196]: 12:56:30.781854 ArgusCheckClientMessage: client localhost[127.0.0.1] sent DONE
>>>> 
>>>> However, I'm not seeing anything being printed out to stdout though, also the output file I've configured via radium doesn't seem to be doing much (as far as I can tell at least), it's incrementing in size but not nearly to the degree I would expect but then again it does appear to be a DBase 3 data file which appears to be a database that is showing a lot of records already according to the output of the file command:
>>>> 
>>>> $ file /flowdata/radium/radium.out
>>>> /flowdata/radium/radium.out: DBase 3 data file with memo(s) (2097152 records)
>>>> 
>>>> Should I be using another instance of radium to test the incoming data is being processed rather than trying to use rastream for that?  Does this make sense and/or is that doable?
>>>> 
>>>> Still very much learning all these tools- help is greatly appreciated, many thanks in advance.
>>>> 
>>>> -Drew
>>>> 
>>>> ---------- Forwarded message ----------
>>>> From: Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>>
>>>> Date: Tue, Oct 17, 2017 at 10:10 AM
>>>> Subject: Re: [ARGUS] Argus & IPFIX?
>>>> To: Drew Dixon <dwdixon at umich.edu <mailto:dwdixon at umich.edu>>
>>>> 
>>>> 
>>>> Hey Drew,
>>>> 
>>>> The open source clients have all the basic functionality to build a system that can handle complex asymmetric monitoring problems, but they not designed specifically to do this, nor are they designed to go very fast...
>>>> 
>>>> If you have any open source argus questions, please send them to the public mailing list.
>>>> 
>>>> If you want to merge asymmetric flows from multiple sources, the best approach is going to be dependent on how quickly you want to know about issues on the wire.  If time is important, you will want to use radium() to collect real-time streaming data, and then use programs like rabins() to aggregate the uni-directional streams into bi-directional streams, and keep the processing pipeline going.  There are time issues to work out, but the concepts in radium and rabins are what are needed to provide near-realtime multi-source aggregation.
>>>> 
>>>> If near real-time processing is not an issue, then having radium collect the IPFIX data, covert it to argus data and use a program like rastream() to break the stream up into files, and then post process the files with racluster() to merge the records to form the bi-directional flow data.  That is the most common way of doing what you’ve described.  The open source racluster() is a bit of a memory hog, so you’ll want give your system a bit of RAM, and then tune the process to handle the number of flows that you are processing.  So far in our experience, 100G doesn’t significantly increase the numbers of simultaneous flows, but rather supports bigger elephants co-existing with the traditional mix of data (research university networks).  You may find that in your specific network, that the flow data processing requirements aren’t much different than 10G, but …, then again you may be more like a wireless cellular network, where 100G really means 10x the flows seen at 10G … just all depends on your network design and customer base.
>>>> 
>>>> Good luck, and keep us posted ….
>>>> Carter
>>>> 
>>>>         <http://qosient.com/>     	 	
>>>> Carter Bullard  <mailto:carter at qosient.com>• CTO
>>>> 150 E 57th Street, Suite 12D <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York+10022&entry=gmail&source=g>
>>>> New York, New York 10022 <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York+10022&entry=gmail&source=g>-2795
>>>> Phone +1.212.588.9133 • Mobile +1.917.497.9494
>>>> 
>>>> 
>>>> On Mon, Oct 16, 2017 at 1:52 PM, Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>> wrote:
>>>> Hey Drew,
>>>> Argus should be able to read most/any IPFIX TCP/UDP data source, at least that is the goal.  To that end, if you have some IPFIX data that the ra* programs can’t read, I’ll spend some time making it work.  So if your using Juniper, have it export UDP IPFIX, and we should be able to read them, as the router advertises the templates in a reasonable timeframe, as we need to see the templates before we can decode the records (really terrible design flaw).
>>>> 
>>>> We, of course recommend that you generate your own flow records rather than read from integrated IPFIX, especially if you’re network is going particularly fast.  QoSient has 1g, 10g, 40g and 100g argus sensor appliances for sale, so if you’re looking to do the do for real, think about generating your own data.
>>>> 
>>>> Hope all is most excellent,
>>>> Carter
>>>> 
>>>>          <http://qosient.com/>     	 	
>>>> Carter Bullard  <mailto:carter at qosient.com>• CTO
>>>> 150 E 57th Street, Suite 12D <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York+10022&entry=gmail&source=g>
>>>> New York, New York 10022 <https://maps.google.com/?q=150+E+57th+Street,+Suite+12D+%0D+%0D+%0D+%0D+New+York,+New+York+10022&entry=gmail&source=g>-2795
>>>> Phone +1.212.588.9133 • Mobile +1.917.497.9494
>>>> 
>>>>  
>>>> 
>>>>> On Oct 16, 2017, at 11:18 AM, Drew Dixon <dwdixon at umich.edu <mailto:dwdixon at umich.edu>> wrote:
>>>>> 
>>>>> Hello,
>>>>> 
>>>>> I'm wondering what the current status of Argus' support of reading IPFIX and if there might be any relevant information/updates on that front which someone could share?  
>>>>> 
>>>>> I did some quick searching online and see mention of IPFIX in relation to Argus but nothing really stating that it's officially supported at this time etc.
>>>>> 
>>>>> Thank you!
>>>>> 
>>>>> -Drew
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20171024/2fddc676/attachment.html>