IPFIX support timeline.

[Carter Bullard via Argus-info]

Seems to me that you’ve had good luck, its a bug in rabins.1.
Can you share the file that kills rabins ??
Carter

> On Feb 4, 2016, at 9:45 PM, Richard Rothwell <Richard.Rothwell at aarnet.edu.au> wrote:
> 
> Hi Carter,
> 
> I have followed up on your suggestions. No luck. And the problem is broader than IPFIX handling.
> 
> Its seems radium can handle the net flow 9 records I am throwing at it.
> No problems there. The argus records output file produced by the –w option has sensible contents.
> 
> FYI I am currently using nfreplay to convert a collection of IPFIX records in files, to a NetFlow 9 network stream and sending that to radium.
> This produces a 1.6Gig Argus records file.
> 
> However rabins falls over whether it is taking records directly from radium or indirectly  via the Argus records file produced by radium.
> Adjusting the –B option to 300s causes rabins to fall over, but without producing any output at all.
> 
> The commands I am using are:
> 
> sudo /usr/local/sbin/radium -S cisco://any:9995 -d -P 562 
> With
> sudo /usr/local/bin/rabins -S localhost:562  -M time 10s -B 10s -w '/mnt/hgfs/centos_shared/rabins_radium.out’
> 
> OR 
> 
> sudo /usr/local/sbin/radium -S cisco://any:9995 -d -P 562 -w '/mnt/hgfs/centos_shad/radium_100_10s.out'
> With
> sudo /usr/local/bin/rabins -r '/mnt/hgfs/centos_shared/radium_100_10s.out' -M time 100s -B 100s –w '/mnt/hgfs/centos_shared/rabins_infile_100_100s_100s.out’
> 
> Etc
> 
> Regards
> 
> 
> 
> From: Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>>
> Date: Friday, 5 February 2016 at 7:39 AM
> To: Site License <Richard.Rothwell at aarnet.edu.au <mailto:Richard.Rothwell at aarnet.edu.au>>
> Cc: Argus <argus-info at lists.andrew.cmu.edu <mailto:argus-info at lists.andrew.cmu.edu>>
> Subject: Re: [ARGUS] IPFIX support timeline.
> 
> Hey Richard,
> We have preliminary support in argus-clients now for IPFIX UDP and TCP.  That needs debugging and additional support as new IEs are used.  It would be reasonable to read the IPFIX data with radium, and have rabins connect to radium to get the converted data.  That way we can figure out if any bugs are in IPFIX conversion or in record processing later on.
> 
> rabins.1 has some very specific issues with flow data coming way out of time order.  We’re going to report on time period t1-t2, and if IPFIX sends data late, rabins throws it away … could be the memory leak relates to data out of bounds ???  If so, you need to add a bit of buffering using the -B option, so that rabins doesn’t flush out a time bin, when more IPFIX data is coming.    With some implementations, you may need a “-B 300s” to make sure the data is ok.  But if you can get some guarantees from IPFIX, then the -B can be shorter.
> 
> If you have a bug report for rabins, please send it to the list.  Try using radium to convert IPFIX to argus format, check to see how out of order the flow records are, then adjust using a ‘-B delay’ option to give the IPFIX data time to show up, and then lets see if you have blow ups or memory leaks ????
> 
> Gloriad.org <http://gloriad.org/>, an NSF IRE service provider, has a great argus -> ELK system they have said they will share.  Not sure the status of that.
> 
> Carter
> 
>> On Feb 4, 2016, at 12:20 AM, Richard Rothwell via Argus-info <argus-info at lists.andrew.cmu.edu <mailto:argus-info at lists.andrew.cmu.edu>> wrote:
>> 
>> Hi list,
>> 
>> I am investigating all of the bits need to get network monitoring up and running for AARNET.
>> Front-end most likely would involve the ELK stack in some way with Argus providing the probes.
>> 
>> However we are interested in getting our data from the routers rather than network interfaces.
>> But we have settled on IPFIX. Feeding IPFIX flows into the Argus rabins client seems to work, sort of.
>> 
>> There are 2 issues I need to address.
>> When will proper IPFIX support be available?
>> What are the limitations of feeding IPFIX flows into the front end of rabins when it expects NetFlow 9. (I’m just the programmer not the network expert.)
>> Feeding IPFIX data into rabins causes it to blow up pretty quick with a major memory leak. I have studied this with heaptracker, but no definite conclusion yet.
>> Regards from Richard
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20160204/bb3bf0f3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20160204/bb3bf0f3/attachment.bin>