rasplit and netflow question
Carter Bullard via Argus-info
argus-info at lists.andrew.cmu.edu
Thu Sep 22 17:14:27 EDT 2016
Hey Mike,
This makes sense ... With netflow v9 and ipfix, there is no defined format for a specific record. The record format is declared in a flow template record, which is sent out periodically, say every 60 seconds or so.
The ra* programs have to wait until the templates arrive before it can decode the buffers. Nothing can be done about that unless the switch can be configured to send templates more frequently !!!
OK, sounds like you're living the netflow v9 dream !!!
Hope all other things are most excellent,
Carter
Sep 22, 2016, at 4:59 PM, Michael Stone via Argus-info <argus-info at lists.andrew.cmu.edu> wrote:
>
>> On Thu, Sep 22, 2016 at 11:49:41AM -0400, I wrote:
>>> On Thu, Sep 22, 2016 at 11:08:49AM -0400, Carter Bullard wrote:
>>> Hmmmmmmm,
>>> rasplit is a very simple program … it maybe that the Netflow records are not being parsed correctly. Depending on how its configured, you can generate poorly formed records from Cisco equipment. If you don’t mind, could you set the -D4 to -D9, and then send the output that indicates the result of the ArgusParseCiscoRecordV9Data routine, which is the most important information.
>>
>> There is no ArgusParseCiscoRecordV9Data
>>
>
> This didn't make much sense to me, because as I said ealier, ra works and rasplit based on file size works, so it must be parsing the netflow. So I ran it for longer, ended up with a very large debug output, and got:
>
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937544 ArgusReadCiscoDatagramSocket (0x7f0d8c22e010) read 1212 bytes, capacity 1212
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937555 ArgusReadCiscoDatagramSocket (0x7f0d8c29f010, 0x7f0d8c22e010) read record header
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937567 ArgusCalloc (1, 88) returning 0x214e290
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937578 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c0028, 1188) tHdr template id 272 len 21
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937589 ArgusCalloc (1, 76) returning 0x214e2f0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937599 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c0080, 1100) tHdr template id 273 len 18
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937609 ArgusCalloc (1, 76) returning 0x214e350
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937618 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c00cc, 1024) tHdr template id 274 len 18
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937628 ArgusCalloc (1, 76) returning 0x214e3b0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937638 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c0118, 948) tHdr template id 275 len 18
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937648 ArgusCalloc (1, 76) returning 0x214e410
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937657 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c0164, 872) tHdr template id 276 len 18
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937667 ArgusCalloc (1, 76) returning 0x214e470
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937676 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c01b0, 796) tHdr template id 277 len 18
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937686 ArgusCalloc (1, 84) returning 0x214e4d0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937695 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c01fc, 720) tHdr template id 278 len 20
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937705 ArgusCalloc (1, 84) returning 0x214e530
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937714 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c0250, 636) tHdr template id 279 len 20
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937724 ArgusCalloc (1, 84) returning 0x214e590
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937733 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c02a4, 552) tHdr template id 280 len 20
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937743 ArgusCalloc (1, 84) returning 0x214e5f0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937752 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c02f8, 468) tHdr template id 281 len 20
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937762 ArgusCalloc (1, 96) returning 0x214e650
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937771 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c034c, 384) tHdr template id 282 len 23
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937781 ArgusCalloc (1, 96) returning 0x214e6c0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937790 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c03ac, 288) tHdr template id 283 len 23
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937800 ArgusCalloc (1, 96) returning 0x214e730
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937809 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c040c, 192) tHdr template id 284 len 23
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937819 ArgusCalloc (1, 96) returning 0x214e7a0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937837 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c046c, 96) tHdr template id 285 len 23
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937847 ArgusParseCiscoRecordV9Template (0x7f0d8c29f010, 0x7f0d8abf20a8, 0x7f0d8b0c046c, 96) returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937857 ArgusParseCiscoRecordV9 (0x7f0d8c22e010, 0x7ffc25e6a208) returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937867 ArgusHandleRecord ((nil), 0x7f0d8c3c0808) returning 0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.937877 ArgusReadCiscoDatagramSocket (0x8c22e010) returning 0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938412 ArgusAdjustGlobalTime real 1474572947.938412 global 1474572947.938412
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938444 ArgusReadCiscoDatagramSocket (0x8c22e010) starting
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938459 ArgusReadCiscoDatagramSocket (0x7f0d8c22e010) read 1408 bytes, capacity 1408
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938470 ArgusReadCiscoDatagramSocket (0x7f0d8c29f010, 0x7f0d8c22e010) read record header
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938507 ArgusAdjustGlobalTime real 1474572947.938507 global 1474572947.938507
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938557 ArgusAlignRecord () returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938569 RaProcessRecord (0x8c22e630) done
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938579 RaScheduleRecord (0x7f0d8c29f010, 0x7f0d8c22e630) scheduled
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938588 ArgusHandleRecord (0xf84b00, 0x7f0d8c3c0808) returning 44
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938598 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x7f0d8abf20a8, 0x7f0d8b0c008a, 17) new flow
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938613 ArgusAdjustGlobalTime real 1474572947.938613 global 1474572947.938613
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938631 ArgusAlignRecord () returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938641 RaProcessRecord (0x8c22e630) done
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938651 RaScheduleRecord (0x7f0d8c29f010, 0x7f0d8c22e630) scheduled
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938660 ArgusHandleRecord (0xf84b00, 0x7f0d8c3c0808) returning 44
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938669 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x7f0d8abf20a8, 0x7f0d8b0c00ec, 16) new flow
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938682 ArgusAdjustGlobalTime real 1474572947.938682 global 1474572947.938682
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938699 ArgusAlignRecord () returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938708 RaProcessRecord (0x8c22e630) done
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938717 RaScheduleRecord (0x7f0d8c29f010, 0x7f0d8c22e630) scheduled
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938727 ArgusHandleRecord (0xf84b00, 0x7f0d8c3c0808) returning 44
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938736 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x7f0d8abf20a8, 0x7f0d8b0c014e, 15) new flow
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938745 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x213b3a0, 0x7f0d8b0c0024, 14) returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938756 ArgusParseCiscoRecordV9 (0x7f0d8c22e010, 0x7ffc25e6a208) returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938765 ArgusHandleRecord ((nil), 0x7f0d8c3c0808) returning 0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938779 ArgusAdjustGlobalTime real 1474572947.938778 global 1474572947.938778
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938796 ArgusAlignRecord () returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938805 RaProcessRecord (0x8c22e630) done
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938828 RaScheduleRecord (0x7f0d8c29f010, 0x7f0d8c22e630) scheduled
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938838 ArgusHandleRecord (0xf84b00, 0x7f0d8c3c0808) returning 44
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938847 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x7f0d8abf20a8, 0x7f0d8b0c0192, 14) new flow
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938860 ArgusAdjustGlobalTime real 1474572947.938859 global 1474572947.938859
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938877 ArgusAlignRecord () returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938886 RaProcessRecord (0x8c22e630) done
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938895 RaScheduleRecord (0x7f0d8c29f010, 0x7f0d8c22e630) scheduled
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938904 ArgusHandleRecord (0xf84b00, 0x7f0d8c3c0808) returning 44
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938913 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x7f0d8abf20a8, 0x7f0d8b0c01d0, 13) new flow
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938926 ArgusAdjustGlobalTime real 1474572947.938926 global 1474572947.938926
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938942 ArgusAlignRecord () returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938952 RaProcessRecord (0x8c22e630) done
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938960 RaScheduleRecord (0x7f0d8c29f010, 0x7f0d8c22e630) scheduled
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938969 ArgusHandleRecord (0xf84b00, 0x7f0d8c3c0808) returning 44
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938978 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x7f0d8abf20a8, 0x7f0d8b0c020e, 12) new flow
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938988 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x213b3a0, 0x7f0d8b0c0150, 11) returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.938998 ArgusParseCiscoRecordV9 (0x7f0d8c22e010, 0x7ffc25e6a208) returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939007 ArgusHandleRecord ((nil), 0x7f0d8c3c0808) returning 0
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939019 ArgusAdjustGlobalTime real 1474572947.939019 global 1474572947.939019
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939036 ArgusAlignRecord () returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939045 RaProcessRecord (0x8c22e630) done
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939054 RaScheduleRecord (0x7f0d8c29f010, 0x7f0d8c22e630) scheduled
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939063 ArgusHandleRecord (0xf84b00, 0x7f0d8c3c0808) returning 44
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939073 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x7f0d8abf20a8, 0x7f0d8b0c0276, 11) new flow
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939085 ArgusAdjustGlobalTime real 1474572947.939085 global 1474572947.939085
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939102 ArgusAlignRecord () returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939111 RaProcessRecord (0x8c22e630) done
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939120 RaScheduleRecord (0x7f0d8c29f010, 0x7f0d8c22e630) scheduled
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939129 ArgusHandleRecord (0xf84b00, 0x7f0d8c3c0808) returning 44
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939138 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x7f0d8abf20a8, 0x7f0d8b0c02d8, 10) new flow
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939150 ArgusAdjustGlobalTime real 1474572947.939150 global 1474572947.939150
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939167 ArgusAlignRecord () returning (nil)
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939177 RaProcessRecord (0x8c22e630) done
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939186 RaScheduleRecord (0x7f0d8c29f010, 0x7f0d8c22e630) scheduled
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939203 ArgusHandleRecord (0xf84b00, 0x7f0d8c3c0808) returning 44
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939213 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x7f0d8abf20a8, 0x7f0d8b0c033a, 9) new flow
> rasplit[8300.00473e8c0d7f0000]: 2016-09-22 15:35:47.939223 ArgusParseCiscoRecordV9Data (0x7f0d8c29f010, 0x7f0d8c22e010, 0x213b3a0, 0x7f0d8b0c0210, 8) returning (nil)
>
> and so forth.
> Mike Stone
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20160922/28ec6210/attachment.html>
More information about the argus
mailing list