Files for dates in the past and future when starting radium collection of netflow data

Wed Jun 4 22:06:12 EDT 2014

Carter,

I do a land office business in Netflow V5 and I have seen instances of dates
in the future. The dates in the past might also be a problem but I would
have a hard time recognizing it. I added some code to something that I do
with RRD to recognize the time in the future problem and I fired up tshark
to capture all of the traffic going to the ports that receive Netflow  into
1 minute capture file roll off after 10 minutes. When I see the problem, I
wait two minutes then save all of the available capture files and do nothing
with them :(

Feeling a bit guilty, I went through my now modest collection of capture
files, feeding each in turn to Argus 3.0.7.6 and then to rasort by date 'til
I found the smallest capture file that causes the problem. 

All of the records in this file were collected in the same minute and all
should be from 2014-05-05 01:41  but some are from 2014-06-23 18:44 

--Dave

-----Original Message-----
From: argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu
[mailto:argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu] On
Behalf Of Carter Bullard
Sent: Wednesday, June 04, 2014 1:22 PM
To: Jesse Bowling
Cc: Argus
Subject: Re: [ARGUS] Files for dates in the past and future when starting
radium collection of netflow data

Hey Jesse,
I don't have any netflow v9 here, so I can't test the fixes I put in place.
Can I make a copy of the clients available to you for testing ????

Carter

On Jun 4, 2014, at 10:48 AM, Carter Bullard <carter at qosient.com> wrote:

> Hey Jesse,
> So wireshark thinks these are netflow v9 records.  Does this jive with
your understanding ???
> The v9 support is less mature than the v5 support, so let me go through
this and see what is up.
> 
> Carter
> 
> On Jun 4, 2014, at 10:12 AM, Jesse Bowling <jessebowling at gmail.com> wrote:
> 
>> Hi Carter,
>> 
>> This is:
>> 
>> Red Hat Enterprise Linux Server release 6.5 (Santiago)
>> Linux test 2.6.32-431.17.1.el6.x86_64 #1 SMP Fri Apr 11 17:27:00 EDT 2014
x86_64 x86_64 x86_64 GNU/Linux
>> 
>> It is a VM, if that makes a difference...
>> 
>> Thanks!
>> 
>> Cheers,
>> 
>> Jesse
>> On Jun 4, 2014, at 9:59 AM, Carter Bullard <carter at qosient.com> wrote:
>> 
>>> Hey Jesse,
>>> Thanks for making the change to the " new " way of doing things.
>>> These records are all screwed up, so there maybe a radium bug.
>>> Let me work on this....
>>> 
>>> Just a few things.  What platform is your radium running on??  32 or 64
bit???
>>> 
>>> Carter
>>> 
>>> On Jun 4, 2014, at 9:12 AM, Jesse Bowling <jessebowling at gmail.com>
wrote:
>>> 
>>>> Hello Carter,
>>>> 
>>>> I uploaded one file to you yesterday based on my previous config. Today
I tried changing the configuration of radium to comment out
"RADIUM_CISCONETFLOW_PORT" and only use " RADIUM_ARGUS_SERVER=cisco://...".
This seemed to make no difference in what I'm seeing, as I still get the
wide spread of records.
>>>> 
>>>> I'm using argus-clients Version 3.0.7.31 from Jun 2nd.
>>>> 
>>>> I'm uploading a new tcpdump + flow file archive momentarily...
>>>> 
>>>> Cheers,
>>>> 
>>>> Jesse
>>>> 
>>>> On Jun 3, 2014, at 11:41 AM, Carter Bullard <carter at qosient.com> wrote:
>>>> 
>>>>> Hey Jesse,
>>>>> Although, what you're doing is fine, you are using an obsoleted method
for getting netflow records.
>>>>> 
>>>>> You should use:
>>>>> #RADIUM_ARGUS_SERVER=cisco://192.168.0.4:9699
>>>>> 
>>>>> where the address is the address of the data source.
>>>>> 
>>>>> More than likely, the date problems are coming from the netflow
records
>>>>> themselves, as we just do what the netflow records say is the time. 
>>>>> You could write a copy of the stream to a file, and we can check if
>>>>> radium is generating good timestamps, but rasplit() is messing up.
>>>>> 
>>>>> If you could upload the packet file to ftp at ftp.qosient.com:incoming
and I'll check it out.
>>>>> 
>>>>> Carter
>>>>> 
>>>>> 
>>>>> On Jun 3, 2014, at 9:59 AM, Jesse Bowling <jessebowling at gmail.com>
wrote:
>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> I have a radium client listening on port 9995 collecting netflow
records from a remote source (v5). When I attach an rasplit client
configured to split on 5 minute files, I get files for dates ranging from
April 26th to July 15th (from a test run this morning).
>>>>>> 
>>>>>> I'm curious what the issue might be, any help?
>>>>>> 
>>>>>> $ egrep -v '^#|^[ \t]*$' /etc/radium.conf 
>>>>>> RADIUM_DAEMON="yes"
>>>>>> RADIUM_MONITOR_ID=<snip>
>>>>>> RADIUM_MAR_STATUS_INTERVAL=60
>>>>>> RADIUM_CISCONETFLOW_PORT=9995
>>>>>> RADIUM_ACCESS_PORT=561
>>>>>> RADIUM_BIND_IP=127.0.0.1
>>>>>> 
>>>>>> Started with:
>>>>>> 
>>>>>> radium -f /etc/radium.conf
>>>>>> /usr/local/bin/rasplit -M time 5m -w
/srv/argus/%Y/%m/%d/argus.%Y.%m.%d.%H.%M.%S -d -S 127.0.0.1
>>>>>> 
>>>>>> I also made a tcpdump of the startup process, Carter, in case that
would be useful. Please let me know where to upload it if it would be handy.
>>>>>> 
>>>>>> Cheers,
>>>>>> 
>>>>>> Jesse
>>>>> 
>>>> 
>>> 
>> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: port3003capture_80863_20140505014127.pcap
Type: application/octet-stream
Size: 17104 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140604/0730f3a5/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6283 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140604/0730f3a5/attachment.bin>