Files for dates in the past and future when starting radium collection of netflow data

Tue Jun 10 02:33:52 EDT 2014

Will do.

--Dave

-----Original Message-----
From: Carter Bullard [mailto:carter at qosient.com] 
Sent: Monday, June 09, 2014 8:10 PM
To: David Edelman
Cc: Jesse Bowling; Argus
Subject: Re: [ARGUS] Files for dates in the past and future when starting
radium collection of netflow data

Hey Dave,
OK, so found the netflow v5 time issue.  Needed to test for negative
and rollover in secs and usecs (nsecs/1000). So that is fixed in the
clients now, although I haven't had a chance to test.

If you could run the new clients that I'll upload later tonight,
against any netflow v5 stream that you may have,  that would
really be helpful !!!!!

Thanks !!!!

Carter

On Jun 4, 2014, at 10:06 PM, David Edelman <dedelman at iname.com> wrote:

> Carter,
> 
> I do a land office business in Netflow V5 and I have seen instances of
dates
> in the future. The dates in the past might also be a problem but I would
> have a hard time recognizing it. I added some code to something that I do
> with RRD to recognize the time in the future problem and I fired up tshark
> to capture all of the traffic going to the ports that receive Netflow
into
> 1 minute capture file roll off after 10 minutes. When I see the problem, I
> wait two minutes then save all of the available capture files and do
nothing
> with them :(
> 
> Feeling a bit guilty, I went through my now modest collection of capture
> files, feeding each in turn to Argus 3.0.7.6 and then to rasort by date
'til
> I found the smallest capture file that causes the problem. 
> 
> All of the records in this file were collected in the same minute and all
> should be from 2014-05-05 01:41  but some are from 2014-06-23 18:44 
> 
> --Dave
> 
> 
> 
> -----Original Message-----
> From: argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu
> [mailto:argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu] On
> Behalf Of Carter Bullard
> Sent: Wednesday, June 04, 2014 1:22 PM
> To: Jesse Bowling
> Cc: Argus
> Subject: Re: [ARGUS] Files for dates in the past and future when starting
> radium collection of netflow data
> 
> Hey Jesse,
> I don't have any netflow v9 here, so I can't test the fixes I put in
place.
> Can I make a copy of the clients available to you for testing ????
> 
> Carter
> 
> On Jun 4, 2014, at 10:48 AM, Carter Bullard <carter at qosient.com> wrote:
> 
>> Hey Jesse,
>> So wireshark thinks these are netflow v9 records.  Does this jive with
> your understanding ???
>> The v9 support is less mature than the v5 support, so let me go through
> this and see what is up.
>> 
>> Carter
>> 
>> On Jun 4, 2014, at 10:12 AM, Jesse Bowling <jessebowling at gmail.com>
wrote:
>> 
>>> Hi Carter,
>>> 
>>> This is:
>>> 
>>> Red Hat Enterprise Linux Server release 6.5 (Santiago)
>>> Linux test 2.6.32-431.17.1.el6.x86_64 #1 SMP Fri Apr 11 17:27:00 EDT
2014
> x86_64 x86_64 x86_64 GNU/Linux
>>> 
>>> It is a VM, if that makes a difference...
>>> 
>>> Thanks!
>>> 
>>> Cheers,
>>> 
>>> Jesse
>>> On Jun 4, 2014, at 9:59 AM, Carter Bullard <carter at qosient.com> wrote:
>>> 
>>>> Hey Jesse,
>>>> Thanks for making the change to the " new " way of doing things.
>>>> These records are all screwed up, so there maybe a radium bug.
>>>> Let me work on this....
>>>> 
>>>> Just a few things.  What platform is your radium running on??  32 or 64
> bit???
>>>> 
>>>> Carter
>>>> 
>>>> On Jun 4, 2014, at 9:12 AM, Jesse Bowling <jessebowling at gmail.com>
> wrote:
>>>> 
>>>>> Hello Carter,
>>>>> 
>>>>> I uploaded one file to you yesterday based on my previous config.
Today
> I tried changing the configuration of radium to comment out
> "RADIUM_CISCONETFLOW_PORT" and only use "
RADIUM_ARGUS_SERVER=cisco://...".
> This seemed to make no difference in what I'm seeing, as I still get the
> wide spread of records.
>>>>> 
>>>>> I'm using argus-clients Version 3.0.7.31 from Jun 2nd.
>>>>> 
>>>>> I'm uploading a new tcpdump + flow file archive momentarily...
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Jesse
>>>>> 
>>>>> On Jun 3, 2014, at 11:41 AM, Carter Bullard <carter at qosient.com>
wrote:
>>>>> 
>>>>>> Hey Jesse,
>>>>>> Although, what you're doing is fine, you are using an obsoleted
method
> for getting netflow records.
>>>>>> 
>>>>>> You should use:
>>>>>> #RADIUM_ARGUS_SERVER=cisco://192.168.0.4:9699
>>>>>> 
>>>>>> where the address is the address of the data source.
>>>>>> 
>>>>>> More than likely, the date problems are coming from the netflow
> records
>>>>>> themselves, as we just do what the netflow records say is the time. 
>>>>>> You could write a copy of the stream to a file, and we can check if
>>>>>> radium is generating good timestamps, but rasplit() is messing up.
>>>>>> 
>>>>>> If you could upload the packet file to ftp at ftp.qosient.com:incoming
> and I'll check it out.
>>>>>> 
>>>>>> Carter
>>>>>> 
>>>>>> 
>>>>>> On Jun 3, 2014, at 9:59 AM, Jesse Bowling <jessebowling at gmail.com>
> wrote:
>>>>>> 
>>>>>>> Hello,
>>>>>>> 
>>>>>>> I have a radium client listening on port 9995 collecting netflow
> records from a remote source (v5). When I attach an rasplit client
> configured to split on 5 minute files, I get files for dates ranging from
> April 26th to July 15th (from a test run this morning).
>>>>>>> 
>>>>>>> I'm curious what the issue might be, any help?
>>>>>>> 
>>>>>>> $ egrep -v '^#|^[ \t]*$' /etc/radium.conf 
>>>>>>> RADIUM_DAEMON="yes"
>>>>>>> RADIUM_MONITOR_ID=<snip>
>>>>>>> RADIUM_MAR_STATUS_INTERVAL=60
>>>>>>> RADIUM_CISCONETFLOW_PORT=9995
>>>>>>> RADIUM_ACCESS_PORT=561
>>>>>>> RADIUM_BIND_IP=127.0.0.1
>>>>>>> 
>>>>>>> Started with:
>>>>>>> 
>>>>>>> radium -f /etc/radium.conf
>>>>>>> /usr/local/bin/rasplit -M time 5m -w
> /srv/argus/%Y/%m/%d/argus.%Y.%m.%d.%H.%M.%S -d -S 127.0.0.1
>>>>>>> 
>>>>>>> I also made a tcpdump of the startup process, Carter, in case that
> would be useful. Please let me know where to upload it if it would be
handy.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> 
>>>>>>> Jesse
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
> 
> <port3003capture_80863_20140505014127.pcap>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6283 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140610/8d4e4d1c/attachment.bin>