Rasplit issue
Carter Bullard
carter at qosient.com
Thu Apr 14 13:35:37 EDT 2011
Hey Vincent,
I just uploaded argus-clients-3.0.5.4 to the developers site that
may fix this problem. We've had an issue on the list that I fixed
today, which deals with time more consistently. It addresses
something that could cause your errant directory issue in rasplit().
If you could give this a try, I would really appreciate it !!!!!
Carter
On Apr 14, 2011, at 11:09 AM, Carter Bullard wrote:
> Hey Vincent,
> Can you share any of these flows that you are getting?
> A file from your errant archive directory would be great !!!!
>
> All the ones in your example had a single packet in them.
> I suspect that they are all dst packets, so that the start and
> end times for the flow are in the destination time field, rather
> than the source.
>
> Carter
>
>
> On Apr 13, 2011, at 2:21 PM, Vincent Stoffer wrote:
>
>> Hi Carter,
>>
>> Thanks for the quick response. I did a bit more testing and followed
>> your instructions. If I run rasplit against the data in the 1969
>> directory, it does indeed place the records into the correctly dated
>> directories in /tmp/test (although each days logs all went into
>> argus.00.log, I take it this is what you are mentioning about the time
>> order?).
>>
>> I also built the 3.0.5.3 argus-clients and re-tested with the
>> newest radium and rasplit and it is still producing the same behavior.
>> If there is anything else I can do to help test or debug, let me know.
>> I did notice one thing during the configure that I thought I should
>> point out...the config stuck for a bit and then declared there was no
>> working mktime...it then went on to finish and built OK.
>>
>> This behavior certainly isn't a big deal at all, just wanted to let you know what
>> I have been seeing. I appreciate you taking a look!
>>
>> Vince
>>
>>
>> * Carter Bullard <carter at qosient.com> [110413 09:32]:
>>> Hey Vincent,
>>> If rasplit() is writing to the wrong directory, but the timestamps are correct,
>>> then there is a bug in the filename generation routine. 1969/12/31 indicates
>>> that the algorithm thinks the time is near zero, or a small negative number.
>>>
>>> OK, so if you have some time to experiment.
>>>
>>> First, let's make sure that the version of rasplit() is the newer one. If so,
>>>
>>> If you run rasplit() against the errant data, does it do the right thing?
>>>
>>> rasplit -R /log/argus-acad/1969/12/31 -M time 1d -w /tmp/test/%Y/%m/%d/argus.%H.log
>>>
>>> if the /tmp/test archive is correct, then you can write the data back into your
>>> archive. Although, the files that the records should go in will be out of time order.
>>> To correct that you will need to run rasort() against the files that were modified.
>>> If you know what day, you can run:
>>>
>>> rasort -R /log/argus-acad/2011/04/12 -M replace
>>>
>>> Which will correct all the daily files.
>>>
>>> If the /tmp/test archive is correct, then I may have something like "reentrant code" issues,
>>> since the problem won't be because of timestamps in the records.
>>> If the /tmp/test archive is incorrect, then send me some of the records that
>>> end up in the wrong year, and I'll use them for debugging.
>>>
>>> We have argus-clients-3.0.5.3 in the developers directory that may not
>>> express this behavior. You may want to try the radium there, as there
>>> are some time issues and file issues already fixed, althougth not specifically
>>> this issue.
>>>
>>> I'll start looking at all my archives to see if I can find a problem similar to yours.
>>>
>>> Carter
>>>
>>>
>>> On Apr 13, 2011, at 11:49 AM, Vincent Stoffer wrote:
>>>
>>>> Hello,
>>>>
>>>> I've recently upgraded to version 3.0.4.1 and I'm seeing an
>>>> interesting issue that appears to be with rasplit. I'm running the
>>>> following command to split my Argus data into hourly files within daily
>>>> directories:
>>>>
>>>> rasplit -d -M time 1h -S localhost -w /log/argus-acad/%Y/%m/%d/argus.%H.log
>>>>
>>>> Most all of the records show up in the right place, however, I'm now seeing some records show up in the file system at:
>>>>
>>>> /log/argus-acad/1969/12/31/argus.16.log
>>>>
>>>> Within that file are just a set of records that are right around the
>>>> change of the hour...for example:
>>>>
>>>> 4/13/11 05:59:59 e tcp xxx.xxx.xxx.xxx.57034 -> xxx.xxx.xxx.xxx.18123 1 78 RST
>>>> 04/13/11 06:00:00 e tcp xxx.xxx.xxx.xxx.57034 -> xxx.xxx.xxx.xxx.18123 1 77 RST
>>>> 04/13/11 06:59:55 e udp xxx.xxx.xxx.xxx.46442 -> xxx.xxx.xxx.xxx.13759 1 62 INT
>>>> 04/13/11 07:00:00 e udp xxx.xxx.xxx.xxx.46442 <- xxx.xxx.xxx.xxx.13759 1 62 RSP
>>>>
>>>> There are some records for every hour change, sometimes more and
>>>> sometimes less...both TCP and UDP. You can see that although the
>>>> records are dropped into the 1969 year directory, the actual timestamps
>>>> on the records appear to be correct. I've been running this same
>>>> configuration for a while without this error, the only thing that changed was the
>>>> argus-server and argus-clients versions on the two respective servers
>>>> and I also recently began running with the -d option instead of
>>>> backgrounding the rasplit process. Any idea what could be happening
>>>> here to cause the writing of this pre-epoch directory?
>>>>
>>>> Thank you,
>>>>
>>>> Vince
>>>>
>>>> --
>>>> __ ___ _ __ ___ ___
>>>> \ \ / / | '_ \ / __/ _ \ Vincent Stoffer Network Security Administrator
>>>> \ V /| | | | | (_| __/ Reed College Portland, Oregon
>>>> \_/ |_|_| |_|\___\___| vince at reed.edu 503-788-6695
>>>>
>>>
>>
>>
>>
>> --
>> __ ___ _ __ ___ ___
>> \ \ / / | '_ \ / __/ _ \ Vincent Stoffer Network Security Administrator
>> \ V /| | | | | (_| __/ Reed College Portland, Oregon
>> \_/ |_|_| |_|\___\___| vince at reed.edu 503-788-6695
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110414/60fbd623/attachment.bin>
More information about the argus
mailing list