Rasplit issue

[Carter Bullard]

Hey Vincent,
Can you share any of these flows that you are getting?
A file from your errant archive directory would be great !!!!

All the ones in your example had a single packet in them.
I suspect that they are all dst packets, so that the start and
end times for the flow are in the destination time field, rather
than the source.  

Carter


On Apr 13, 2011, at 2:21 PM, Vincent Stoffer wrote:

> Hi Carter,
> 
> Thanks for the quick response.  I did a bit more testing and followed
> your instructions.  If I run rasplit against the data in the 1969
> directory, it does indeed place the records into the correctly dated
> directories in /tmp/test (although each days logs all went into
> argus.00.log, I take it this is what you are mentioning about the time
> order?).  
> 
> I also built the 3.0.5.3 argus-clients and re-tested with the
> newest radium and rasplit and it is still producing the same behavior.
> If there is anything else I can do to help test or debug, let me know.
> I did notice one thing during the configure that I thought I should
> point out...the config stuck for a bit and then declared there was no
> working mktime...it then went on to finish and built OK.
> 
> This behavior certainly isn't a big deal at all, just wanted to let you know what
> I have been seeing.  I appreciate you taking a look!
> 
> Vince
> 
> 
> * Carter Bullard <carter at qosient.com> [110413 09:32]:
>> Hey Vincent,
>> If rasplit() is writing to the wrong directory, but the timestamps are correct,
>> then there is a bug in the filename generation routine.  1969/12/31 indicates
>> that the algorithm thinks the time is near zero, or a small negative number.
>> 
>> OK, so if you have some time to experiment.
>> 
>> First, let's make sure that the version of rasplit() is the newer one.  If so,
>> 
>> If you run rasplit() against the errant data, does it do the right thing?
>> 
>>   rasplit -R /log/argus-acad/1969/12/31 -M time 1d -w /tmp/test/%Y/%m/%d/argus.%H.log
>> 
>> if the /tmp/test archive is correct, then you can write the data back into  your
>> archive.  Although, the files that the records should go in will be out of time order.
>> To correct that you will need to run rasort() against the files that were modified.
>> If you know what day, you can run:
>> 
>>   rasort -R /log/argus-acad/2011/04/12 -M replace
>> 
>> Which will correct all the daily files.
>> 
>> If the /tmp/test archive is correct, then I may have something like "reentrant code" issues,
>> since the problem won't be because of timestamps in the records.
>> If the /tmp/test archive is incorrect, then send me some of the records that
>> end up in the wrong year, and I'll use them for debugging.
>> 
>> We have argus-clients-3.0.5.3 in the developers directory that may not
>> express this behavior.  You may want to try the radium there, as there
>> are some time issues and file issues already fixed, althougth not specifically
>> this issue.
>> 
>> I'll start looking at all my archives to see if I can find a problem similar to yours.
>> 
>> Carter
>> 
>> 
>> On Apr 13, 2011, at 11:49 AM, Vincent Stoffer wrote:
>> 
>>> Hello,
>>> 
>>> I've recently upgraded to version 3.0.4.1 and I'm seeing an
>>> interesting issue that appears to be with rasplit.  I'm running the
>>> following command to split my Argus data into hourly files within daily
>>> directories:
>>> 
>>> rasplit -d -M time 1h -S localhost -w /log/argus-acad/%Y/%m/%d/argus.%H.log
>>> 
>>> Most all of the records show up in the right place, however, I'm now seeing some records show up in the file system at:
>>> 
>>> /log/argus-acad/1969/12/31/argus.16.log
>>> 
>>> Within that file are just a set of records that are right around the
>>> change of the hour...for example:
>>> 
>>> 4/13/11  05:59:59  e         tcp     xxx.xxx.xxx.xxx.57034     -> xxx.xxx.xxx.xxx.18123         1         78   RST
>>> 04/13/11 06:00:00  e         tcp     xxx.xxx.xxx.xxx.57034     -> xxx.xxx.xxx.xxx.18123         1         77   RST
>>> 04/13/11 06:59:55  e         udp     xxx.xxx.xxx.xxx.46442     -> xxx.xxx.xxx.xxx.13759         1         62   INT
>>> 04/13/11 07:00:00  e         udp     xxx.xxx.xxx.xxx.46442    <- xxx.xxx.xxx.xxx.13759         1         62   RSP
>>> 
>>> There are some records for every hour change, sometimes more and
>>> sometimes less...both TCP and UDP.  You can see that although the
>>> records are dropped into the 1969 year directory, the actual timestamps
>>> on the records appear to be correct.  I've been running this same
>>> configuration for a while without this error, the only thing that changed was the
>>> argus-server and argus-clients versions on the two respective servers
>>> and I also recently began running with the -d option instead of
>>> backgrounding the rasplit process.  Any idea what could be happening
>>> here to cause the writing of this pre-epoch directory?
>>> 
>>> Thank you,
>>> 
>>> Vince
>>> 
>>> -- 
>>> __   ___ _ __   ___ ___ 
>>> \ \ / / | '_ \ / __/ _ \   Vincent Stoffer   Network Security Administrator
>>> \ V /| | | | | (_|  __/   Reed College      Portland, Oregon
>>> \_/ |_|_| |_|\___\___|   vince at reed.edu    503-788-6695
>>> 
>> 
> 
> 
> 
> -- 
> __   ___ _ __   ___ ___ 
> \ \ / / | '_ \ / __/ _ \   Vincent Stoffer   Network Security Administrator
> \ V /| | | | | (_|  __/   Reed College      Portland, Oregon
>  \_/ |_|_| |_|\___\___|   vince at reed.edu    503-788-6695
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110414/680fe2c4/attachment.bin>