Rasplit issue

[Vincent Stoffer]

Hi Carter,

Thanks for the quick response.  I did a bit more testing and followed
your instructions.  If I run rasplit against the data in the 1969
directory, it does indeed place the records into the correctly dated
directories in /tmp/test (although each days logs all went into
argus.00.log, I take it this is what you are mentioning about the time
order?).  

I also built the 3.0.5.3 argus-clients and re-tested with the
newest radium and rasplit and it is still producing the same behavior.
If there is anything else I can do to help test or debug, let me know.
I did notice one thing during the configure that I thought I should
point out...the config stuck for a bit and then declared there was no
working mktime...it then went on to finish and built OK.

This behavior certainly isn't a big deal at all, just wanted to let you know what
I have been seeing.  I appreciate you taking a look!

Vince


* Carter Bullard <carter at qosient.com> [110413 09:32]:
> Hey Vincent,
> If rasplit() is writing to the wrong directory, but the timestamps are correct,
> then there is a bug in the filename generation routine.  1969/12/31 indicates
> that the algorithm thinks the time is near zero, or a small negative number.
> 
> OK, so if you have some time to experiment.
> 
> First, let's make sure that the version of rasplit() is the newer one.  If so,
> 
> If you run rasplit() against the errant data, does it do the right thing?
> 
>    rasplit -R /log/argus-acad/1969/12/31 -M time 1d -w /tmp/test/%Y/%m/%d/argus.%H.log
> 
> if the /tmp/test archive is correct, then you can write the data back into  your
> archive.  Although, the files that the records should go in will be out of time order.
> To correct that you will need to run rasort() against the files that were modified.
> If you know what day, you can run:
> 
>    rasort -R /log/argus-acad/2011/04/12 -M replace
> 
> Which will correct all the daily files.
> 
> If the /tmp/test archive is correct, then I may have something like "reentrant code" issues,
> since the problem won't be because of timestamps in the records.
> If the /tmp/test archive is incorrect, then send me some of the records that
> end up in the wrong year, and I'll use them for debugging.
> 
> We have argus-clients-3.0.5.3 in the developers directory that may not
> express this behavior.  You may want to try the radium there, as there
> are some time issues and file issues already fixed, althougth not specifically
> this issue.
> 
> I'll start looking at all my archives to see if I can find a problem similar to yours.
>  
> Carter
> 
> 
> On Apr 13, 2011, at 11:49 AM, Vincent Stoffer wrote:
> 
> > Hello,
> > 
> > I've recently upgraded to version 3.0.4.1 and I'm seeing an
> > interesting issue that appears to be with rasplit.  I'm running the
> > following command to split my Argus data into hourly files within daily
> > directories:
> > 
> > rasplit -d -M time 1h -S localhost -w /log/argus-acad/%Y/%m/%d/argus.%H.log
> > 
> > Most all of the records show up in the right place, however, I'm now seeing some records show up in the file system at:
> > 
> > /log/argus-acad/1969/12/31/argus.16.log
> > 
> > Within that file are just a set of records that are right around the
> > change of the hour...for example:
> > 
> > 4/13/11  05:59:59  e         tcp     xxx.xxx.xxx.xxx.57034     -> xxx.xxx.xxx.xxx.18123         1         78   RST
> > 04/13/11 06:00:00  e         tcp     xxx.xxx.xxx.xxx.57034     -> xxx.xxx.xxx.xxx.18123         1         77   RST
> > 04/13/11 06:59:55  e         udp     xxx.xxx.xxx.xxx.46442     -> xxx.xxx.xxx.xxx.13759         1         62   INT
> > 04/13/11 07:00:00  e         udp     xxx.xxx.xxx.xxx.46442    <- xxx.xxx.xxx.xxx.13759         1         62   RSP
> > 
> > There are some records for every hour change, sometimes more and
> > sometimes less...both TCP and UDP.  You can see that although the
> > records are dropped into the 1969 year directory, the actual timestamps
> > on the records appear to be correct.  I've been running this same
> > configuration for a while without this error, the only thing that changed was the
> > argus-server and argus-clients versions on the two respective servers
> > and I also recently began running with the -d option instead of
> > backgrounding the rasplit process.  Any idea what could be happening
> > here to cause the writing of this pre-epoch directory?
> > 
> > Thank you,
> > 
> > Vince
> > 
> > -- 
> > __   ___ _ __   ___ ___ 
> > \ \ / / | '_ \ / __/ _ \   Vincent Stoffer   Network Security Administrator
> > \ V /| | | | | (_|  __/   Reed College      Portland, Oregon
> >  \_/ |_|_| |_|\___\___|   vince at reed.edu    503-788-6695
> > 
> 



-- 
__   ___ _ __   ___ ___ 
\ \ / / | '_ \ / __/ _ \   Vincent Stoffer   Network Security Administrator
 \ V /| | | | | (_|  __/   Reed College      Portland, Oregon
  \_/ |_|_| |_|\___\___|   vince at reed.edu    503-788-6695