Rasplit issue
Vincent Stoffer
vince at reed.edu
Fri Apr 15 12:48:15 EDT 2011
Hi Carter,
So I saw your note about 3.0.5.4, skipped it and built 3.0.5.5 this
morning and tested again. The problem still exists, however, now the
records that show up in the 1969 directory don't have any timestamps on
them at all. I will follow up with a message off-list with examples.
Thank you,
Vince
* Carter Bullard <carter at qosient.com> [110414 10:35]:
> Hey Vincent,
> I just uploaded argus-clients-3.0.5.4 to the developers site that
> may fix this problem. We've had an issue on the list that I fixed
> today, which deals with time more consistently. It addresses
> something that could cause your errant directory issue in rasplit().
>
> If you could give this a try, I would really appreciate it !!!!!
>
> Carter
>
> On Apr 14, 2011, at 11:09 AM, Carter Bullard wrote:
>
> > Hey Vincent,
> > Can you share any of these flows that you are getting?
> > A file from your errant archive directory would be great !!!!
> >
> > All the ones in your example had a single packet in them.
> > I suspect that they are all dst packets, so that the start and
> > end times for the flow are in the destination time field, rather
> > than the source.
> >
> > Carter
> >
> >
> > On Apr 13, 2011, at 2:21 PM, Vincent Stoffer wrote:
> >
> >> Hi Carter,
> >>
> >> Thanks for the quick response. I did a bit more testing and followed
> >> your instructions. If I run rasplit against the data in the 1969
> >> directory, it does indeed place the records into the correctly dated
> >> directories in /tmp/test (although each days logs all went into
> >> argus.00.log, I take it this is what you are mentioning about the time
> >> order?).
> >>
> >> I also built the 3.0.5.3 argus-clients and re-tested with the
> >> newest radium and rasplit and it is still producing the same behavior.
> >> If there is anything else I can do to help test or debug, let me know.
> >> I did notice one thing during the configure that I thought I should
> >> point out...the config stuck for a bit and then declared there was no
> >> working mktime...it then went on to finish and built OK.
> >>
> >> This behavior certainly isn't a big deal at all, just wanted to let you know what
> >> I have been seeing. I appreciate you taking a look!
> >>
> >> Vince
> >>
> >>
> >> * Carter Bullard <carter at qosient.com> [110413 09:32]:
> >>> Hey Vincent,
> >>> If rasplit() is writing to the wrong directory, but the timestamps are correct,
> >>> then there is a bug in the filename generation routine. 1969/12/31 indicates
> >>> that the algorithm thinks the time is near zero, or a small negative number.
> >>>
> >>> OK, so if you have some time to experiment.
> >>>
> >>> First, let's make sure that the version of rasplit() is the newer one. If so,
> >>>
> >>> If you run rasplit() against the errant data, does it do the right thing?
> >>>
> >>> rasplit -R /log/argus-acad/1969/12/31 -M time 1d -w /tmp/test/%Y/%m/%d/argus.%H.log
> >>>
> >>> if the /tmp/test archive is correct, then you can write the data back into your
> >>> archive. Although, the files that the records should go in will be out of time order.
> >>> To correct that you will need to run rasort() against the files that were modified.
> >>> If you know what day, you can run:
> >>>
> >>> rasort -R /log/argus-acad/2011/04/12 -M replace
> >>>
> >>> Which will correct all the daily files.
> >>>
> >>> If the /tmp/test archive is correct, then I may have something like "reentrant code" issues,
> >>> since the problem won't be because of timestamps in the records.
> >>> If the /tmp/test archive is incorrect, then send me some of the records that
> >>> end up in the wrong year, and I'll use them for debugging.
> >>>
> >>> We have argus-clients-3.0.5.3 in the developers directory that may not
> >>> express this behavior. You may want to try the radium there, as there
> >>> are some time issues and file issues already fixed, althougth not specifically
> >>> this issue.
> >>>
> >>> I'll start looking at all my archives to see if I can find a problem similar to yours.
> >>>
> >>> Carter
> >>>
> >>>
> >>> On Apr 13, 2011, at 11:49 AM, Vincent Stoffer wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> I've recently upgraded to version 3.0.4.1 and I'm seeing an
> >>>> interesting issue that appears to be with rasplit. I'm running the
> >>>> following command to split my Argus data into hourly files within daily
> >>>> directories:
> >>>>
> >>>> rasplit -d -M time 1h -S localhost -w /log/argus-acad/%Y/%m/%d/argus.%H.log
> >>>>
> >>>> Most all of the records show up in the right place, however, I'm now seeing some records show up in the file system at:
> >>>>
> >>>> /log/argus-acad/1969/12/31/argus.16.log
> >>>>
> >>>> Within that file are just a set of records that are right around the
> >>>> change of the hour...for example:
> >>>>
> >>>> 4/13/11 05:59:59 e tcp xxx.xxx.xxx.xxx.57034 -> xxx.xxx.xxx.xxx.18123 1 78 RST
> >>>> 04/13/11 06:00:00 e tcp xxx.xxx.xxx.xxx.57034 -> xxx.xxx.xxx.xxx.18123 1 77 RST
> >>>> 04/13/11 06:59:55 e udp xxx.xxx.xxx.xxx.46442 -> xxx.xxx.xxx.xxx.13759 1 62 INT
> >>>> 04/13/11 07:00:00 e udp xxx.xxx.xxx.xxx.46442 <- xxx.xxx.xxx.xxx.13759 1 62 RSP
> >>>>
> >>>> There are some records for every hour change, sometimes more and
> >>>> sometimes less...both TCP and UDP. You can see that although the
> >>>> records are dropped into the 1969 year directory, the actual timestamps
> >>>> on the records appear to be correct. I've been running this same
> >>>> configuration for a while without this error, the only thing that changed was the
> >>>> argus-server and argus-clients versions on the two respective servers
> >>>> and I also recently began running with the -d option instead of
> >>>> backgrounding the rasplit process. Any idea what could be happening
> >>>> here to cause the writing of this pre-epoch directory?
> >>>>
> >>>> Thank you,
> >>>>
> >>>> Vince
> >>>>
> >>>> --
> >>>> __ ___ _ __ ___ ___
> >>>> \ \ / / | '_ \ / __/ _ \ Vincent Stoffer Network Security Administrator
> >>>> \ V /| | | | | (_| __/ Reed College Portland, Oregon
> >>>> \_/ |_|_| |_|\___\___| vince at reed.edu 503-788-6695
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> __ ___ _ __ ___ ___
> >> \ \ / / | '_ \ / __/ _ \ Vincent Stoffer Network Security Administrator
> >> \ V /| | | | | (_| __/ Reed College Portland, Oregon
> >> \_/ |_|_| |_|\___\___| vince at reed.edu 503-788-6695
> >>
> >
>
--
__ ___ _ __ ___ ___
\ \ / / | '_ \ / __/ _ \ Vincent Stoffer Network Security Administrator
\ V /| | | | | (_| __/ Reed College Portland, Oregon
\_/ |_|_| |_|\___\___| vince at reed.edu 503-788-6695
More information about the argus
mailing list