[ARGUS] log file roll-over

[John Nagro]

Awesome. This really has been a huge help, i really appreciate your
time. I have a lot of poking to do but i will let you know what
progress i make and if i have any more questions.

-John

On Wed, 23 Jun 2004 13:42:19 -0700, Peter Van Epp <vanepp at sfu.ca> wrote:
> 
>         Yep. They span a shell and run zcat on the file internally. All very
> convienient :-).
> 
> Peter Van Epp / Operations and Technical Support
> Simon Fraser University, Burnaby, B.C. Canada
> 
> 
> 
> On Wed, Jun 23, 2004 at 04:38:26PM -0400, John Nagro wrote:
> > So youre saying in general the tools can read the compressed version???
> >
> > On Wed, 23 Jun 2004 13:30:22 -0700, Peter Van Epp <vanepp at sfu.ca> wrote:
> > >
> > >         Yep all the tools will work from either live data from a socket or
> > > from a file via the -r flag as far as I know (I haven't used ragraph but I
> > > assume it will be the same as all the rest). Replacing the -r argus.out with
> > > -r archive/2004/06/23/argus.2004.06.23.12.00.00.gz here will get me the
> > > previous hour's data in stead of the current data in argus.out.
> > >         If you want to graph things, you may want to look at the cflowd stuff
> > > too (although that may be what ragraph is using under the covers I haven't
> > > been paying too close attention to the graphical side of argus :-)). I haven't
> > > used that either but I believe it can graph either Cisco netflow or argus
> > > data. There should be references to it in the mailing list archives.
> > >
> > > Peter Van Epp / Operations and Technical Support
> > > Simon Fraser University, Burnaby, B.C. Canada
> > >
> > >
> > >
> > > On Wed, Jun 23, 2004 at 04:14:58PM -0400, John Nagro wrote:
> > > > This information is very helpfull, thank you. Once i archive the data
> > > > using this script, can thinsg like ragraph still use the archived
> > > > data? as well as the current argus.out?
> > > >
> > > > -John
> > > >
> > > > On Wed, 23 Jun 2004 13:06:31 -0700, Peter Van Epp <vanepp at sfu.ca> wrote:
> > > > >
> > > > >         Ah, now it begins to make sense. We are talking two different things
> > > > > here. I'm (because my volume is low enough) running argus and archiving on
> > > > > the same box, and not doing it in real time. So in my case
> > > > >
> > > > > argus_bpf -w argus.out
> > > > >
> > > > > is running in the background and every hour argusachive swipes and archives
> > > > > the data file. Then I run ra (or any of the other tools) against the saved
> > > > > file as in  "ra -r argus.out -c -nn". It sounds like you are running argus
> > > > > on a sensor machine (the best thing to do at high volumes for performance
> > > > > reasons) and writing the output data to a socket. On another machine you
> > > > > have ra (or the other tools) listening to that socket and processing the data
> > > > > in real time. In this instance you will get the current data that is coming
> > > > > from the argus sensor in real time. It won't be archived anywhere. The usual
> > > > > answer here is to run ra writing to a file and use argus archive to save the
> > > > > data (you can also have another copy of ra reading the data from the socket
> > > > > and processing it in real time if you have the horsepower and the need). It
> > > > > looks like this:
> > > > >
> > > > > Machine 1 Sensor                        Machine 2
> > > > >
> > > > > argus_bpf -P 950 (etc)          ra -S address_machine_1 -P950 -w argus.out
> > > > >
> > > > > which writes the argus          This machine accepts the data from the sensor
> > > > > data to socket 950              machine and writes it to file argus.out. Here
> > > > >                                 argusarchive is run out of cron to archive the
> > > > >                                 argus data to disk without impacting the
> > > > >                                 sensor machine (the disk writes appear to cause
> > > > >                                 packet loss on the sensor machine at high
> > > > >                                 speeds).
> > > > >
> > > > >                                 ra -S address -P950 -c -nn
> > > > >
> > > > >                                 would process the data stream in real time
> > > > >                                 independent of the archive stream, and this
> > > > >                                 sounds like what you are doing now. This one
> > > > >                                 is optional, you can chose to run
> > > > >                                 ra -r argus.out (or an archive file) -c -nn
> > > > >                                 as long as you have the top ra reading the
> > > > >                                 data stream and storing it to disk.
> > > > >
> > > > > Is this more on the lines of what you wanted to know?
> > > > >
> > > > >
> > > > > Peter Van Epp / Operations and Technical Support
> > > > > Simon Fraser University, Burnaby, B.C. Canada
> > > > >
> > > > > On Wed, Jun 23, 2004 at 03:42:36PM -0400, John Nagro wrote:
> > > > > > Ah yes, thank you, for some reason the debian package you get from apt
> > > > > > doesnt install that part. But this still isnt roll-over, this simple
> > > > > > swaps out the file once its reached a certain size. How does this
> > > > > > effect my ability to analyze data? for example i run the server
> > > > > > software on a system, and i intend on connecting to it using the
> > > > > > client software (-S <computer> option in most tools). If cron has
> > > > > > *just* swapped out the file, what sort of data will i get? none?
> > > > > >
> > > > > > -John
> > > > > >
> > > > >
> > >
>