ratail

Thu Oct 12 14:32:55 EDT 2006

Carter,

   Your response cleared up a couple of other issues for me, it was very
informative thank you. I think however, I have left a couple vague spots
in my summary of the problem. The argus server is not what is dying, it
is the ra process on the remote box that dies with multiple ra -S
commands to the same server. It does not appear to be in the argus
server itself.

   Does this change your opinion?

Thanks,

Karl

On Thu, 2006-10-12 at 14:15 -0400, Carter Bullard wrote:
> Karl,
> Sorry you're having problems, but you are having unique problems.
> And the solution is to fix your argus setup, rather than try to use
> a ratail() like function.
> 
> argus is configured to handle up to 16 simultaneous clients, and
> so we need to figure out what is wrong with your servers.   The
> argus.conf file may have a hint as to what is wrong.
> 
> 
> Also radium() is there to provide you with a multiplexing function
> for argus data.  You can have radium attach to up to 256 servers,
> and then access the resulting output as a single stream.   As long
> as you configured your argi to have unique source id's then all is
> simple and easy.
> 
> 
> You may need to run a server that is acting up with the "-D 3" option,
> after compiling the server with debug support
> ( touch .debug;./configure;
> make clean;make).
> 
> 
> run this, not as a daemon, and then connect to it with multiple ra's,
> and
> see what the debug output sez.  If you have problems send mail.
> You should see this type of output:
> 
> 
> isis:/home/carter/argus/argus/argus root# ../bin/argus -D3
> argus[26387]: 12 Oct 06 13:58:06.805972 ArgusNewModeler() returning
> 0x2800820
> argus[26387]: 12 Oct 06 13:58:06.806717 ArgusNewSource() returning
> 0x179020
> argus[26387]: 12 Oct 06 13:58:06.806783 ArgusNewOutput() returning
> retn 0x2804220
> argus[26387]: 12 Oct 06 13:58:06.816900 setArgusPortNum(561) returning
> argus[26387]: 12 Oct 06 13:58:06.817194 ArgusParseResourceFile:
> ArgusFilter "" 
> argus[26387]: 12 Oct 06 13:58:06.817246 ArgusParseResourceFile
> (/etc/argus.conf) returning
> argus[26387]: 12 Oct 06 13:58:06.817364 setArgusInterfaceStatus(1)
> argus[26387]: 12 Oct 06 13:58:06.818717 setArgusDevice(en0) returning
> argus[26387]: 12 Oct 06 13:58:06.819275 ArgusInitSource()
> pcap_open_live() returned 0x6005c0
> argus[26387]: 12 Oct 06 13:58:06.819372 Arguslookup_pcap_callback(1)
> returning ArgusEtherPacket(): 0x10ee4
> argus[26387]: 12 Oct 06 13:58:06.819842 ArgusInitSource() returning
> argus[26387]: 12 Oct 06 13:58:06.819896 ArgusEstablishListen(561,
> 0xbffff888) binding: 0
> argus[26387]: 12 Oct 06 13:58:06.819970 ArgusEstablishListen(561,
> 0xbffff888) returning 7
> argus[26387]: 12 Oct 06 13:58:06.820008 ArgusInitOutput() done
> argus[26387]: 12 Oct 06 13:58:06.820034 started
> 
> 
> argus[26470]: 12 Oct 06 14:11:39.492847 ArgusInitModeler() done
> argus[26470]: 12 Oct 06 14:11:39.492970 setArgusInterfaceStatus(1)
> argus[26470]: 12 Oct 06 14:11:39.493039 ArgusGetInterfaceStatus:
> interface en0 is up
> 
> 
> argus[26470]: 12 Oct 06 14:11:39.493243 setArgusInterfaceStatus(1)
> 
> 
> and then when you connect using ra, you will get this type of output:
> 
> 
> argus[26470]: 12 Oct 06 14:11:41.528996 ArgusOutputProcess() select
> returned with tasks
> argus[26470]: 12 Oct 06 14:11:41.531154 connect from
> anubis.newyork.qosient.com
> 
> 
> argus[26470]: 12 Oct 06 14:11:41.531419 ArgusCheckClientStatus() new
> client 0
> argus[26470]: 12 Oct 06 14:11:41.531539 ArgusNewSocket (10) returning
> 0x1d2020
> argus[26470]: 12 Oct 06 14:11:41.531649 ArgusCheckClientStatus()
> returning
> argus[26470]: 12 Oct 06 14:11:41.732534 ArgusOutputProcess() select
> returned with tasks
> argus[26470]: 12 Oct 06 14:11:41.732648 ArgusCheckClientMessage
> (0x2804234, 10) recv() returned 7 bytes
> argus[26470]: 12 Oct 06 14:11:41.732678 ArgusCheckClientMessage
> (0x2804234, 10) read 'START: ' from remote
> 
> 
> 
> 
> When you disconnect the ra(), you should get something like this:
> 
> 
> 
> 
> argus[26470]: 12 Oct 06 14:13:00.113676 ArgusOutputProcess() select
> returned with tasks
> argus[26470]: 12 Oct 06 14:13:00.113769 ArgusCheckClientMessage
> (0x2804254, 11) recv() returned 0 bytes
> argus[26470]: 12 Oct 06 14:13:00.113804 ArgusDeleteList (0x6018a0, 4)
> returning
> argus[26470]: 12 Oct 06 14:13:00.113941 ArgusDeleteSocket (0x1e3020)
> returning
> 
> 
> 
> 
> 
> 
> So for record collection, there is a lot of new support in argus-3.0.
> I would recommend that you think about using rasplit() as a part
> of your collection system.   something like this would work for your
> example:
> 
> 
>    rasplit -M time 5m -S argus-north... -w /var/log/argus/$srcid/%Y/%
> m/%d/file.%Y.%m%d.%H.%M.%S
> 
> 
> with the "-M time 5m" you will output the records into 5 minute files,
> and the outputfile name can have record contents ('$srcid') and
> strftime()
> syntax ('%H.%M.%S'),  so you get it right.   I think you will find it
> more
> flexible.  Then you can run a cron job to process the 5 minute files.
> 
> 
> Oh, and just a side note, don't need a port number if you're
> using the default 561 in the server directive, although it
> makes it clear what you're trying to do.  and if you're writing
> records out to a pipe or a file, the "-s ..." options are not
> necessary.
> 
> 
> Carter
> 
> 
> 
> 
> 
> On Oct 12, 2006, at 11:12 AM, Karl Tatgenhorst wrote:
> 
> > Hi,
> > 
> > 
> >     My subject is not rat tail :-) One of my coworkers built a
> > process
> > that monitors data from the argus stream and alerts on it for us
> > (IDS
> > style), however we bumped into a problem. We run this process on the
> > same server that we collect data from our Argus sensors on. When we
> > start the process with the -S <server> switch the existing processes
> > using -S die. It seems as though each sensor can only have one ra -S
> > process on this box. The box is enormously scaled for this and is
> > not
> > being overloaded. The code I am using is rc.30
> > 
> > 
> > here are the commands I keep running for storage:
> > 
> > 
> > ra -S argus-south.uchicago.edu:561 -s +suser +duser -
> > w /var/log/argus/unprocessed/tmp.argus_south
> > 
> > 
> > ra -S argus-north.uchicago.edu:561 -s +suser +duser -
> > w /var/log/argus/unprocessed/tmp.argus_north
> > 
> > 
> > 
> > 
> > This is the command that my coworker starts. When he does this for
> > whichever server, the other monitor for that server dies.
> > 
> > 
> >  ra -S argus-north.uchicago.edu:561 -s saddr sport daddr dport
> > suser:128
> > duser:128
> > 
> > 
> > 
> > 
> > I seem to remember a thread about this before, so I suggested
> > running ra
> > against my tmp files. These files are on a ram-san so IO wait is not
> > an
> > issue. The problem then comes in that he uses ra and tail -f then
> > tail
> > craps out because the last bit of the file is strange. Without tail
> > the
> > file gets to the end and finishes instead of watching for growth. So
> > I
> > was wondering if anyone out there was up to building an ra that
> > would
> > open the file again every few seconds or so at the last known offset
> > much like tail -f or if someone knows another solution to our
> > problem.
> > 
> > 
> > 
> > 
> > Thanks,
> > 
> > 
> > Karl
> > 
> > 
> > 
> > 
> 
> Carter Bullard
> CEO/President
> QoSient, LLC
> 150 E. 57th Street Suite 12D
> New York, New York 10022
> 
> 
> +1 212 588-9133 Phone
> +1 212 588-9134 Fax
> 
> 
> 
> 