ratail

Thu Oct 12 15:45:37 EDT 2006

Carter,

   Thanks for the help. I will be running some tests tomorrow. 

  Another question, I am now using radium. Any idea what this error
means:

Oct 12 14:38:56 localhost radium[2118]: 14:38:56.457311
ArgusCheckClientStatus: accept: Socket operation on non-socket

I am getting it many times a second.

Thanks,

Karl

On Thu, 2006-10-12 at 14:57 -0400, Carter Bullard wrote:
> Hey Karl,
> Clients terminate when the socket to the server abruptly closes,  
> fails, or ,.....,
> if the argus sends the clients a CLOSE, ERROR, or SHUTDOWN
> message.
> 
> If an independent ra* client terminates, because another client
> accesses a server, the only 'healthy' way it can do that is if argus
> send the clients CLOSE, or SHUTDOWN messages.  Now its not
> suppose to do that, but never know with a bug what's going on.
> I was just going down the chain to find out who was willing to fuss
> up to either failing or forcibly closing the connections.
> 
> Another approach is to have the ra's run in debug mode (same
> process, I think -D 3 should do it on ra) , and they should tell us why
> they are closing.
> 
> Carter
> 
> 
> On Oct 12, 2006, at 2:32 PM, Karl Tatgenhorst wrote:
> 
> > Carter,
> >
> >    Your response cleared up a couple of other issues for me, it was  
> > very
> > informative thank you. I think however, I have left a couple vague  
> > spots
> > in my summary of the problem. The argus server is not what is  
> > dying, it
> > is the ra process on the remote box that dies with multiple ra -S
> > commands to the same server. It does not appear to be in the argus
> > server itself.
> >
> >    Does this change your opinion?
> >
> > Thanks,
> >
> > Karl
> >
> > On Thu, 2006-10-12 at 14:15 -0400, Carter Bullard wrote:
> >> Karl,
> >> Sorry you're having problems, but you are having unique problems.
> >> And the solution is to fix your argus setup, rather than try to use
> >> a ratail() like function.
> >>
> >> argus is configured to handle up to 16 simultaneous clients, and
> >> so we need to figure out what is wrong with your servers.   The
> >> argus.conf file may have a hint as to what is wrong.
> >>
> >>
> >> Also radium() is there to provide you with a multiplexing function
> >> for argus data.  You can have radium attach to up to 256 servers,
> >> and then access the resulting output as a single stream.   As long
> >> as you configured your argi to have unique source id's then all is
> >> simple and easy.
> >>
> >>
> >> You may need to run a server that is acting up with the "-D 3"  
> >> option,
> >> after compiling the server with debug support
> >> ( touch .debug;./configure;
> >> make clean;make).
> >>
> >>
> >> run this, not as a daemon, and then connect to it with multiple ra's,
> >> and
> >> see what the debug output sez.  If you have problems send mail.
> >> You should see this type of output:
> >>
> >>
> >> isis:/home/carter/argus/argus/argus root# ../bin/argus -D3
> >> argus[26387]: 12 Oct 06 13:58:06.805972 ArgusNewModeler() returning
> >> 0x2800820
> >> argus[26387]: 12 Oct 06 13:58:06.806717 ArgusNewSource() returning
> >> 0x179020
> >> argus[26387]: 12 Oct 06 13:58:06.806783 ArgusNewOutput() returning
> >> retn 0x2804220
> >> argus[26387]: 12 Oct 06 13:58:06.816900 setArgusPortNum(561)  
> >> returning
> >> argus[26387]: 12 Oct 06 13:58:06.817194 ArgusParseResourceFile:
> >> ArgusFilter ""
> >> argus[26387]: 12 Oct 06 13:58:06.817246 ArgusParseResourceFile
> >> (/etc/argus.conf) returning
> >> argus[26387]: 12 Oct 06 13:58:06.817364 setArgusInterfaceStatus(1)
> >> argus[26387]: 12 Oct 06 13:58:06.818717 setArgusDevice(en0) returning
> >> argus[26387]: 12 Oct 06 13:58:06.819275 ArgusInitSource()
> >> pcap_open_live() returned 0x6005c0
> >> argus[26387]: 12 Oct 06 13:58:06.819372 Arguslookup_pcap_callback(1)
> >> returning ArgusEtherPacket(): 0x10ee4
> >> argus[26387]: 12 Oct 06 13:58:06.819842 ArgusInitSource() returning
> >> argus[26387]: 12 Oct 06 13:58:06.819896 ArgusEstablishListen(561,
> >> 0xbffff888) binding: 0
> >> argus[26387]: 12 Oct 06 13:58:06.819970 ArgusEstablishListen(561,
> >> 0xbffff888) returning 7
> >> argus[26387]: 12 Oct 06 13:58:06.820008 ArgusInitOutput() done
> >> argus[26387]: 12 Oct 06 13:58:06.820034 started
> >>
> >>
> >> argus[26470]: 12 Oct 06 14:11:39.492847 ArgusInitModeler() done
> >> argus[26470]: 12 Oct 06 14:11:39.492970 setArgusInterfaceStatus(1)
> >> argus[26470]: 12 Oct 06 14:11:39.493039 ArgusGetInterfaceStatus:
> >> interface en0 is up
> >>
> >>
> >> argus[26470]: 12 Oct 06 14:11:39.493243 setArgusInterfaceStatus(1)
> >>
> >>
> >> and then when you connect using ra, you will get this type of output:
> >>
> >>
> >> argus[26470]: 12 Oct 06 14:11:41.528996 ArgusOutputProcess() select
> >> returned with tasks
> >> argus[26470]: 12 Oct 06 14:11:41.531154 connect from
> >> anubis.newyork.qosient.com
> >>
> >>
> >> argus[26470]: 12 Oct 06 14:11:41.531419 ArgusCheckClientStatus() new
> >> client 0
> >> argus[26470]: 12 Oct 06 14:11:41.531539 ArgusNewSocket (10) returning
> >> 0x1d2020
> >> argus[26470]: 12 Oct 06 14:11:41.531649 ArgusCheckClientStatus()
> >> returning
> >> argus[26470]: 12 Oct 06 14:11:41.732534 ArgusOutputProcess() select
> >> returned with tasks
> >> argus[26470]: 12 Oct 06 14:11:41.732648 ArgusCheckClientMessage
> >> (0x2804234, 10) recv() returned 7 bytes
> >> argus[26470]: 12 Oct 06 14:11:41.732678 ArgusCheckClientMessage
> >> (0x2804234, 10) read 'START: ' from remote
> >>
> >>
> >>
> >>
> >> When you disconnect the ra(), you should get something like this:
> >>
> >>
> >>
> >>
> >> argus[26470]: 12 Oct 06 14:13:00.113676 ArgusOutputProcess() select
> >> returned with tasks
> >> argus[26470]: 12 Oct 06 14:13:00.113769 ArgusCheckClientMessage
> >> (0x2804254, 11) recv() returned 0 bytes
> >> argus[26470]: 12 Oct 06 14:13:00.113804 ArgusDeleteList (0x6018a0, 4)
> >> returning
> >> argus[26470]: 12 Oct 06 14:13:00.113941 ArgusDeleteSocket (0x1e3020)
> >> returning
> >>
> >>
> >>
> >>
> >>
> >>
> >> So for record collection, there is a lot of new support in argus-3.0.
> >> I would recommend that you think about using rasplit() as a part
> >> of your collection system.   something like this would work for your
> >> example:
> >>
> >>
> >>    rasplit -M time 5m -S argus-north... -w /var/log/argus/$srcid/%Y/%
> >> m/%d/file.%Y.%m%d.%H.%M.%S
> >>
> >>
> >> with the "-M time 5m" you will output the records into 5 minute  
> >> files,
> >> and the outputfile name can have record contents ('$srcid') and
> >> strftime()
> >> syntax ('%H.%M.%S'),  so you get it right.   I think you will find it
> >> more
> >> flexible.  Then you can run a cron job to process the 5 minute files.
> >>
> >>
> >> Oh, and just a side note, don't need a port number if you're
> >> using the default 561 in the server directive, although it
> >> makes it clear what you're trying to do.  and if you're writing
> >> records out to a pipe or a file, the "-s ..." options are not
> >> necessary.
> >>
> >>
> >> Carter
> >>
> >>
> >>
> >>
> >>
> >> On Oct 12, 2006, at 11:12 AM, Karl Tatgenhorst wrote:
> >>
> >>> Hi,
> >>>
> >>>
> >>>     My subject is not rat tail :-) One of my coworkers built a
> >>> process
> >>> that monitors data from the argus stream and alerts on it for us
> >>> (IDS
> >>> style), however we bumped into a problem. We run this process on the
> >>> same server that we collect data from our Argus sensors on. When we
> >>> start the process with the -S <server> switch the existing processes
> >>> using -S die. It seems as though each sensor can only have one ra -S
> >>> process on this box. The box is enormously scaled for this and is
> >>> not
> >>> being overloaded. The code I am using is rc.30
> >>>
> >>>
> >>> here are the commands I keep running for storage:
> >>>
> >>>
> >>> ra -S argus-south.uchicago.edu:561 -s +suser +duser -
> >>> w /var/log/argus/unprocessed/tmp.argus_south
> >>>
> >>>
> >>> ra -S argus-north.uchicago.edu:561 -s +suser +duser -
> >>> w /var/log/argus/unprocessed/tmp.argus_north
> >>>
> >>>
> >>>
> >>>
> >>> This is the command that my coworker starts. When he does this for
> >>> whichever server, the other monitor for that server dies.
> >>>
> >>>
> >>>  ra -S argus-north.uchicago.edu:561 -s saddr sport daddr dport
> >>> suser:128
> >>> duser:128
> >>>
> >>>
> >>>
> >>>
> >>> I seem to remember a thread about this before, so I suggested
> >>> running ra
> >>> against my tmp files. These files are on a ram-san so IO wait is not
> >>> an
> >>> issue. The problem then comes in that he uses ra and tail -f then
> >>> tail
> >>> craps out because the last bit of the file is strange. Without tail
> >>> the
> >>> file gets to the end and finishes instead of watching for growth. So
> >>> I
> >>> was wondering if anyone out there was up to building an ra that
> >>> would
> >>> open the file again every few seconds or so at the last known offset
> >>> much like tail -f or if someone knows another solution to our
> >>> problem.
> >>>
> >>>
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Karl
> >>>
> >>>
> >>>
> 