ratail

Thu Oct 12 14:15:21 EDT 2006

Karl,
Sorry you're having problems, but you are having unique problems.
And the solution is to fix your argus setup, rather than try to use
a ratail() like function.

argus is configured to handle up to 16 simultaneous clients, and
so we need to figure out what is wrong with your servers.   The
argus.conf file may have a hint as to what is wrong.

Also radium() is there to provide you with a multiplexing function
for argus data.  You can have radium attach to up to 256 servers,
and then access the resulting output as a single stream.   As long
as you configured your argi to have unique source id's then all is
simple and easy.

You may need to run a server that is acting up with the "-D 3" option,
after compiling the server with debug support ( touch .debug;./ 
configure;
make clean;make).

run this, not as a daemon, and then connect to it with multiple ra's,  
and
see what the debug output sez.  If you have problems send mail.
You should see this type of output:

isis:/home/carter/argus/argus/argus root# ../bin/argus -D3
argus[26387]: 12 Oct 06 13:58:06.805972 ArgusNewModeler() returning  
0x2800820
argus[26387]: 12 Oct 06 13:58:06.806717 ArgusNewSource() returning  
0x179020
argus[26387]: 12 Oct 06 13:58:06.806783 ArgusNewOutput() returning  
retn 0x2804220
argus[26387]: 12 Oct 06 13:58:06.816900 setArgusPortNum(561) returning
argus[26387]: 12 Oct 06 13:58:06.817194 ArgusParseResourceFile:  
ArgusFilter ""
argus[26387]: 12 Oct 06 13:58:06.817246 ArgusParseResourceFile (/etc/ 
argus.conf) returning
argus[26387]: 12 Oct 06 13:58:06.817364 setArgusInterfaceStatus(1)
argus[26387]: 12 Oct 06 13:58:06.818717 setArgusDevice(en0) returning
argus[26387]: 12 Oct 06 13:58:06.819275 ArgusInitSource()  
pcap_open_live() returned 0x6005c0
argus[26387]: 12 Oct 06 13:58:06.819372 Arguslookup_pcap_callback(1)  
returning ArgusEtherPacket(): 0x10ee4
argus[26387]: 12 Oct 06 13:58:06.819842 ArgusInitSource() returning
argus[26387]: 12 Oct 06 13:58:06.819896 ArgusEstablishListen(561,  
0xbffff888) binding: 0
argus[26387]: 12 Oct 06 13:58:06.819970 ArgusEstablishListen(561,  
0xbffff888) returning 7
argus[26387]: 12 Oct 06 13:58:06.820008 ArgusInitOutput() done
argus[26387]: 12 Oct 06 13:58:06.820034 started

argus[26470]: 12 Oct 06 14:11:39.492847 ArgusInitModeler() done
argus[26470]: 12 Oct 06 14:11:39.492970 setArgusInterfaceStatus(1)
argus[26470]: 12 Oct 06 14:11:39.493039 ArgusGetInterfaceStatus:  
interface en0 is up

argus[26470]: 12 Oct 06 14:11:39.493243 setArgusInterfaceStatus(1)

and then when you connect using ra, you will get this type of output:

argus[26470]: 12 Oct 06 14:11:41.528996 ArgusOutputProcess() select  
returned with tasks
argus[26470]: 12 Oct 06 14:11:41.531154 connect from  
anubis.newyork.qosient.com

argus[26470]: 12 Oct 06 14:11:41.531419 ArgusCheckClientStatus() new  
client 0
argus[26470]: 12 Oct 06 14:11:41.531539 ArgusNewSocket (10) returning  
0x1d2020
argus[26470]: 12 Oct 06 14:11:41.531649 ArgusCheckClientStatus()  
returning
argus[26470]: 12 Oct 06 14:11:41.732534 ArgusOutputProcess() select  
returned with tasks
argus[26470]: 12 Oct 06 14:11:41.732648 ArgusCheckClientMessage  
(0x2804234, 10) recv() returned 7 bytes
argus[26470]: 12 Oct 06 14:11:41.732678 ArgusCheckClientMessage  
(0x2804234, 10) read 'START: ' from remote

When you disconnect the ra(), you should get something like this:

argus[26470]: 12 Oct 06 14:13:00.113676 ArgusOutputProcess() select  
returned with tasks
argus[26470]: 12 Oct 06 14:13:00.113769 ArgusCheckClientMessage  
(0x2804254, 11) recv() returned 0 bytes
argus[26470]: 12 Oct 06 14:13:00.113804 ArgusDeleteList (0x6018a0, 4)  
returning
argus[26470]: 12 Oct 06 14:13:00.113941 ArgusDeleteSocket (0x1e3020)  
returning

So for record collection, there is a lot of new support in argus-3.0.
I would recommend that you think about using rasplit() as a part
of your collection system.   something like this would work for your
example:

    rasplit -M time 5m -S argus-north... -w /var/log/argus/$srcid/%Y/% 
m/%d/file.%Y.%m%d.%H.%M.%S

with the "-M time 5m" you will output the records into 5 minute files,
and the outputfile name can have record contents ('$srcid') and  
strftime()
syntax ('%H.%M.%S'),  so you get it right.   I think you will find it  
more
flexible.  Then you can run a cron job to process the 5 minute files.

Oh, and just a side note, don't need a port number if you're
using the default 561 in the server directive, although it
makes it clear what you're trying to do.  and if you're writing
records out to a pipe or a file, the "-s ..." options are not necessary.

Carter

On Oct 12, 2006, at 11:12 AM, Karl Tatgenhorst wrote:

> Hi,
>
>     My subject is not rat tail :-) One of my coworkers built a process
> that monitors data from the argus stream and alerts on it for us (IDS
> style), however we bumped into a problem. We run this process on the
> same server that we collect data from our Argus sensors on. When we
> start the process with the -S <server> switch the existing processes
> using -S die. It seems as though each sensor can only have one ra -S
> process on this box. The box is enormously scaled for this and is not
> being overloaded. The code I am using is rc.30
>
> here are the commands I keep running for storage:
>
> ra -S argus-south.uchicago.edu:561 -s +suser +duser -
> w /var/log/argus/unprocessed/tmp.argus_south
>
> ra -S argus-north.uchicago.edu:561 -s +suser +duser -
> w /var/log/argus/unprocessed/tmp.argus_north
>
>
> This is the command that my coworker starts. When he does this for
> whichever server, the other monitor for that server dies.
>
>  ra -S argus-north.uchicago.edu:561 -s saddr sport daddr dport  
> suser:128
> duser:128
>
>
> I seem to remember a thread about this before, so I suggested  
> running ra
> against my tmp files. These files are on a ram-san so IO wait is  
> not an
> issue. The problem then comes in that he uses ra and tail -f then tail
> craps out because the last bit of the file is strange. Without tail  
> the
> file gets to the end and finishes instead of watching for growth. So I
> was wondering if anyone out there was up to building an ra that would
> open the file again every few seconds or so at the last known offset
> much like tail -f or if someone knows another solution to our problem.
>
>
> Thanks,
>
> Karl
>
>

Carter Bullard
CEO/President
QoSient, LLC
150 E. 57th Street Suite 12D
New York, New York 10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20061012/8c586bfb/attachment.html>