ratail
Carter Bullard
carter at qosient.com
Thu Oct 12 15:51:05 EDT 2006
Haven't seen that. The radium.conf file is different, a mix of
argus.conf
and .rarc. You may be stepping on something in that file.
If you send it, I'll run down the error.
Carter
On Oct 12, 2006, at 3:45 PM, Karl Tatgenhorst wrote:
>
>
> Carter,
>
> Thanks for the help. I will be running some tests tomorrow.
>
> Another question, I am now using radium. Any idea what this error
> means:
>
> Oct 12 14:38:56 localhost radium[2118]: 14:38:56.457311
> ArgusCheckClientStatus: accept: Socket operation on non-socket
>
> I am getting it many times a second.
>
> Thanks,
>
> Karl
>
> On Thu, 2006-10-12 at 14:57 -0400, Carter Bullard wrote:
>> Hey Karl,
>> Clients terminate when the socket to the server abruptly closes,
>> fails, or ,.....,
>> if the argus sends the clients a CLOSE, ERROR, or SHUTDOWN
>> message.
>>
>> If an independent ra* client terminates, because another client
>> accesses a server, the only 'healthy' way it can do that is if argus
>> send the clients CLOSE, or SHUTDOWN messages. Now its not
>> suppose to do that, but never know with a bug what's going on.
>> I was just going down the chain to find out who was willing to fuss
>> up to either failing or forcibly closing the connections.
>>
>> Another approach is to have the ra's run in debug mode (same
>> process, I think -D 3 should do it on ra) , and they should tell
>> us why
>> they are closing.
>>
>> Carter
>>
>>
>> On Oct 12, 2006, at 2:32 PM, Karl Tatgenhorst wrote:
>>
>>> Carter,
>>>
>>> Your response cleared up a couple of other issues for me, it was
>>> very
>>> informative thank you. I think however, I have left a couple vague
>>> spots
>>> in my summary of the problem. The argus server is not what is
>>> dying, it
>>> is the ra process on the remote box that dies with multiple ra -S
>>> commands to the same server. It does not appear to be in the argus
>>> server itself.
>>>
>>> Does this change your opinion?
>>>
>>> Thanks,
>>>
>>> Karl
>>>
>>> On Thu, 2006-10-12 at 14:15 -0400, Carter Bullard wrote:
>>>> Karl,
>>>> Sorry you're having problems, but you are having unique problems.
>>>> And the solution is to fix your argus setup, rather than try to use
>>>> a ratail() like function.
>>>>
>>>> argus is configured to handle up to 16 simultaneous clients, and
>>>> so we need to figure out what is wrong with your servers. The
>>>> argus.conf file may have a hint as to what is wrong.
>>>>
>>>>
>>>> Also radium() is there to provide you with a multiplexing function
>>>> for argus data. You can have radium attach to up to 256 servers,
>>>> and then access the resulting output as a single stream. As long
>>>> as you configured your argi to have unique source id's then all is
>>>> simple and easy.
>>>>
>>>>
>>>> You may need to run a server that is acting up with the "-D 3"
>>>> option,
>>>> after compiling the server with debug support
>>>> ( touch .debug;./configure;
>>>> make clean;make).
>>>>
>>>>
>>>> run this, not as a daemon, and then connect to it with multiple
>>>> ra's,
>>>> and
>>>> see what the debug output sez. If you have problems send mail.
>>>> You should see this type of output:
>>>>
>>>>
>>>> isis:/home/carter/argus/argus/argus root# ../bin/argus -D3
>>>> argus[26387]: 12 Oct 06 13:58:06.805972 ArgusNewModeler() returning
>>>> 0x2800820
>>>> argus[26387]: 12 Oct 06 13:58:06.806717 ArgusNewSource() returning
>>>> 0x179020
>>>> argus[26387]: 12 Oct 06 13:58:06.806783 ArgusNewOutput() returning
>>>> retn 0x2804220
>>>> argus[26387]: 12 Oct 06 13:58:06.816900 setArgusPortNum(561)
>>>> returning
>>>> argus[26387]: 12 Oct 06 13:58:06.817194 ArgusParseResourceFile:
>>>> ArgusFilter ""
>>>> argus[26387]: 12 Oct 06 13:58:06.817246 ArgusParseResourceFile
>>>> (/etc/argus.conf) returning
>>>> argus[26387]: 12 Oct 06 13:58:06.817364 setArgusInterfaceStatus(1)
>>>> argus[26387]: 12 Oct 06 13:58:06.818717 setArgusDevice(en0)
>>>> returning
>>>> argus[26387]: 12 Oct 06 13:58:06.819275 ArgusInitSource()
>>>> pcap_open_live() returned 0x6005c0
>>>> argus[26387]: 12 Oct 06 13:58:06.819372 Arguslookup_pcap_callback
>>>> (1)
>>>> returning ArgusEtherPacket(): 0x10ee4
>>>> argus[26387]: 12 Oct 06 13:58:06.819842 ArgusInitSource() returning
>>>> argus[26387]: 12 Oct 06 13:58:06.819896 ArgusEstablishListen(561,
>>>> 0xbffff888) binding: 0
>>>> argus[26387]: 12 Oct 06 13:58:06.819970 ArgusEstablishListen(561,
>>>> 0xbffff888) returning 7
>>>> argus[26387]: 12 Oct 06 13:58:06.820008 ArgusInitOutput() done
>>>> argus[26387]: 12 Oct 06 13:58:06.820034 started
>>>>
>>>>
>>>> argus[26470]: 12 Oct 06 14:11:39.492847 ArgusInitModeler() done
>>>> argus[26470]: 12 Oct 06 14:11:39.492970 setArgusInterfaceStatus(1)
>>>> argus[26470]: 12 Oct 06 14:11:39.493039 ArgusGetInterfaceStatus:
>>>> interface en0 is up
>>>>
>>>>
>>>> argus[26470]: 12 Oct 06 14:11:39.493243 setArgusInterfaceStatus(1)
>>>>
>>>>
>>>> and then when you connect using ra, you will get this type of
>>>> output:
>>>>
>>>>
>>>> argus[26470]: 12 Oct 06 14:11:41.528996 ArgusOutputProcess() select
>>>> returned with tasks
>>>> argus[26470]: 12 Oct 06 14:11:41.531154 connect from
>>>> anubis.newyork.qosient.com
>>>>
>>>>
>>>> argus[26470]: 12 Oct 06 14:11:41.531419 ArgusCheckClientStatus()
>>>> new
>>>> client 0
>>>> argus[26470]: 12 Oct 06 14:11:41.531539 ArgusNewSocket (10)
>>>> returning
>>>> 0x1d2020
>>>> argus[26470]: 12 Oct 06 14:11:41.531649 ArgusCheckClientStatus()
>>>> returning
>>>> argus[26470]: 12 Oct 06 14:11:41.732534 ArgusOutputProcess() select
>>>> returned with tasks
>>>> argus[26470]: 12 Oct 06 14:11:41.732648 ArgusCheckClientMessage
>>>> (0x2804234, 10) recv() returned 7 bytes
>>>> argus[26470]: 12 Oct 06 14:11:41.732678 ArgusCheckClientMessage
>>>> (0x2804234, 10) read 'START: ' from remote
>>>>
>>>>
>>>>
>>>>
>>>> When you disconnect the ra(), you should get something like this:
>>>>
>>>>
>>>>
>>>>
>>>> argus[26470]: 12 Oct 06 14:13:00.113676 ArgusOutputProcess() select
>>>> returned with tasks
>>>> argus[26470]: 12 Oct 06 14:13:00.113769 ArgusCheckClientMessage
>>>> (0x2804254, 11) recv() returned 0 bytes
>>>> argus[26470]: 12 Oct 06 14:13:00.113804 ArgusDeleteList
>>>> (0x6018a0, 4)
>>>> returning
>>>> argus[26470]: 12 Oct 06 14:13:00.113941 ArgusDeleteSocket
>>>> (0x1e3020)
>>>> returning
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> So for record collection, there is a lot of new support in
>>>> argus-3.0.
>>>> I would recommend that you think about using rasplit() as a part
>>>> of your collection system. something like this would work for
>>>> your
>>>> example:
>>>>
>>>>
>>>> rasplit -M time 5m -S argus-north... -w /var/log/argus/$srcid/
>>>> %Y/%
>>>> m/%d/file.%Y.%m%d.%H.%M.%S
>>>>
>>>>
>>>> with the "-M time 5m" you will output the records into 5 minute
>>>> files,
>>>> and the outputfile name can have record contents ('$srcid') and
>>>> strftime()
>>>> syntax ('%H.%M.%S'), so you get it right. I think you will
>>>> find it
>>>> more
>>>> flexible. Then you can run a cron job to process the 5 minute
>>>> files.
>>>>
>>>>
>>>> Oh, and just a side note, don't need a port number if you're
>>>> using the default 561 in the server directive, although it
>>>> makes it clear what you're trying to do. and if you're writing
>>>> records out to a pipe or a file, the "-s ..." options are not
>>>> necessary.
>>>>
>>>>
>>>> Carter
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Oct 12, 2006, at 11:12 AM, Karl Tatgenhorst wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>> My subject is not rat tail :-) One of my coworkers built a
>>>>> process
>>>>> that monitors data from the argus stream and alerts on it for us
>>>>> (IDS
>>>>> style), however we bumped into a problem. We run this process
>>>>> on the
>>>>> same server that we collect data from our Argus sensors on.
>>>>> When we
>>>>> start the process with the -S <server> switch the existing
>>>>> processes
>>>>> using -S die. It seems as though each sensor can only have one
>>>>> ra -S
>>>>> process on this box. The box is enormously scaled for this and is
>>>>> not
>>>>> being overloaded. The code I am using is rc.30
>>>>>
>>>>>
>>>>> here are the commands I keep running for storage:
>>>>>
>>>>>
>>>>> ra -S argus-south.uchicago.edu:561 -s +suser +duser -
>>>>> w /var/log/argus/unprocessed/tmp.argus_south
>>>>>
>>>>>
>>>>> ra -S argus-north.uchicago.edu:561 -s +suser +duser -
>>>>> w /var/log/argus/unprocessed/tmp.argus_north
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> This is the command that my coworker starts. When he does this for
>>>>> whichever server, the other monitor for that server dies.
>>>>>
>>>>>
>>>>> ra -S argus-north.uchicago.edu:561 -s saddr sport daddr dport
>>>>> suser:128
>>>>> duser:128
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I seem to remember a thread about this before, so I suggested
>>>>> running ra
>>>>> against my tmp files. These files are on a ram-san so IO wait
>>>>> is not
>>>>> an
>>>>> issue. The problem then comes in that he uses ra and tail -f then
>>>>> tail
>>>>> craps out because the last bit of the file is strange. Without
>>>>> tail
>>>>> the
>>>>> file gets to the end and finishes instead of watching for
>>>>> growth. So
>>>>> I
>>>>> was wondering if anyone out there was up to building an ra that
>>>>> would
>>>>> open the file again every few seconds or so at the last known
>>>>> offset
>>>>> much like tail -f or if someone knows another solution to our
>>>>> problem.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>>
>>
>
>
Carter Bullard
CEO/President
QoSient, LLC
150 E. 57th Street Suite 12D
New York, New York 10022
+1 212 588-9133 Phone
+1 212 588-9134 Fax
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20061012/623d2ff9/attachment.html>
More information about the argus
mailing list