Error Starting Argus Daemon
Carter Bullard
carter at qosient.com
Thu Jan 31 08:50:15 EST 2013
Hey Neal,
Hmmmm, you've run down the path, but we may need to back up a bit, and
test a few things. Having argus, itself, write to an output file is deprecated,
as we have a set of clients that do a better job than argus does, and we want
argus to use its cycles to process packets, not write to the disk.
But, that doesn't mean you can't do it, so lets try a few things.
First, with all the changes, lets get it running, verify that it does the basic things,
and, then move on, to a reasonable installation.
Lets not run as a daemon, in the first pass, either comment out ARGUS_DAEMON
in the argus.conf file, or add a " -d " to toggle that option. Be sure to use the " -d "
after the "-F argus.conf" option on the command line. Order matters.
Don't use the Debug option if you are running argus as a daemon. Thats a lot of
debug messages that will be going into the system syslog, and that probably
isn't going to perform very well. Use the -D when argus is running from a shell,
in the foreground.
What interface is argus opening? When argus starts, it should declare which
interface is " up " which will tell us which interface it's opening. Is there traffic on
that interface? When run as a daemon, argus should send that message to
the syslog.
Looks to me that argus maybe writing management records, but not data records,
so lets see if we can get argus to read an active packet stream.
The error messages that you are seeing are very important, because they usually
mean that some aspect of argus has failed. Argus is a multi-threaded program,
and while its suppose to exit if any of its parts fail, the bug maybe that a part of
argus runs, even though most of it is dead. So, we need to fix the socket option
error, and the write() error.
Can you share your argus.conf file?
Carter
On Jan 30, 2013, at 10:02 AM, "Welland, Neal" <N.Welland at warwick.ac.uk> wrote:
> Hey Cater,
>
> I can see the daemon is running from a quick ps:
>
> root 28873 23841 0 14:30:39 pts/1 0:00 /opt/csw/bin/sudo /usr/local/sbin/argus -D8 -F /etc/argus.conf
>
> The file its configured to write to seems to be having its time stamp updated:
>
> $ ls -lrt /datapool/cachi_int-rz1
> total 16
> -rw-r--r-- 1 argus argus 7296 Jan 30 14:49 tap.argus
>
> But it's size suggests it's not seeing any interface traffic.
>
> The ra() client isn't installed on this host, and if I try and connect remotely, I get:
>
> $ ra -nS cachi
> ra[16833]: 01-30-13 14:53:56 ArgusReadConnection: 137.205.91.192 connection closed.
>
> Which produces the following logs on the server:
>
> Jan 30 14:53:05 cachi argus[28886]: [ID 444193 daemon.notice] connect from cookie-hostage.warwick.ac.uk with IP options (ignored): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Jan 30 14:53:05 cachi argus[28886]: [ID 899512 daemon.error] setsockopt IP_OPTIONS NULL: Option not supported by protocol
> Jan 30 14:53:05 cachi argus[28886]: [ID 811558 daemon.error] 30 Jan 13 14:53:05.649044 ArgusInitOutput: write(): No such file or directory
>
> I've mounted the partition containing tap.argus on a machine with ra() installed. Running $ ra -nr tap.argus produced NO output.
>
> Running $ racount -nr tap.argus
> racount records total_pkts src_pkts dst_pkts total_bytes src_bytes dst_bytes
> sum 60 0 0 0 0 0 0
>
> Which suggests time stamps are being written to the file, but nothing else.
>
> Regards, Neal.
>
> -----Original Message-----
> From: Carter Bullard [mailto:carter at qosient.com]
> Sent: 30 January 2013 14:01
> To: Welland, Neal
> Cc: argus-info at lists.andrew.cmu.edu
> Subject: Re: [ARGUS] Error Starting Argus Daemon
>
> Hey Neal,
> Try connecting to argus with ra() to see if its running.
> Argus doesn't log anything unless you subscribe to the data stream.
>
> Carter
>
> On Jan 30, 2013, at 8:56 AM, "Welland, Neal" <N.Welland at warwick.ac.uk> wrote:
>
>> Hey Carter,
>>
>> I modified argus.c as you suggested:
>>
>> # diff argus.c argus.c.orig
>> 583d582
>> < #ifdef ARGUS_NOT_DEFINED
>> 614d612
>> < #endif
>>
>> I recompiled and executed: $ /opt/csw/bin/sudo /usr/local/sbin/argus
>> -D8 -F /etc/argus.conf
>>
>> Which gave the following output:
>>
>> argus[27211.01000000]: 30 Jan 13 13:50:53.132830 ArgusCalloc (1, 1772)
>> returning 80c0f58
>> argus[27211.01000000]: 30 Jan 13 13:50:53.133228 ArgusNewModeler()
>> returning 80c0f58
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140150 ArgusCalloc (1,
>> 4227852) returning 80c3f18
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140193
>> ArgusNewSource(80c0f58) returning 80c3f18
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140216 ArgusCalloc (1, 196)
>> returning 84cc230
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140232 ArgusCalloc (1, 88)
>> returning 84cc300
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140244 ArgusNewQueue ()
>> returning 84cc300
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140266 ArgusCalloc (1, 80)
>> returning 84cc360
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140277 ArgusNewList ()
>> returning 84cc360
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140287 ArgusCalloc (1, 80)
>> returning 84cc3b8
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140300 ArgusNewList ()
>> returning 84cc3b8
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140310 ArgusNewOutput()
>> returning retn 84cc230
>> argus[27211.01000000]: 30 Jan 13 13:50:53.140342
>> setArgusMarReportInterval(60) returning
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152243 setArgusID(80c3f34,
>> 0x89cd5bc0) done
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152296 setArgusPortNum(561)
>> returning
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152318
>> clearArgusDevice(80c3f18) returning
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152330 ArgusCalloc (1, 80)
>> returning 84d1220
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152341 ArgusNewList ()
>> returning 84d1220
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152850 ArgusCalloc (1, 40)
>> returning 84d22c0
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152875 ArgusPushFrontList
>> (84d1220, 84d22c0, 1) returning 84d17c9
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152890 setArgusDevice(nxge0)
>> returning
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152915 ArgusDeleteList (0,
>> 2) returning
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152926 ArgusCalloc (1, 80)
>> returning 84d1278
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152937 ArgusNewList ()
>> returning 84d1278
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152947 ArgusCalloc (1, 12)
>> returning 84d1c30
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152958 ArgusPushFrontList
>> (84d1278, 84d1c30, 1) returning 804685f
>> argus[27211.01000000]: 30 Jan 13 13:50:53.152985
>> setArgusMarReportInterval(60) returning
>>
>> The daemon seems to be running now, but not actually logging.
>>
>> $ cat /var/adm/messages
>> Jan 30 13:50:53 cachi argus[27211]: [ID 139070 daemon.warning] 30 Jan
>> 13 13:50:53.157620 started
>>
>> Interestingly, if I didn't specifying -F, the daemon either shutdown or seemed to be in a loop looking for an interface.
>>
>> Where do we go from here?
>>
>> Regards, Neal.
>>
>> -----Original Message-----
>> From: Carter Bullard [mailto:carter at qosient.com]
>> Sent: 28 January 2013 22:16
>> To: Welland, Neal
>> Cc: argus-info at lists.andrew.cmu.edu
>> Subject: Re: [ARGUS] Error Starting Argus Daemon
>>
>> * PGP - S/MIME Signed by an unverified key: 28/01/2013 at 22:15:45
>>
>> Hey Neal,
>> Best I can do in interpreting the error, is that EPERM is being returned as the error code for the call to pthread_create(), which is an error with permissions on setting the
>> scheduling priority of the ArgusOutput thread.
>>
>> The scheduling assignments are done in argus.c, around line #605.
>> One way to get past this is to comment out all the scheduling logic, and see if we get farther. To do this, apply this patch:
>>
>> ==== //depot/argus/argus/argus/argus.c#90 -
>> /Volumes/Users/carter/argus/argus/argus/argus.c ====
>> 605a606
>>> #ifdef ARGUS_NOT_DEFINED
>> 635a637
>>> #endif
>>
>> which says add an ' #ifdef ARGUS_NOT_DEFINED ' to line 605 in your ./argus/argus.c, which will be false, and close the logic with an #endif on line 635. This basically comments out all references to pthread_attr_getschedpolicy() and pthread_attr_setschedpolicy() calls in the code.
>>
>> After you comment out this and compile, you should get farther?
>>
>> Hope all is most excellent, and sorry for the inconvenience,
>>
>> Carter
>>
>> On Jan 28, 2013, at 6:45 AM, "Welland, Neal" <N.Welland at warwick.ac.uk> wrote:
>>
>>> Good morning,
>>>
>>> Apologies for the delay. Had to get gdb installed!
>>>
>>> I eventually got argus to recompile with debugging symbols. I had to tweak bin/mkdep - it explicitly sets the PATH, but did not include the location of gcc on our installation!
>>>
>>> Anyway, here is the output from the gdb session:
>>>
>>> bash-3.2$ sudo /opt/csw/bin/gdb bin/argus
>>> Password:
>>> GNU gdb (GDB) 7.2
>>> Copyright (C) 2010 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later
>>> <http://gnu.org/licenses/gpl.html>
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
>>> and "show warranty" for details.
>>> This GDB was configured as "i386-pc-solaris2.10".
>>> For bug reporting instructions, please see:
>>> <http://www.gnu.org/software/gdb/bugs/>...
>>> Reading symbols from /var/tmp/argus-3.0.4/bin/argus...done.
>>> (gdb) break ArgusShutDown
>>> Breakpoint 1 at 0x805a927: file argus.c, line 810.
>>> (gdb) run -X
>>> Starting program: /var/tmp/argus-3.0.4/bin/argus -X [Thread debugging
>>> using libthread_db enabled] [New Thread 1 (LWP 1)]
>>> [New LWP 2 ]
>>> [Switching to Thread 1 (LWP 1)]
>>>
>>> Breakpoint 1, ArgusShutDown (sig=1) at argus.c:810
>>> 810 ArgusShutDownFlag++;
>>> (gdb) where
>>> #0 ArgusShutDown (sig=1) at argus.c:810
>>> #1 0x08085256 in ArgusLog (priority=3, fmt=0x809f62c
>>> "ArgusInitOutput() pthread_create error %s\n") at argus_util.c:1952
>>> #2 0x08075419 in ArgusInitOutput (output=0x84c9a68) at
>>> ArgusOutput.c:381
>>> #3 0x0805a766 in main (argc=2, argv=0x8047d40) at argus.c:614
>>> (gdb)
>>>
>>> I hope this helps pin-point the problem.
>>>
>>> Regards, Neal.
>>>
>>> -----Original Message-----
>>> From: Carter Bullard [mailto:carter at qosient.com]
>>> Sent: 25 January 2013 15:57
>>> To: Welland, Neal
>>> Cc: argus-info at lists.andrew.cmu.edu
>>> Subject: Re: [ARGUS] Error Starting Argus Daemon
>>>
>>>> Old - S/MIME Signed by an unverified key: 25/01/2013 at 15:57:22
>>>
>>> Hey Neal,
>>> That is much better, although still perplexing. The " ArgusShutDown(SIGHUP) ", is a problem, and figuring that out is the next step.
>>>
>>> Are you happy to run gdb() on this to see what is up? If so, compiling argus with the required debugging symbols, demands another recompile. In the argus distribution root directory:
>>>
>>> % make clobber
>>> % touch .devel
>>> % ./configure
>>> % make
>>>
>>> Then starting argus in gdb() and setting it up, involves:
>>>
>>> % sudo gdb bin/argus
>>> (gdb) break ArgusShutDown
>>> (gdb) run -X
>>> Starting program: /Volumes/Users/carter/argus/argus/bin/argus -X
>>>
>>> Breakpoint 1, ArgusShutDown (.....
>>> (gdb)
>>>
>>> When it breaks, I need to know where argus is, when it generates, the error, ..., so:
>>>
>>> (gdb) where
>>>
>>> If I can get the printout, that will help a great deal.
>>> Carter
>>>
>>> * Carter Bullard <carter at qosient.com>
>>> * Issuer: "VeriSign - Unverified
>>
>>
>> * Carter Bullard <carter at qosient.com>
>> * Issuer: "VeriSign - Unverified
>>
>>
>
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2589 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130131/fb2c92f8/attachment.bin>
More information about the argus
mailing list