FWD: RE: Argus, and moving 'live files'
Chris Newton
newton at unb.ca
Sat Mar 10 19:11:53 EST 2001
Ok, I didn't have argus compiled with debug turned on... but, I think I do
now. I couldn't really find any info on how to do it.. but, it looked like
from the configure script, it tested for a .debug. So, I 'touch'ed one of
those, and recompiled. The binaries were bigger than my last compile, so I
assume that worked.
That version of argus is running now... it might take a couple/few more
days for this to occur again. I'll report as soon as I see it happen.
Thanks for the help
Chris
>===== Original Message From <carter at qosient.com> =====
>If you compiled with debug support (~/.debug file present)
>then you should send some USR1 signals to the process
>with the median PID. Assuming that argus is pid 200, 201,
>and 202, send 4 USR1 messages to pid 201.
>
> # kill -USR1 201
> # kill -USR1 201
> # kill -USR1 201
> # kill -USR1 201
>
>And lets see what its saying for itself. If you need
>to, send as many as 8. If you don't get anything at
>all, and debug support was compiled in, then there is a
>really good problem to solve.
>
>If you have any gdb() experience, attach to process 201
>and step through to see where it thinks it is.
>
>I'm going out in just a few minutes, so I'll have to
>pay attention to this on Sunday.
>
>Carter
>
>Carter Bullard
>QoSient, LLC
>300 E. 56th Street, Suite 18K
>New York, New York 10022
>
>carter at qosient.com
>Phone +1 212 588-9133
>Fax +1 212 588-9134
>http://qosient.com
>
>> -----Original Message-----
>> From: Chris Newton [mailto:newton at unb.ca]
>> Sent: Saturday, March 10, 2001 6:39 PM
>> To: Carter Bullard; argus; Peter Van Epp
>> Subject: RE: FWD: RE: Argus, and moving 'live files'
>>
>>
>> Nope:
>>
>> [newton at epic flowlogs]$ ls -l argus-2001-03-10-18:14:50
>> -rw-r--r-- 1 root root 323624 Mar 10 18:14
>> argus-2001-03-10-18:14:50
>>
>> And, in fact, it seems a little small, compared to the
>> others from the same
>> time. So, maybe it isn't the moving of the files thats
>> causing the problem...
>> Here is an ls of the other files from that hour... right up
>> to when it
>> stopped outputing. So, either that was a slow minute.. or,
>> argus stopped
>> writing to that file part way through the minute.
>>
>>
>> [newton at epic flowlogs]$ ls -l argus-2001-03-10-18:*
>> -rw-r--r-- 1 root root 569956 Mar 10 18:00
>> argus-2001-03-10-18:00:49
>> -rw-r--r-- 1 root root 580148 Mar 10 18:01
>> argus-2001-03-10-18:01:49
>> -rw-r--r-- 1 root root 593456 Mar 10 18:02
>> argus-2001-03-10-18:02:49
>> -rw-r--r-- 1 root root 584564 Mar 10 18:03
>> argus-2001-03-10-18:03:49
>> -rw-r--r-- 1 root root 507604 Mar 10 18:04
>> argus-2001-03-10-18:04:49
>> -rw-r--r-- 1 root root 451776 Mar 10 18:05
>> argus-2001-03-10-18:05:49
>> -rw-r--r-- 1 root root 500492 Mar 10 18:06
>> argus-2001-03-10-18:06:49
>> -rw-r--r-- 1 root root 499104 Mar 10 18:07
>> argus-2001-03-10-18:07:50
>> -rw-r--r-- 1 root root 467036 Mar 10 18:08
>> argus-2001-03-10-18:08:50
>> -rw-r--r-- 1 root root 431164 Mar 10 18:09
>> argus-2001-03-10-18:09:50
>> -rw-r--r-- 1 root root 465376 Mar 10 18:10
>> argus-2001-03-10-18:10:50
>> -rw-r--r-- 1 root root 415984 Mar 10 18:11
>> argus-2001-03-10-18:11:50
>> -rw-r--r-- 1 root root 500576 Mar 10 18:12
>> argus-2001-03-10-18:12:50
>> -rw-r--r-- 1 root root 521964 Mar 10 18:13
>> argus-2001-03-10-18:13:50
>> -rw-r--r-- 1 root root 323624 Mar 10 18:14
>> argus-2001-03-10-18:14:50
>> [newton at epic flowlogs]$
>>
>>
>>
>> >===== Original Message From <carter at qosient.com> =====
>> >Is the file /usr/local/nva/flowlogs/argus-2001-03-10-18:14:50
>> >getting bigger?
>> >
>> >Carter
>> >
>> >Carter Bullard
>> >QoSient, LLC
>> >300 E. 56th Street, Suite 18K
>> >New York, New York 10022
>> >
>> >carter at qosient.com
>> >Phone +1 212 588-9133
>> >Fax +1 212 588-9134
>> >http://qosient.com
>> >
>> >> -----Original Message-----
>> >> From: Chris Newton [mailto:newton at unb.ca]
>> >> Sent: Saturday, March 10, 2001 6:22 PM
>> >> To: Carter Bullard; argus; Peter Van Epp
>> >> Subject: RE: FWD: RE: Argus, and moving 'live files'
>> >>
>> >>
>> >> Ok, happened again, tonight at 8:12pm.
>> >>
>> >> Here is the lsof output, below.... which is interesting,
>> >> because it shows
>> >> Argus having open the file
>> >> /usr/local/nva/flowlogs/argus-2001-03-10-18:14:50,
>> >> What is odd about that is that argus was started with this
>> >> command line:
>> >>
>> >> /usr/local/nva/bin/argus -d -F /usr/local/nva/conf/argus.conf -w
>> >> /usr/local/nva/flowlogs/argus.out
>> >>
>> >> And, I have a program called argproc that runs continually.
>> >> What it does is
>> >> moves the argus.out file to argus-{data/timestamp}, waits for
>> >> 1 minute (minus
>> >> the amount of time it took to move that file), and does it
>> >> again. Right now
>> >> it is complaining that the argus.out file doesnt exist when
>> >> it tries to do the
>> >> move.
>> >>
>> >> And, yup, all three argii are running:
>> >> [root at epic conf]# ps axfw |grep argus
>> >> 519 ? R 83:22 /usr/local/nva/bin/argus -d -F
>> >> /usr/local/nva/conf/argus.conf -w /usr/local/nva/flowlogs/argus.out
>> >> 522 ? S 1:59 \_ /usr/local/nva/bin/argus -d -F
>> >> /usr/local/nva/conf/argus.conf -w /usr/local/nva/flowlogs/argus.out
>> >> 523 ? S 3:16 \_ /usr/local/nva/bin/argus -d -F
>> >> /usr/local/nva/conf/argus.conf -w /usr/local/nva/flowlogs/argus.ou
>> >>
>> >>
>> >> Some output from top, showing at least one of the argii
>> >> consuming CPU, in
>> >> this case, about 4.1% CPU.
>> >>
>> >>
>> >> 519 root 14 0 12808 12M 628 S 0 4.0 10.1 83:29
>> >> /usr/local/nva/bin/argus -d -F /usr/local/nva/conf/argus.conf -w
>> >>
>> >> Operating system is redhat 6.2, upgraded to 2.4 kernel:
>> >>
>> >> [root at epic conf]# uname -a
>> >> Linux epic.csd.unb.ca 2.4.0-test12 #1 Sat Dec 16 23:51:30 AST
>> >> 2000 i686
>> >> unknown
>> >>
>> >> [root at epic conf]# /usr/sbin/lsof |grep argus
>> >> argus 519 root cwd DIR 3,1 4096 2 /
>> >> argus 519 root rtd DIR 3,1 4096 2 /
>> >> argus 519 root txt REG 3,1 551451 754834
>> >> /usr/local/nva/bin/argus
>> >> argus 519 root mem REG 3,1 340663 311343
>> >> /lib/ld-2.1.3.so
>> >> argus 519 root mem REG 3,1 527442 311361
>> >> /lib/libm-2.1.3.so
>> >> argus 519 root mem REG 3,1 4101324 311350
>> >> /lib/libc-2.1.3.so
>> >> argus 519 root mem REG 3,1 246652 311381
>> >> /lib/libnss_files-2.1.3.so
>> >> argus 519 root 0r REG 3,1 8105 754878
>> >> /usr/local/nva/conf/argus.conf
>> >> argus 519 root 3u sock 0,0
>> >> 1331 can't
>> >> identify protocol
>> >> argus 519 root 4r FIFO 0,0
>> 1337 pipe
>> >> argus 519 root 5w FIFO 0,0
>> 1337 pipe
>> >> argus 519 root 6w CHR 1,3
>> >> 180352 /dev/null
>> >> argus 522 root cwd DIR 3,1 4096 2 /
>> >> argus 522 root rtd DIR 3,1 4096 2 /
>> >> argus 522 root txt REG 3,1 551451 754834
>> >> /usr/local/nva/bin/argus
>> >> argus 522 root mem REG 3,1 340663 311343
>> >> /lib/ld-2.1.3.so
>> >> argus 522 root mem REG 3,1 527442 311361
>> >> /lib/libm-2.1.3.so
>> >> argus 522 root mem REG 3,1 4101324 311350
>> >> /lib/libc-2.1.3.so
>> >> argus 522 root mem REG 3,1 246652 311381
>> >> /lib/libnss_files-2.1.3.so
>> >> argus 522 root 0r REG 3,1 8105 754878
>> >> /usr/local/nva/conf/argus.conf
>> >> argus 522 root 1u CHR 5,1
>> >> 180385 /dev/console
>> >> argus 522 root 2u CHR 5,1
>> >> 180385 /dev/console
>> >> argus 522 root 3u sock 0,0
>> >> 1331 can't
>> >> identify protocol
>> >> argus 522 root 4r FIFO 0,0
>> 1337 pipe
>> >> argus 522 root 5w FIFO 0,0
>> 1337 pipe
>> >> argus 523 root cwd DIR 3,1 4096 2 /
>> >> argus 523 root rtd DIR 3,1 4096 2 /
>> >> argus 523 root txt REG 3,1 551451 754834
>> >> /usr/local/nva/bin/argus
>> >> argus 523 root mem REG 3,1 340663 311343
>> >> /lib/ld-2.1.3.so
>> >> argus 523 root mem REG 3,1 527442 311361
>> >> /lib/libm-2.1.3.so
>> >> argus 523 root mem REG 3,1 4101324 311350
>> >> /lib/libc-2.1.3.so
>> >> argus 523 root mem REG 3,1 246652 311381
>> >> /lib/libnss_files-2.1.3.so
>> >> argus 523 root 0r REG 3,1 8105 754878
>> >> /usr/local/nva/conf/argus.conf
>> >> argus 523 root 1u CHR 5,1
>> >> 180385 /dev/console
>> >> argus 523 root 2u CHR 5,1
>> >> 180385 /dev/console
>> >> argus 523 root 3u sock 0,0
>> >> 1331 can't
>> >> identify protocol
>> >> argus 523 root 4r FIFO 0,0
>> 1337 pipe
>> >> argus 523 root 5w FIFO 0,0
>> 1337 pipe
>> >> argus 523 root 6r FIFO 0,0
>> 1338 pipe
>> >> argus 523 root 7w FIFO 0,0
>> 1338 pipe
>> >> argus 523 root 8u REG 3,1 323624 821089
>> >> /usr/local/nva/flowlogs/argus-2001-03-10-18:14:50
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> >===== Original Message From <carter at qosient.com> =====
>> >> >Hey Chris,
>> >> >What others have seen, is that the second process that argus
>> >> >spawns, is either eating up a lot of CPU or none at all.
>> >> >This is the flow record multiplexor, and so if its not doing
>> >> >what its suppose to do, then nothing is going to come out of
>> >> >the argus.
>> >> >
>> >> >There are several debugging strategies to find out what is
>> >> >going on. The first is to do a simple ps() to make sure that all
>> >> >the processes are there. In the case of writing out to a file,
>> >> >you should have at least 3 argus processes running all the time.
>> >> >If you do have 3 processes, you can use gdb to attach to each
>> >> >running process, and then step through them for a few
>> >> >instructions to see what they are doing.
>> >> >
>> >> >Another strategy is to turn debug support on for each process.
>> >> >If you've compiled in debug support, then you can send SIGUSR1
>> >> >signals to any argus process to turn on its debug reporting.
>> >> >So as an example, assuming that the 3 processes are 200, 201
>> >> >and 202:
>> >> >
>> >> > # kill -USR1 202
>> >> >
>> >> >will turn on debug reporting and set the debug level to one.
>> >> >Sending another SIGUSR1 will increment the debug level. To
>> >> >turn it off, send a SIGUSR2 to the process.
>> >> >
>> >> > # kill -USR2 202
>> >> >
>> >> >So you can test them all, by getting their debug level to 3 or
>> >> >4 and see what they think is going on.
>> >> >
>> >> >Carter
>> >> >
>> >> >Carter Bullard
>> >> >QoSient, LLC
>> >> >300 E. 56th Street, Suite 18K
>> >> >New York, New York 10022
>> >> >
>> >> >carter at qosient.com
>> >> >Phone +1 212 588-9133
>> >> >Fax +1 212 588-9134
>> >> >http://qosient.com
>> >> >
>> >> >> -----Original Message-----
>> >> >> From: Chris Newton [mailto:newton at unb.ca]
>> >> >> Sent: Thursday, March 08, 2001 1:33 PM
>> >> >> To: Carter Bullard; argus; Peter Van Epp
>> >> >> Subject: RE: FWD: RE: Argus, and moving 'live files'
>> >> >>
>> >> >>
>> >> >> >===== Original Message From <carter at qosient.com> =====
>> >> >> >Hey Guys,
>> >> >> > Chris, more than likely your problem doesn't have anything
>> >> >> >to do with the file moving itself. If Argus breaks, you will
>> >> >> >see that your file moving strategy will suddenly stop, as
>> >> >> >there won't be a file to move any more. So the file moving
>> >> >> >makes the problem much more apparent.
>> >> >>
>> >> >> Thats whats happening. I get errors from my script that
>> >> >> the 'argus-output'
>> >> >> file does not exist, and therefore, can't be moved. Argus is
>> >> >> still running
>> >> >> happily though.
>> >> >>
>> >> >> It happens out of the blue (the couple of times it has
>> >> >> happened). The
>> >> >> moving script runs happily along.. then, boom... errors, 'no
>> >> >> such file'. I
>> >> >> check, sure enough, Argus isn't recreating the new
>> >> >> 'argus-output' file
>> >> >> anymore. Kill restart argus, everything returns to normal.
>> >> >>
>> >> >> Chris
>> >> >>
>> >>
>> >> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
>> >>
>> >> Chris Newton, Systems Analyst
>> >> Computing Services, University of New Brunswick
>> >> newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)
>> >>
>> >>
>>
>> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
>>
>> Chris Newton, Systems Analyst
>> Computing Services, University of New Brunswick
>> newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)
>>
>>
_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
Chris Newton, Systems Analyst
Computing Services, University of New Brunswick
newton at unb.ca 506-447-3212(voice) 506-453-3590(fax)
More information about the argus
mailing list