[Re: additional gdb info for argus/dag 10G crashes]
Aashish Sharma
aashish at ncsa.uiuc.edu
Sat Nov 11 11:34:27 EST 2006
Hi Carter :
Here is the output after following the steps you mentioned at the bottom of this email. I am still trying to get dag cards working with Argus.
[root at host argus]# gdb /usr/local/sbin/argus
GNU gdb Red Hat Linux (6.3.0.0-1.132.EL4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".
(gdb) set args -i dag0 -i dag1 -w as.o
(gdb) list 1454
Line number 1449 out of range; argus.c has 1175 lines.
(gdb) list ArgusGetPackets
1389
1390
1391
1392 void
1393 ArgusGetPackets (struct ArgusSourceStruct *src)
1394 {
1395 fd_set ArgusReadMask, ArgusWriteMask, ArgusExceptMask;
1396 int tmp, i, width = 0, noerror = 1, fd;
1397 struct timeval wait;
1398
(gdb) list 1454
1449 found++;
1450 src->ArgusThisIndex = i;
1451 switch (src->ArgusInterface[i].ArgusInterfaceType) {
1452 case ARGUSLIBPPKTFILE:
1453 if ((pcap_dispatch (src->ArgusInterface[i].ArgusPd, 1, src->ArgusInterface[i].ArgusCallBack, (u_char *)src)) < 0) {
1454 if (!(strncmp (pcap_geterr(src->ArgusInterface[i].ArgusPd), "recvfrom", 8))) {
1455 #ifdef ARGUSDEBUG
1456 // carter email 11/06/2006 ArgusDebug (3, "ArgusGetPackets: pcap_dispatch() returned %s\n", pcap_geterr(src->ArgusInterface[i].ArgusPd));
1457 ArgusLog (LOG_ERR, "ArgusGetPackets: interface %d: %s\n", i, pcap_geterr(src->ArgusInterface[i].ArgusPd));
1458
(gdb) break 1454
Breakpoint 1 at 0x8056b37: file ArgusSource.c, line 1454.
(gdb) run -i dag0 -i dag1 -w as.o -D3
Starting program: /usr/local/sbin/argus -i dag0 -i dag1 -w as.o -D3
argus[23933]: 11 Nov 06 10:15:08.463558 ArgusNewModeler() returning 0x9004008
argus[23933]: 11 Nov 06 10:15:08.463839 ArgusNewSource() returning 0xb7dcf008
argus[23933]: 11 Nov 06 10:15:08.463877 ArgusNewOutput() returning retn 0x9005930
argus[23933]: 11 Nov 06 10:15:08.463968 clearArgusDevice(0xb7dcf008) returning
argus[23933]: 11 Nov 06 10:15:08.464002 setArgusDevice(dag0) returning
argus[23933]: 11 Nov 06 10:15:08.464027 setArgusDevice(dag1) returning
argus[23933]: 11 Nov 06 10:15:08.464051 ArgusDeleteList (0x0, 2) returning
argus[23933]: 11 Nov 06 10:15:08.464087 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.622240 ArgusInitSource() pcap_open_live(dag1) returned 0x9005ba8
argus[23933]: 11 Nov 06 10:15:08.622373 Arguslookup_pcap_callback(1) returning ArgusEtherPacket(): 0x8054be7
argus[23933]: 11 Nov 06 10:15:08.775399 ArgusInitSource() pcap_open_live(dag0) returned 0x9005da8
argus[23933]: 11 Nov 06 10:15:08.775505 Arguslookup_pcap_callback(1) returning ArgusEtherPacket(): 0x8054be7
argus[23933]: 11 Nov 06 10:15:08.775627 ArgusInitSource() returning
argus[23933]: 11 Nov 06 10:15:08.775825 ArgusNewSocket (11) returning 0x9009360
argus[23933]: 11 Nov 06 10:15:08.804675 ArgusDeleteList (0x90053a0, 2) returning
argus[23933]: 11 Nov 06 10:15:08.804744 ArgusInitOutput() done
argus[23933]: 11 Nov 06 10:15:08.804786 started
argus[23933]: 11 Nov 06 10:15:08.805052 ArgusInitModeler() done
argus[23933]: 11 Nov 06 10:15:08.805271 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.805424 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.805467 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.805512 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.825844 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.825897 setArgusInterfaceStatus(1)
Breakpoint 1, ArgusGetPackets (src=0xb7dcf008) at ArgusSource.c:1454
1454 if (!(strncmp (pcap_geterr(src->ArgusInterface[i].ArgusPd), "recvfrom", 8))) {
(gdb) print pcap_geterr(src->ArgusInterface[i].ArgusPd)
$1 = 151018628
(gdb) bt
#0 ArgusGetPackets (src=0xb7dcf008) at ArgusSource.c:1454
#1 0x0804b8d4 in main (argc=8, argv=0xbfe0af54) at argus.c:462
(gdb) s[root at host argus]# gdb /usr/local/sbin/argus
GNU gdb Red Hat Linux (6.3.0.0-1.132.EL4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".
(gdb) set args -i dag0 -i dag1 -w as.o
(gdb) list 1454
Line number 1449 out of range; argus.c has 1175 lines.
(gdb) list ArgusGetPackets
1389
1390
1391
1392 void
1393 ArgusGetPackets (struct ArgusSourceStruct *src)
1394 {
1395 fd_set ArgusReadMask, ArgusWriteMask, ArgusExceptMask;
1396 int tmp, i, width = 0, noerror = 1, fd;
1397 struct timeval wait;
1398
(gdb) list 1454
1449 found++;
1450 src->ArgusThisIndex = i;
1451 switch (src->ArgusInterface[i].ArgusInterfaceType) {
1452 case ARGUSLIBPPKTFILE:
1453 if ((pcap_dispatch (src->ArgusInterface[i].ArgusPd, 1, src->ArgusInterface[i].ArgusCallBack, (u_char *)src)) < 0) {
1454 if (!(strncmp (pcap_geterr(src->ArgusInterface[i].ArgusPd), "recvfrom", 8))) {
1455 #ifdef ARGUSDEBUG
1456 // carter email 11/06/2006 ArgusDebug (3, "ArgusGetPackets: pcap_dispatch() returned %s\n", pcap_geterr(src->ArgusInterface[i].ArgusPd));
1457 ArgusLog (LOG_ERR, "ArgusGetPackets: interface %d: %s\n", i, pcap_geterr(src->ArgusInterface[i].ArgusPd));
1458
(gdb) break 1454
Breakpoint 1 at 0x8056b37: file ArgusSource.c, line 1454.
(gdb) run -i dag0 -i dag1 -w as.o -D3
Starting program: /usr/local/sbin/argus -i dag0 -i dag1 -w as.o -D3
argus[23933]: 11 Nov 06 10:15:08.463558 ArgusNewModeler() returning 0x9004008
argus[23933]: 11 Nov 06 10:15:08.463839 ArgusNewSource() returning 0xb7dcf008
argus[23933]: 11 Nov 06 10:15:08.463877 ArgusNewOutput() returning retn 0x9005930
argus[23933]: 11 Nov 06 10:15:08.463968 clearArgusDevice(0xb7dcf008) returning
argus[23933]: 11 Nov 06 10:15:08.464002 setArgusDevice(dag0) returning
argus[23933]: 11 Nov 06 10:15:08.464027 setArgusDevice(dag1) returning
argus[23933]: 11 Nov 06 10:15:08.464051 ArgusDeleteList (0x0, 2) returning
argus[23933]: 11 Nov 06 10:15:08.464087 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.622240 ArgusInitSource() pcap_open_live(dag1) returned 0x9005ba8
argus[23933]: 11 Nov 06 10:15:08.622373 Arguslookup_pcap_callback(1) returning ArgusEtherPacket(): 0x8054be7
argus[23933]: 11 Nov 06 10:15:08.775399 ArgusInitSource() pcap_open_live(dag0) returned 0x9005da8
argus[23933]: 11 Nov 06 10:15:08.775505 Arguslookup_pcap_callback(1) returning ArgusEtherPacket(): 0x8054be7
argus[23933]: 11 Nov 06 10:15:08.775627 ArgusInitSource() returning
argus[23933]: 11 Nov 06 10:15:08.775825 ArgusNewSocket (11) returning 0x9009360
argus[23933]: 11 Nov 06 10:15:08.804675 ArgusDeleteList (0x90053a0, 2) returning
argus[23933]: 11 Nov 06 10:15:08.804744 ArgusInitOutput() done
argus[23933]: 11 Nov 06 10:15:08.804786 started
argus[23933]: 11 Nov 06 10:15:08.805052 ArgusInitModeler() done
argus[23933]: 11 Nov 06 10:15:08.805271 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.805424 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.805467 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.805512 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.825844 setArgusInterfaceStatus(1)
argus[23933]: 11 Nov 06 10:15:08.825897 setArgusInterfaceStatus(1)
Breakpoint 1, ArgusGetPackets (src=0xb7dcf008) at ArgusSource.c:1454
1454 if (!(strncmp (pcap_geterr(src->ArgusInterface[i].ArgusPd), "recvfrom", 8))) {
(gdb) print pcap_geterr(src->ArgusInterface[i].ArgusPd)
$1 = 151018628
(gdb) bt
#0 ArgusGetPackets (src=0xb7dcf008) at ArgusSource.c:1454
#1 0x0804b8d4 in main (argc=8, argv=0xbfe0af54) at argus.c:462
(gdb)
So does that mean its libpcap which is culprit ?
Thanks a lot for your help,
Aashish Sharma
NCSA
On Mon, Nov 06, 2006 at 08:42:46AM -0500, Carter Bullard wrote:
>
> Hey Aashish,
>
> We do a lot of interface status checking, in argus, and I suspect that
>
> the dag's maybe either not happy with that, or its sending a signal
>
> that we aren't catching, and we're just exiting, when we could ignore
>
> it. Try this:
>
> (gdb) list ArgusGetPackets
>
> (gdb) list 1454
>
> find the line, in argus-3.0 it should be line 1454, and it looks like
> this:
>
> if (!(strncmp (pcap_geterr(src->ArgusInterface[i].ArgusPd),
> "recvfrom", 8))) {
>
> If this is the place then:
>
> (gdb) break 1454
>
> (gdb) run -i dag0 -i dag1 -w /dev/null
>
> and if it breaks there we'll know that its in the dag libpcap
> interface where
>
> its breaking down. The ArgusDebug () statement that prints out the
> pcap
>
> error triggers at debug level 3. replace these lines starting at line
> 1455:
>
> #ifdef ARGUSDEBUG
>
> ArgusDebug (3, "ArgusGetPackets: pcap_dispatch() returned %s\n",
> pcap_geterr(src->ArgusInterface[i].ArgusPd));
>
> #endif
>
> with:
>
> ArgusLog (LOG_ERR, "ArgusGetPackets: interface %d: %s\n", i,
> pcap_geterr(src->ArgusInterface[i].ArgusPd));
>
> recompile, and then let it rip to see what the dag has declared is the
> error.
>
> When it breaks the first time you can try to tease out the error
> message if you type:
>
> (gdb) print pcap_geterr(src->ArgusInterface[i].ArgusPd)
>
> but the stack maybe corrupted, it so, do the snip, paste and
> recompile, and try again!!!!!
>
> In the example you sent, you are hitting some arbitrary queue
> limits that are compiled into argus-2.0.6 to control its load. To get
> past them
----- End forwarded message -----
More information about the argus
mailing list