RC.39 (and 40) 'argus' segfault on Fedora Core 6

Michael Hornung hornung at cac.washington.edu
Wed Feb 28 15:57:42 EST 2007


The box isn't acting unstable in any other ways, so I would put it lower 
on the list, but I haven't actually tested it with mtest86 or anything.  
I'll do that to be sure.

-Mike

On Wed, 28 Feb 2007 at 15:38, Carter Bullard wrote:

|Hey Michael,
|Oh yes, bad memory can cause all sorts of problems.
|since we're not getting this kind of instability in other places, that
|type of situation would be high on my list, but ....there can still
|be software problems.
|
|You should be able to see in your system log if the machine thinks
|there are memory errors?
|
|Carter
|
|
|
|
|Michael Hornung wrote:
|
|> I got a different segfault today:
|> 
|> Program received signal SIGSEGV, Segmentation fault.
|> 0x0805940f in ArgusCreateFlowKey (model=0x87af008, flow=0x87b0290,
|> hstruct=0x87b0200) at ArgusUtil.c:704
|> 704           hstruct->hash ^= *ptr++;
|> 
|> 
|> (gdb) where
|> #0  0x0805940f in ArgusCreateFlowKey (model=0x87af008, flow=0x87b0290,
|> hstruct=0x87b0200) at ArgusUtil.c:704
|> #1  0x0804e6af in ArgusProcessPacket (model=0x87af008, p=0x87b260a "",
|> length=90, tvp=0xbfc03298, type=0) at ArgusModeler.c:1004
|> #2  0x0805545d in ArgusEtherPacket (user=0xb7e2b008 "", h=0xbfc03298,
|> p=0x87b260a "") at ArgusSource.c:608
|> #3  0x08063e18 in pcap_read_linux ()
|> #4  0x0805718c in ArgusGetPackets (src=0xb7e2b008) at ArgusSource.c:1477
|> #5  0x0804b2eb in main (argc=1, argv=0xbfc03664) at argus.c:460
|> 
|> 
|> (gdb) print hstruct->hash
|> $1 = 4251857491
|> (gdb) print ptr
|> $2 = (unsigned int *) 0xc023000
|> (gdb) print *ptr
|> Cannot access memory at address 0xc023000
|> 
|> 
|> Could this be a result of bad physical memory in this box?
|> 
|> I captured traffic with tcpdump while running argus and will see if I can
|> reproduce the crash that way.
|> 
|> -Mike
|> 
|> On Wed, 28 Feb 2007 at 08:04, Michael Hornung wrote:
|> 
|> |(gdb) print ArgusMallocList
|> |$1 = (struct ArgusMemoryList *) 0x9e1d538
|> |(gdb) print ArgusMallocList->end
|> |$2 = (struct ArgusMemoryHeader *) 0x54a96bb8
|> |(gdb) print ArgusMallocList->end->nxt
|> |Cannot access memory at address 0x54a96bb8
|> |
|> |-Mike
|> |
|> |On Tue, 27 Feb 2007 at 23:30, Carter Bullard wrote:
|> |
|> ||This is very odd.  When you get to someplace in gdb, with a Segmentation
|> ||fault, try to find out what variable is having problems by printing the
|> actual
|> ||values:
|> ||
|> ||  (gdb) print ArgusMallocList
|> ||  (gdb) print ArgusMallocList->end
|> ||  (gdb) print ArgusMallocList->end->nxt
|> ||
|> ||I would guess that ArgusMallocList doesn't exist, or is corrupted.
|> ||This can happen for a number of reasons, but it may be useful to
|> ||try to get a packet trace that generates your errors.  Maybe a
|> ||bit of data, but if we can replicate the problem, we can fix it.
|> ||
|> ||Carter
|> ||
|> ||
|> ||
|> ||Michael Hornung wrote:
|> ||
|> ||> On Tue, 27 Feb 2007 at 14:04, Peter Van Epp wrote:
|> ||> ||> |touch .devel
|> ||> |touch .debug
|> ||> |./configure
|> ||> |make clean |make
|> ||> |
|> ||> |in the top argus directory it will compile with debug symbols which will
|> ||> |get interesting data if you type "where" at the gdb prompt.
|> ||> ||> ||> # gdb /usr/local/sbin/argus
|> ||> GNU gdb Red Hat Linux (6.5-15.fc6rh)
|> ||> ...
|> ||> (gdb) run 2>run.log
|> ||> Starting program: /usr/local/sbin/argus 2>run.log
|> ||> ||> Program received signal SIGSEGV, Segmentation fault.
|> ||> 0x08075df8 in ArgusFreeListRecord (buf=0xaa96df0) at argus_util.c:1362
|> ||> 1362                ArgusMallocList->end->nxt = mem;
|> ||> ||> ||> (gdb) where
|> ||> #0  0x08075df8 in ArgusFreeListRecord (buf=0xaa96df0) at
|> argus_util.c:1362
|> ||> #1  0x0805a744 in ArgusWriteOutSocket (output=0x9e1b2e0,
|> client=0x9e1b2f4)
|> ||>    at ArgusUtil.c:1281
|> ||> #2  0x0805be78 in ArgusOutputProcess (arg=0x9e1b2e0) at ArgusOutput.c:428
|> ||> #3  0x0804e983 in ArgusProcessPacket (model=0x9e1a008, p=0x9e1d442 "",
|> ||> length=1514, tvp=0xbfb8d768, type=-1) at ArgusModeler.c:1055
|> ||> #4  0x0805545d in ArgusEtherPacket (user=0xb7e59008 "", h=0xbfb8d768,
|> ||> p=0x9e1d442 "") at ArgusSource.c:608
|> ||> #5  0x08063e18 in pcap_read_linux ()
|> ||> #6  0x0805718c in ArgusGetPackets (src=0xb7e59008) at ArgusSource.c:1477
|> ||> #7  0x0804b2eb in main (argc=1, argv=0xbfb8db34) at argus.c:460
|> ||> ||> ||> (gdb) bt full
|> ||> #0  0x08075df8 in ArgusFreeListRecord (buf=0xaa96df0) at
|> argus_util.c:1362
|> ||>        mem = (struct ArgusMemoryHeader *) 0xaa96df0
|> ||>        rec = (struct ArgusRecordStruct *) 0xaa96df0
|> ||> #1  0x0805a744 in ArgusWriteOutSocket (output=0x9e1b2e0,
|> client=0x9e1b2f4)
|> ||>    at ArgusUtil.c:1281
|> ||>        asock = (struct ArgusSocketStruct *) 0xb4ffb50
|> ||>        list = (struct ArgusListStruct *) 0xa1e47d0
|> ||>        rec = (struct ArgusRecordStruct *) 0xaa96df0
|> ||>        retn = 276
|> ||>        count = 0
|> ||>        len = 276
|> ||>        ocnt = 11128989
|> ||>        statbuf = {st_dev = 836293388809535488, __pad1 = 39296,  __st_ino
|> =
|> ||> 165781512, st_mode = 165796208, st_nlink = 3216561320,  st_uid =
|> 165781512,
|> ||> st_gid = 0, st_rdev = 13815025949856902614,  __pad2 = 36845, st_size =
|> ||> 578914913796227081, st_blksize = 165781512,  st_blocks =
|> 1739248179131534,
|> ||> st_atim = {tv_sec = 30, tv_nsec = 0},  st_mtim = {tv_sec = 1172616364,
|> ||> tv_nsec = 165781512}, st_ctim = {
|> ||>    tv_sec = -1078405756, tv_nsec = -1078405928}, st_ino = 38789285994}
|> ||>        ptr = (unsigned char *) 0xb4ffb9c "\020 "
|> ||> #2  0x0805be78 in ArgusOutputProcess (arg=0x9e1b2e0) at ArgusOutput.c:428
|> ||>        arguswriterecord = 1
|> ||>        done = 0
|> ||>        rec = (struct ArgusRecordStruct *) 0xaa97028
|> ||>        output = (struct ArgusOutputStruct *) 0x9e1b2e0
|> ||>        ArgusUpDate = {tv_sec = 0, tv_usec = 500000}
|> ||>        ArgusNextUpdate = {tv_sec = 0, tv_usec = 500000}
|> ||>        i = 0
|> ||>        val = 0
|> ||>        count = 0
|> ||>        retn = (void *) 0x0
|> ||> #3  0x0804e983 in ArgusProcessPacket (model=0x9e1a008, p=0x9e1d442 "",
|> ||> length=1514, tvp=0xbfb8d768, type=-1) at ArgusModeler.c:1055
|> ||>        retn = 0
|> ||>        tflow = (struct ArgusSystemFlow *) 0x9e1b290
|> ||>        flow = (struct ArgusFlowStruct *) 0x9e22b40
|> ||>        nflow = (struct ArgusFlowStruct *) 0xdaa8c08
|> ||>        ptr = 0x9e1d468 "\b\002"
|> ||>        value = 0
|> ||> #4  0x0805545d in ArgusEtherPacket (user=0xb7e59008 "", h=0xbfb8d768,
|> ||> p=0x9e1d442 "") at ArgusSource.c:608
|> ||>        ep = (struct ether_header *) 0x9e1d442
|> ||>        ind = 0
|> ||>        src = (struct ArgusSourceStruct *) 0xb7e59008
|> ||>        tvp = (struct timeval *) 0xbfb8d768
|> ||>        caplen = 160
|> ||>        length = 1514
|> ||>        statbuf = {st_dev = 0, __pad1 = 0, __st_ino = 0, st_mode = 0,
|> ||> st_nlink = 10354372, st_uid = 3086764936, st_gid = 0,  st_rdev =
|> ||> 44261669504811007, __pad2 = 18120, st_size = -4631715752896591472,
|> ||> st_blksize = 10255072, st_blocks = -5189186049726920576, st_atim = {
|> ||>    tv_sec = 1, tv_nsec = 1}, st_mtim = {tv_sec = 0, tv_nsec = 134516346},
|> ||> st_ctim = {tv_sec = 0, tv_nsec = 134899988}, st_ino = 10354372}
|> ||> #5  0x08063e18 in pcap_read_linux ()
|> ||> No symbol table info available.
|> ||> #6  0x0805718c in ArgusGetPackets (src=0xb7e59008) at ArgusSource.c:1477
|> ||>        ArgusReadMask = {__fds_bits = {128, 0 <repeats 31 times>}}
|> ||>        ArgusWriteMask = {__fds_bits = {0 <repeats 32 times>}}
|> ||>        ArgusExceptMask = {__fds_bits = {0 <repeats 32 times>}}
|> ||>        tmp = 1
|> ||>        i = 0
|> ||>        width = 7
|> ||>        noerror = 1
|> ||>        fd = 7
|> ||>        found = 1
|> ||>        up = 1
|> ||>        wait = {tv_sec = 0, tv_usec = 20000}
|> ||> #7  0x0804b2eb in main (argc=1, argv=0xbfb8db34) at argus.c:460
|> ||>        commandlinew = 0
|> ||>        doconf = 0
|> ||>        dodebug = 0
|> ||>        i = 1
|> ||>        pid = 0
|> ||>        tmparg = 0x8049f30 "[\201��005"
|> ||>        filter = 0x0
|> ||>        statbuf = {st_dev = 64768, __pad1 = 0, __st_ino = 2688737,
|> st_mode =
|> ||> 33133, st_nlink = 1, st_uid = 500, st_gid = 500, st_rdev = 0,  __pad2 =
|> 0,
|> ||> st_size = 11071, st_blksize = 4096, st_blocks = 32, st_atim = {
|> ||>    tv_sec = 1172616251, tv_nsec = 0}, st_mtim = {tv_sec = 1172616251,
|> ||> tv_nsec = 0}, st_ctim = {tv_sec = 1172616251, tv_nsec = 0},  st_ino =
|> ||> 2688737}
|> ||>        host = (struct hostent *) 0x80a6720
|> ||>        commandlinei = 0
|> ||>        op = -1
|> ||>        path = "/etc/argus.conf\000argus", '\0' <repeats 8170 times>
|> ||> ||> ||> _____________________________________________________
|> ||> Michael Hornung          Computing & Communications
|> hornung at washington.edu
|> ||> University of Washington
|> ||> ||
|> ||
|> 
|
|


More information about the argus mailing list