**SPAM** Re: rasqlinsert issue
Carter Bullard
carter at qosient.com
Mon Jun 11 18:33:07 EDT 2012
Hey Peter,
The RaMySQLInit parameters would be parameters that were in the .rarc file,
so this just means that it will find its parameters from the command-line.
A few of these parameters (RaArchive and RaFormat) are future features
also, so no harm done.
I've seen where gdb() just doesn't want to break where you want it to,
and sometimes its been fixable, sometimes not. But getting the right
binary, compiled recently and sitting in the source, sometimes does
help to solve some of the weird issues.
But you are absolutely correct, depending on the version of Linux,
a sub-thread may not break, as gdb() is busy watching the parent
thread, and doesn't follow the execution of the children. This is
definitely true of forked processes, and some kernels have threads
that look more like processes than light-weight threads.
Carter
On Jun 11, 2012, at 3:43 PM, Peter Van Epp wrote:
> Noting that I haven't used rasql at all nor gdb with threads enabled
> as they appear to be here (or at least when last I tried years ago I was
> unsuccessful at making gdb work and removed the .threads and rebuilt :-)),
> I see two odd things in this:
>
>> rasqlinsert[19688.2087fef7ff7f0000]: 23:12:00.408815 RaMySQLInit ()
>> RaSource (null) RaArchive (null) RaFormat (null)
>
> this call seems to have all null arguments which may try and allocate a 0
> length buffer (or it may be perfectly normal :-)).
> Then a thread starts and immediately exits:
>
>> [New Thread 0x7ffff5a4b700 (LWP 19692)]
>> [Thread 0x7ffff5a4b700 (LWP 19692) exited]
>
> is it possible that gdb is looking at that thread and thus isn't tripping on
> the breakpoint? I remember having to break gdb at thread creation and do
> someting to switch to the new thread (which I was never successful at), but
> as noted thats long ago. It may be an idea to rebuild the clients without
> .threads defined and see if that helps gdb since it looks like the breakpoint
> isn't tripping for some reason.
>
> Peter Van Epp
>
> On Tue, Jun 12, 2012 at 12:46:01AM +0800, CS Lee wrote:
>> hi Carter,
>>
>> It seems to be the same -
>>
>> gdb /usr/local/stow/argusc-3.0.7.1-update/bin/rasqlinsert
>> GNU gdb (Ubuntu/Linaro 7.3-0ubuntu2) 7.3-2011.08
>> Copyright (C) 2011 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>>>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "x86_64-linux-gnu".
>> For bug reporting instructions, please see:
>> <http://bugs.launchpad.net/gdb-linaro/>...
>> Reading symbols from
>> /usr/local/stow/argusc-3.0.7.1-update/bin/rasqlinsert...done.
>> (gdb) break ArgusLog
>> Breakpoint 1 at 0x46f219: file ./argus_util.c, line 20377.
>> (gdb) run -D 1 -n -S 127.0.0.1:562 -Z b -w
>> mysql://root@localhost/argusdb/tbl_argus
>> -M cache -s stime ltime dur srcid flgs proto saddr sport dir daddr dport
>> pkts bytes appbytes state spkts dpkts sbytes dbytes sappbytes dappbytes
>> smaxsz dmaxsz sminsz dminsz smac dmac sco dco sas das - tcp
>> Starting program: /usr/local/stow/argusc-3.0.7.1-update/bin/rasqlinsert -D
>> 1 -n -S 127.0.0.1:562 -Z b -w mysql://root@localhost/argusdb/tbl_argus -M
>> cache -s stime ltime dur srcid flgs proto saddr sport dir daddr dport pkts
>> bytes appbytes state spkts dpkts sbytes dbytes sappbytes dappbytes smaxsz
>> dmaxsz sminsz dminsz smac dmac sco dco sas das - tcp
>> [Thread debugging using libthread_db enabled]
>> [New Thread 0x7ffff5a4b700 (LWP 19692)]
>> [Thread 0x7ffff5a4b700 (LWP 19692) exited]
>> rasqlinsert[19688.2087fef7ff7f0000]: 23:12:00.408707
>> ArgusCreateSQLSaveTable (tbl_argus) returning
>> rasqlinsert[19688.2087fef7ff7f0000]: 23:12:00.408815 RaMySQLInit ()
>> RaSource (null) RaArchive (null) RaFormat (null)
>> rasqlinsert[19688.2087fef7ff7f0000]: 23:12:00.409690 ArgusInitAddrtoname
>> (0xf7ea4010, 0x0, 0x0)
>> rasqlinsert[19688.2087fef7ff7f0000]: 23:12:00.409742 main: reading files
>> completed
>> rasqlinsert[19688.2087fef7ff7f0000]: 23:12:00.409806 Trying 127.0.0.1 port
>> 562 Expecting Argus records
>> rasqlinsert[19688.2087fef7ff7f0000]: 23:12:00.409894 connected
>> rasqlinsert[19688.2087fef7ff7f0000]: 23:12:00.409921 ArgusGetServerSocket
>> (0x7ffff7e42010) returning 8
>> rasqlinsert[19688]: 23:12:19.961357 ArgusMergeRecords: ArgusCalloc error
>> Success
>> rasqlinsert[19688.2087fef7ff7f0000]: 23:12:19.961510
>> ArgusShutDown (-1)
>> [Inferior 1 (process 19688) exited
>> normally]
>> (gdb) where
>> No stack.
>>
>>
>> On Tue, Jun 12, 2012 at 12:34 AM, Carter Bullard <carter at qosient.com> wrote:
>>
>>> Set the break point like this:
>>>
>>> (gdb) break ArgusLog
>>> Breakpoint 1 at 0x10006a892: file argus_util.c, line 20390.
>>>
>>> This sets the breakpoint at the first executable line in the routine.
>>>
>>> Carter
>>>
>>>
>>>
>>>
>>> On Jun 11, 2012, at 12:24 PM, CS Lee wrote:
>>>
>>> hi carter,
>>>
>>> Here's what I get if I set the breakpoint at ArgusLog
>>>
>>> gdb /usr/local/stow/argusc-3.0.7.1-update/bin/rasqlinsert
>>> GNU gdb (Ubuntu/Linaro 7.3-0ubuntu2) 7.3-2011.08
>>> Copyright (C) 2011 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later <
>>> http://gnu.org/licenses/gpl.html>
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
>>> and "show warranty" for details.
>>> This GDB was configured as "x86_64-linux-gnu".
>>> For bug reporting instructions, please see:
>>> <http://bugs.launchpad.net/gdb-linaro/>...
>>> Reading symbols from
>>> /usr/local/stow/argusc-3.0.7.1-update/bin/rasqlinsert...done.
>>> (gdb) l ArgusLog
>>> 20372
>>> 20373 extern char *print_time(struct timeval *);
>>> 20374
>>> 20375 void
>>> 20376 ArgusLog (int priority, char *fmt, ...)
>>> 20377 {
>>> 20378 va_list ap;
>>> 20379 char buf[MAXSTRLEN], *ptr = buf;
>>> 20380 struct timeval now;
>>> 20381 char *label = NULL;
>>> (gdb) b 20376
>>> Breakpoint 1 at 0x46f219: file ./argus_util.c, line 20376.
>>> (gdb) run -D 1 -n -S 127.0.0.1:562 -Z b -w
>>> mysql://root@localhost/argusdb/tbl_argus -M cache -s stime ltime dur
>>> srcid flgs proto saddr sport dir daddr dport pkts bytes appbytes state
>>> spkts dpkts sbytes dbytes sappbytes dappbytes smaxsz dmaxsz sminsz dminsz
>>> smac dmac sco dco sas das - tcp
>>> Starting program: /usr/local/stow/argusc-3.0.7.1-update/bin/rasqlinsert -D
>>> 1 -n -S 127.0.0.1:562 -Z b -w mysql://root@localhost/argusdb/tbl_argus -M
>>> cache -s stime ltime dur srcid flgs proto saddr sport dir daddr dport pkts
>>> bytes appbytes state spkts dpkts sbytes dbytes sappbytes dappbytes smaxsz
>>> dmaxsz sminsz dminsz smac dmac sco dco sas das - tcp
>>> [Thread debugging using libthread_db enabled]
>>> [New Thread 0x7ffff5a4b700 (LWP 19451)]
>>> [Thread 0x7ffff5a4b700 (LWP 19451) exited]
>>> rasqlinsert[19447.2087fef7ff7f0000]: 22:51:02.166069
>>> ArgusCreateSQLSaveTable (tbl_argus) returning
>>> rasqlinsert[19447.2087fef7ff7f0000]: 22:51:02.166170 RaMySQLInit ()
>>> RaSource (null) RaArchive (null) RaFormat (null)
>>> rasqlinsert[19447.2087fef7ff7f0000]: 22:51:02.167015 ArgusInitAddrtoname
>>> (0xf7ea4010, 0x0, 0x0)
>>> rasqlinsert[19447.2087fef7ff7f0000]: 22:51:02.167091 main: reading files
>>> completed
>>> rasqlinsert[19447.2087fef7ff7f0000]: 22:51:02.167150 Trying 127.0.0.1 port
>>> 562 Expecting Argus records
>>> rasqlinsert[19447.2087fef7ff7f0000]: 22:51:02.167238 connected
>>> rasqlinsert[19447.2087fef7ff7f0000]: 22:51:02.167317 ArgusGetServerSocket
>>> (0x7ffff7e42010) returning 8
>>>
>>> rasqlinsert[19447]: 22:51:17.968518 ArgusMergeRecords: ArgusCalloc error
>>> Success
>>> rasqlinsert[19447.2087fef7ff7f0000]: 22:51:17.968644 ArgusShutDown (-1)
>>> [Inferior 1 (process 19447) exited normally]
>>> (gdb)
>>> (gdb) where
>>> No stack.
>>> (gdb) c
>>> The program is not being run.
>>> (gdb)
>>>
>>>
>>> On Tue, Jun 12, 2012 at 12:09 AM, Carter Bullard <carter at qosient.com>wrote:
>>>
>>>> Hey CS Lee,
>>>> You failed to mention that in your rasqlinsert(), an ArgusCalloc in the
>>>> routine
>>>> ArgusMergeRecords is complaining. You need to not set the break points
>>>> you
>>>> are setting, and instead, set a breakpoint in ArgusLog. This is where
>>>> the error
>>>> message will be formatted and printed.
>>>>
>>>> So break in ArgusLog. That will catch the bug. When it breaks, type "
>>>> where ",
>>>> and if you're in the routine, ArgusMergeRecords(), then send that output
>>>> here.
>>>> If not, just type " c " to continue.
>>>>
>>>> I suspect that you are trying to allocate a buffer with 0 length.
>>>> If you can figure out what line in ArgusMergeRecords is the error, I can
>>>> fix quickly.
>>>>
>>>> Carter
>>>>
>>>>
>>>>
>>>>
>>>> On Jun 11, 2012, at 11:40 AM, CS Lee wrote:
>>>>
>>>> hi guys,
>>>>
>>>> Has anyone used radium with label file and having problem when using
>>>> rasqlinsert to log to database?
>>>>
>>>> Currently in my setup, if I don't enable this in radium.conf
>>>>
>>>> RADIUM_CLASSIFIER_FILE="/nsmon/etc/ralabel.conf"
>>>>
>>>> In ralabel.conf
>>>> RALABEL_ARIN_COUNTRY_CODES=yes
>>>> RA_DELEGATED_IP="/nsmon/junk/delegated-ipv4-latest"
>>>> RALABEL_GEOIP_ASN=yes
>>>> RALABEL_GEOIP_ASN_FILE="/nsmon/junk/GeoIPASNum.dat"
>>>>
>>>> Everything is running fine, however if I enable them then rasqlinsert
>>>> keeps exiting, so I run gdb as shown below -
>>>>
>>>> gdb /usr/local/stow/argusc-3.0.7.1-update/bin/rasqlinsert
>>>> GNU gdb (Ubuntu/Linaro 7.3-0ubuntu2) 7.3-2011.08
>>>> Copyright (C) 2011 Free Software Foundation, Inc.
>>>> License GPLv3+: GNU GPL version 3 or later <
>>>> http://gnu.org/licenses/gpl.html>
>>>> This is free software: you are free to change and redistribute it.
>>>> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
>>>> and "show warranty" for details.
>>>> This GDB was configured as "x86_64-linux-gnu".
>>>> For bug reporting instructions, please see:
>>>> <http://bugs.launchpad.net/gdb-linaro/>...
>>>> Reading symbols from
>>>> /usr/local/stow/argusc-3.0.7.1-update/bin/rasqlinsert...done.
>>>> (gdb) l ArgusMergeRecord
>>>> Function "ArgusMergeRecord" not defined.
>>>> (gdb) l ArgusMergeRecords
>>>> 7352 }
>>>> 7353
>>>> 7354
>>>> 7355 void
>>>> 7356 ArgusMergeRecords (struct ArgusAggregatorStruct *na, struct
>>>> ArgusRecordStruct *ns1, struct ArgusRecordStruct *ns2)
>>>> 7357 {
>>>> 7358 struct ArgusAgrStruct *agr = NULL;
>>>> 7359 double seconds;
>>>> 7360 int i;
>>>> 7361
>>>> (gdb) b 7356
>>>> Breakpoint 1 at 0x494389: file ./argus_client.c, line 7356.
>>>> (gdb) run -D 3 -Z b -n -S 127.0.0.1:562 -w
>>>> mysql://root@localhost/argusdb/tbl_argus -M cache -s stime ltime dur
>>>> srcid flgs proto saddr sport dir daddr dport pkts bytes appbytes state
>>>> spkts dpkts sbytes dbytes sappbytes dappbytes smaxsz dmaxsz sminsz dminsz
>>>> smac dmac sco dco sas das - tcp
>>>> Starting program: /usr/local/stow/argusc-3.0.7.1-update/bin/rasqlinsert
>>>> -D 3 -Z b -n -S 127.0.0.1:562 -w mysql://root@localhost/argusdb/tbl_argus-M cache -s stime ltime dur srcid flgs proto saddr sport dir daddr dport
>>>> pkts bytes appbytes state spkts dpkts sbytes dbytes sappbytes dappbytes
>>>> smaxsz dmaxsz sminsz dminsz smac dmac sco dco sas das - tcp
>>>> [Thread debugging using libthread_db enabled]
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.913931 ArgusAddHostList
>>>> (0xf7ea4010, 127.0.0.1:562, 1, 6) returning 1
>>>> rasqlinsert[19024.2087fef7ff7f0000]: 21:44:31.914602 Argusfinish_parse
>>>> (0xb9e6c0)
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.915390 ArgusFilterCompile
>>>> () done 0
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.915945
>>>> RaTopNewProcess(0xf7ea4010) returns 0xbb1270
>>>> [New Thread 0x7ffff5a4b700 (LWP 19025)]
>>>> [Thread 0x7ffff5a4b700 (LWP 19025) exited]
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.932558
>>>> ArgusCreateSQLSaveTable (tbl_argus) returning
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.932657 RaMySQLInit ()
>>>> RaSource (null) RaArchive (null) RaFormat (null)
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.933495 ArgusInitAddrtoname
>>>> (0xf7ea4010, 0x0, 0x0)
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.933547
>>>> ArgusParseInit(0xf7ea4010, NULL)
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.933580 main: reading files
>>>> completed
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.933628 Trying 127.0.0.1
>>>> port 562 Expecting Argus records
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.933705 connected
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.933731 ArgusGetServerSocket
>>>> (0x7ffff7e42010) returning 8
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.988647
>>>> ArgusReadConnection() read 16 bytes
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.988746
>>>> ArgusParseInit(0xf7ea4010 0xf7e42010
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:31.988781
>>>> ArgusWriteConnection(0xf7e42010, 0xfffbe100, 20) returning 20
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:32.130161
>>>> ArgusWriteConnection(0xf7e42010, 0xfffbe100, 7) returning 7
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:32.130207
>>>> ArgusReadConnection(0xf7e42010, 2) returning 1
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:32.130255
>>>> ArgusReadStream(0x7ffff7ea4010) starting
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:44:32.134189
>>>> ArgusProcessThisRecord () sql query SELECT record FROM tbl_argus WHERE
>>>> srcid="10.10.10.10" and proto="tcp" and saddr="192.168.5.10" and
>>>> sport="48073" and daddr="192.168.5.20" and dport="4307"
>>>>
>>>> Breakpoint 1, ArgusMergeRecords (na=0xbb0d80, ns1=0xbd43c0, ns2=0xbd4920)
>>>> at ./argus_client.c:7357
>>>> 7357 {
>>>> (gdb) where
>>>> #0 ArgusMergeRecords (na=0xbb0d80, ns1=0xbd43c0, ns2=0xbd4920) at
>>>> ./argus_client.c:7357
>>>> #1 0x0000000000415af5 in RaProcessThisRecord (parser=0x7ffff7ea4010,
>>>> ns=0x7ffff7e42620) at ./rasqlinsert.c:5467
>>>> #2 0x0000000000414574 in RaProcessRecord (parser=0x7ffff7ea4010,
>>>> ns=0x7ffff7e42620) at ./rasqlinsert.c:5119
>>>> #3 0x000000000043985a in RaScheduleRecord (parser=0x7ffff7ea4010,
>>>> argus=0x7ffff7e42620) at ./argus_util.c:2346
>>>> #4 0x0000000000439f2f in ArgusHandleDatum (parser=0x7ffff7ea4010,
>>>> input=0x7ffff7e42010, ptr=0x7ffff421b010, filter=0x7ffff7fc5708) at
>>>> ./argus_util.c:2457
>>>> #5 0x00000000004819f9 in ArgusReadStreamSocket (parser=0x7ffff7ea4010,
>>>> input=0x7ffff7e42010) at ./argus_client.c:426
>>>> #6 0x00000000004825a9 in ArgusReadStream (parser=0x7ffff7ea4010,
>>>> queue=0xb9d1e0) at ./argus_client.c:875
>>>> #7 0x00000000004218c9 in main (argc=46, argv=0x7fffffffe468) at
>>>> ./argus_main.c:360
>>>> (gdb) up
>>>> #1 0x0000000000415af5 in RaProcessThisRecord (parser=0x7ffff7ea4010,
>>>> ns=0x7ffff7e42620) at ./rasqlinsert.c:5467
>>>> 5467 ArgusMergeRecords
>>>> (ArgusParser->ArgusAggregator, pns, tns);
>>>> (gdb) l
>>>> 5462
>>>> 5463 pns->bins->status |= RA_DIRTYBINS;
>>>> 5464
>>>> 5465 } else {
>>>> 5466 if (parser->RaCumulativeMerge)
>>>> 5467 ArgusMergeRecords
>>>> (ArgusParser->ArgusAggregator, pns, tns);
>>>> 5468 else {
>>>> 5469 int i;
>>>> 5470 for (i = 0; i < ARGUSMAXDSRTYPE; i++) {
>>>> 5471 if (tns->dsrs[i] != NULL) {
>>>> (gdb) l
>>>> 5472 if (pns->dsrs[i] != NULL)
>>>> 5473 ArgusFree(pns->dsrs[i]);
>>>> 5474 pns->dsrs[i] = tns->dsrs[i];
>>>> 5475 tns->dsrs[i] = NULL;
>>>> 5476 }
>>>> 5477 }
>>>> 5478 }
>>>> 5479
>>>> 5480 ArgusDeleteRecordStruct(ArgusParser, tns);
>>>> 5481 pns->status |= ARGUS_RECORD_MODIFIED;
>>>> (gdb) b 5466
>>>> Breakpoint 2 at 0x415abd: file ./rasqlinsert.c, line 5466.
>>>> (gdb) c
>>>> Continuing.
>>>> rasqlinsert[19021.2087fef7ff7f0000]: 21:54:51.999805
>>>> ArgusProcessThisRecord () sql query SELECT record FROM tbl_argus WHERE
>>>> srcid="10.10.10.10" and proto="tcp" and saddr="192.168.50.10" and
>>>> sport="63073" and daddr="192.168.50.20" and dport="443"
>>>>
>>>> Breakpoint 2, RaProcessThisRecord (parser=0x7ffff7ea4010,
>>>> ns=0x7ffff7e42620) at ./rasqlinsert.c:5466
>>>> 5466 if (parser->RaCumulativeMerge)
>>>> (gdb) n
>>>> 5467 ArgusMergeRecords
>>>> (ArgusParser->ArgusAggregator, pns, tns);
>>>>
>>>> Carter, do let me know if I'm running gdb correctly so I can provide more
>>>> useful debugging output.
>>>>
>>>> --
>>>> Best Regards,
>>>>
>>>> CS Lee<geek00L[at]gmail.com>
>>>>
>>>> http://geek00l.blogspot.com
>>>> http://defcraft.net
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards,
>>>
>>> CS Lee<geek00L[at]gmail.com>
>>>
>>> http://geek00l.blogspot.com
>>> http://defcraft.net
>>>
>>>
>>>
>>
>>
>> --
>> Best Regards,
>>
>> CS Lee<geek00L[at]gmail.com>
>>
>> http://geek00l.blogspot.com
>> http://defcraft.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120611/92d1fe16/attachment.bin>
More information about the argus
mailing list