rasqlinsert disconnect issues

Wed Oct 8 07:28:32 EDT 2014

This is what happened:

rasqlinsert[8240.90db20b7]: Fri 2014-10-03 00:01:36.947 ArgusSQLQuery (INSERT INTO argus.testmacAddrs_2014_10_03 (stime,ltime,srcid,saddr,smac,record) VALUES ("1412294417.397","1412294417.397","10.25.246.7","150.110.95.41","00:00:00:00:00:01",...))
rasqlinsert[8240.90db20b7]: Fri 2014-10-03 00:01:36.947 ArgusMySQLInsertProcess: residual buffer Count 3 SQL Query len 1203
rasqlinsert[8240.908b90b4]: Fri 2014-10-03 00:01:36.948 ArgusScheduleSQLQuery (0xb7f75008, 0x9a65240, 0x9b91240, INSERT INTO argus.testmacAddrs_2014_10_03 (stime,ltime,srcid,saddr,smac,record) VALUES ("1412294415.621","1412294417.621","10.25.246.7","161.161.0.123","00:00:00:00:00:02",...), 32) done
rasqlinsert[8240.90abd0b5]: Fri 2014-10-03 00:01:36.948 ArgusSQLQuery (UPDATE argus.testmacAddrs_2014_10_03 SET stime="1412294403.322",ltime="1412294415.318",record="..." WHERE srcid="10.25.246.7" and saddr="192.58.128.30" and smac="00:00:00:00:00:03")
rasqlinsert[8240.90db20b7]: 1412294496.949 ArgusSQLQuery (INSERT INTO argus.testmacAddrs_2014_10_03 (stime,ltime,srcid,saddr,smac,record) VALUES ("1412294415.621","1412294417.621","10.25.246.7","161.161.0.123","00:00:00:00:00:02",...))
rasqlinsert[8240.90db20b7]: Fri 2014-10-03 00:01:36.949 ArgusMySQLInsertProcess: residual buffer Count 1 SQL Query len 472
rasqlinsert[8240.908b90b4]: Fri 2014-10-03 00:01:36.950 ArgusScheduleSQLQuery (0xb7f75008, 0x9a65240, 0x9b80130, INSERT INTO argus.testmacAddrs_2014_10_03 (stime,ltime,srcid,saddr,smac,record) VALUES ("1412294414.885","1412294417.877","10.25.246.7","161.161.0.55","00:00:00:00:00:01",...), 32) done
rasqlinsert[8240.90db20b7]: Fri 2014-10-03 00:01:36.950 ArgusSQLQuery (INSERT INTO argus.testmacAddrs_2014_10_03 (stime,ltime,srcid,saddr,smac,record) VALUES ("1412294414.885","1412294417.877","10.25.246.7","161.161.0.55","00:00:00:00:00:01",...))
rasqlinsert[8240.90db20b7]: Fri 2014-10-03 00:01:36.950 ArgusMySQLInsertProcess: residual buffer Count 1 SQL Query len 613
rasqlinsert[8240.90abd0b5]: Fri 2014-10-03 00:01:36.950 ArgusSQLQuery (UPDATE argus.testmacAddrs_2014_10_03 SET stime="1412294403.322",ltime="1412294415.318",record="..." WHERE srcid="10.25.246.7" and saddr="10.55.202.32" and smac="00:00:00:00:00:02")
rasqlinsert[8240.908b90b4]: Fri 2014-10-03 00:01:36.950 ArgusProcessData: flushed 86 records
Segmentation fault

I’m running it under gdb to see if I can get a better handle on the context of the fault.

—Dave

On Oct 1, 2014, at 11:34 PM, Carter Bullard <carter at qosient.com> wrote:

> D3 should print the sql calls, which maybe all that you need to see.
> Carter
> 
> On Oct 1, 2014, at 9:46 PM, David Edelman <dedelman at iname.com> wrote:
> 
>> Carter,
>> 
>> I’m running rasqlinsert using –S localhost:561 to process the output of radium which is doing labeling. I’ll fire up an instance of the release code with the same parameter but a different table name and see if there is anything obvious. I’ve built the release code with both .debug and .devel do you have a recommendation for a debug value?
>> 
>> —Dave
>> 
>> From: Carter Bullard <carter at qosient.com>
>> Date: Monday, September 29, 2014 at 1:51 PM
>> To: David Edelman <dedelman at iname.com>
>> Cc: "John T. Myers" <myersj0 at gmail.com>, Argus <argus-info at lists.andrew.cmu.edu>
>> Subject: Re: [ARGUS] rasqlinsert disconnect issues
>> 
>> Hey Dave,
>> Well, that’s not what we’re striving for, so
>> if we can capture what that is all about, I’ll
>> fix as soon as I can.
>> 
>> Carter
>> 
>> On Sep 28, 2014, at 5:47 PM, David Edelman <dedelman at iname.com> wrote:
>> 
>>> The running but seems to no update the database was a problem that I reported and you fixed in one of the very last release candidates. The current 3.0.8 does not have that problem as best I can tell but it does stop after between 8-20 hours with no log messages that I have been able to find.
>>>  
>>> --Dave
>>>  
>>> From: Carter Bullard [mailto:carter at qosient.com] 
>>> Sent: Sunday, September 28, 2014 11:34 AM
>>> To: David Edelman
>>> Cc: John T. Myers; Argus
>>> Subject: Re: [ARGUS] rasqlinsert disconnect issues
>>>  
>>> Well the earlier rasqlinsert.1 code does have a problem that the new one tried
>>> to fix.  I’ll try to see what may be up with mysql_real_query() error messages,
>>> and see if we missed something there.
>>>  
>>> But in the OP, the problem is that rasqlinsert.1 is running, but not updating
>>> the database, in this second report, rasqlinsert.1 is failing ??
>>>  
>>> Carter
>>>  
>>>  
>>> On Sep 25, 2014, at 8:11 PM, David Edelman <dedelman at iname.com> wrote:
>>> 
>>> 
>>>> Carter,
>>>>  
>>>> I am seeing something similar with Netflow data processed by Radium (labels added) and rasqlinsert reading the processed radium data from port 561. In my case the rasqlinsert process dies without any error messages (it is built with .debug and .devel) It isn’t practical for me to enable debuigger output since the failure can be many hours into the run.
>>>>  
>>>> The one clue that I do have it that the release candidate set (3.0.8-rc1) worked fine.
>>>>  
>>>> I haven’t had time to do much more than go back to the working release.
>>>>  
>>>> --Dave
>>>>  
>>>> From: argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu [mailto:argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu] On Behalf Of Carter Bullard
>>>> Sent: Thursday, September 25, 2014 1:34 PM
>>>> To: John T. Myers
>>>> Cc: Argus
>>>> Subject: Re: [ARGUS] rasqlinsert disconnect issues
>>>>  
>>>> Hey John,
>>>> rasqlinsert() is multi-threaded, and its possible that
>>>> the cache concurrency thread, the one that is managing the
>>>> database cache, exited if the database fails.  
>>>>  
>>>> rasqlinsert() should close, if that thread is done.
>>>>  
>>>> So you are seeing rasqlinsert() is still running, but
>>>> not updating the database ??  Is rasqlinsert() getting
>>>> bigger ???  (should be generating INSERT and UPDATE 
>>>> requests, but the DB thread is not processing them ???)
>>>>  
>>>> Carter
>>>>  
>>>>  
>>>> On Sep 25, 2014, at 10:35 AM, John T. Myers <myersj0 at gmail.com> wrote:
>>>> 
>>>> 
>>>> 
>>>>> Hello,
>>>>>  
>>>>> I was wondering if it’s normal behavior for rasqlinsert to cease inserting netflow into the database if the connection becomes interrupted? It seems if the mysql database is restarted or any part of the connection between rasqlinsert and the db is broken, it will not attempt to re-connect and continue flow insertion.
>>>>>  
>>>>> Thanks!
>>>>> John
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20141008/7722fabe/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4136 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20141008/7722fabe/attachment.bin>