rasqlinsert -s -record doesn't seem to work

[Carter Bullard]

Hey Paul,
Good to see that I'm not going insane ;O)

There is lots of opportunity to do any number of strategies for databases.  I did the daily tables,
because they can be leveraged to do a lot of analytics,  and the logic was already implemented
in rasplit() and rastream() so it was very easy to do.

The biggest benefit is that its easy and fast to drop a table when you  want to roll data out of
the data based on the retention policy.  Not sure what it does to purge a days worth of data when
the single table is partitioned.

The logic we use for generating the new time based table can easily be used to drive whatever
mysql commands are needed to add a new partition to the existing table.
I suspect that we just need a command line configuration " token " to chose
which ever method is desired.

This would be very easy to do.

Sniff around.  You'll see that there is lots of work to do, but it may not be as bad
as you initially imagined.

Carter

On Oct 24, 2012, at 1:02 PM, Paul Schmehl <pschmehl_lists at tx.rr.com> wrote:

> --On October 24, 2012 9:35:16 AM -0400 Carter Bullard <carter at qosient.com> wrote:
> 
>> Hey Paul,
>> We've had this fix in rasqlinsert for quite sometime. Starting on line
>> 8410 of rasqlinsert():
>> 
>>      if ((MySQLVersionMajor > 4) || ((MySQLVersionMajor == 4) &&
>>                                      (MySQLVersionMinor >= 1)))
>>         sprintf (&sbuf[strlen(sbuf)], ") ENGINE=%s",
>> ArgusParser->MySQLDBEngine);       else
>>         sprintf (&sbuf[strlen(sbuf)], ") TYPE=%s",
>> ArgusParser->MySQLDBEngine);
>> 
>> Does your code not have this code ? What version of software are you
>> using?
>> 
> 
> 3.0.4.1 - clearly this port has a number of issues.  I'm working on an update to it now since the maintainer seems to have lost interest.
> 
> At least now I understand why it wasn't working.
> 
> Next question: why use daily tables instead of one table with daily partitioning?  Have you don't any testing between the two?  Searching multiple tables requires joins that will increase query time.  Searching one table partitioned by day should be much faster.
> 
> Of course it's damn hard to know what's best to do until I get a current working version of argus setup.  So, I'm working on the port update now and doing some speed testing on queries on the one 120 million row table.
> 
> -- 
> Paul Schmehl, Senior Infosec Analyst
> As if it wasn't already obvious, my opinions
> are my own and not those of my employer.
> *******************************************
> "It is as useless to argue with those who have
> renounced the use of reason as to administer
> medication to the dead." Thomas Jefferson
> "There are some ideas so wrong that only a very
> intelligent person could believe in them." George Orwell
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2589 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20121024/7989f4ae/attachment.bin>