rasqlinsert() discussion - default keys and schema

[Carter Bullard]

Gentle people,
There have been some issues with the default  key used by rasqlinsert(), specifically the error messages when the
print specification and the key definition don't agree.

I wanted to take the time to describe this issue, and then get some feedback on what to do.

rasqlinsert() is a program that takes in argus data, and populates a database table.  It automates the creation of
the database and the table,  and either "INSERT"s, "UPDATE"s or "DELETE"s data into the table, depending
on the data set, the keys and the mode of operation.

rasqlinsert() has a number of modes of operation.  In its default mode, rasqlinsert() acts
just like ratop();   its an argus data aggregator that keeps its in-memory data cache sync'd with a
mysql database table.  If rasqlinsert() adds a cache entry, it will schedule a database INSERT,
if the data is in the cache, and it is modified with new data coming in, rasqlinsert() will UPDATE
the data in the table, and if rasqlinsert() times out the cache entry, it will schedule a DELETE
of the row from the data table.

You have control of the schema (-s 'fields....'), the key (-m 'fields ....'), the type of table, the update
rate into the database table (.rarc RA_UPDATE_INTERVAL [default 0.453613 sec]), and the
timeout values.  There are idle timers only when there is a key defined, so if you are appending
data without a key (-m none") there are no timers.   rasqlinsert() uses the "-f racluster.conf"
aggregation configuration option, where you can specify timers on any flow type.

rasqlinsert(), when it starts, will create a data schema using the fields in the print specification
provided on the command line or in the .rarc file, and it will use the default key,
"srcid saddr daddr proto sport dport".   When rasqlinsert() starts up, it will check that all of the
key fields are in the print specification;  if not, it will complain, saying you need to add a field.

This has caused some confusion, as some don't know that rasqlinsert() is using a key, or what
that key is, or that the print specification needs to conform to the key, or that they have defined
a print specification, or that they need to supply one, etc....

The question is, "Should rasqlinsert() add to the print specification the needed fields automatically,
or should it stop and complain?"  "Should rasqlinsert(), printout a block of information every time its
run, stating its configuration, so there aren't any mysteries?"

I don't think that this particular issue is a big one, but I'm going to use it to start a discussion of the
use of rasqlinsert(), so its a good starting topic.    I am planning to document rasqlinsert() in the next
several weeks, on the email list, to get some of these issues out, and get some dialog going.

In doing so, many may discover that rasqlinsert() isn't doing what they expect.  Please if you are
using the mysql support, take a look at this series of emails, and comment on anything that you see.
That should help us to get a really good set of tools that can be used to do stuff.

Thanks for all the support !!!!!  And don't hesitate to chime in at any point !!!!!

Carter

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110516/d27efd6c/attachment.bin>