database indexing support additions

[Carter Bullard]

Hey John,
So, rasqltimeindex() doesn't care if the file is sorted or not, and generates
correct byte offsets for each second in the file, regardless.  When the file is
not time sorted, however, the index may not necessarily provide an improvement
over simple linear search for the time based data.  In your case, however,
the index will still help alot, as your file will not be grossly out of order.

I understand about DB vs file.  I don't necessarily have a apriori requirement
for MySQL to run an argus archive.  So, we can implement this as a file,
rather than in a database table, but a database has a bit better searching
capabilities than a simple sequential file.

One thing we could do is prepend the file with this type of index, to help
in generic file data processing.  Maybe have a minute index rather than
a second index?

Carter

On Apr 9, 2010, at 2:22 PM, John Gerth wrote:

> Nice...a couple of obvious questions come to mind
> 
> First, does rasqlindex detect an unsorted archive file?
> I use a relatively large 60 second reporting interval
> and frequently see flows out of order, but in principle
> can't flows be reported out-of-order for any interval?
> 
> Although I've become a fan of databases, I'm still a
> belt-and-suspenders guy for data and like having the
> flat files as archive.  Given the simplicity of the
> time index table, I was wondering why you chose to
> only allow that as DB table since it would be
> straightforward to allow a flat file for that too.
> 
> /J
> 




-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100409/20b934b0/attachment.bin>