rasqlinsert only filling in about 1500 rows?

[Carter Bullard]

Hey Chance,
Still working on the manpage/documentation.
In double checking the issues in your email, I found a bug regarding  
the keys,
so grab argus-clients-3.0.2.tar.gz again, and use it as a starting  
point.

So the real question is what are you trying to accomplish?
If you want to just store the data straight from the sensor into the  
database, you
don't want to have a database key (this way you can have multiple  
records in
the database that reference the same flow).

    rasqlinsert -m none

If you want the database to contain a single row entry per flow, then  
you will want
rasqlinsert() to aggregate flow records based on an aggregation key  
and you will
want the database to use the same key strategy.  This is the default  
mode for all
argus record aggregators like racluster(), rabins(), ratop(), and now  
rasqlinsert().
The schema for the table should have the keys specified in its mysql()  
descriiption.

If you want to change the key, you use the "-m field field field"  
option to do so, but
make sure that the table you use has the same schema, if it already  
exists in the
database.  rasqlinsert() doesn't do schema verfication/validation yet.

If you don't want the entries to be deleted as they time out, you will  
need to add this option:
    "-M cache"

This basically sez use the database as the flow cache.

Once you get the new client code from the server, the way you are  
running rasqlinsert(), you
should expect the table to have a primary key that includes the  
standard 5-tuple fields plus the
srcid.  If you don't get that when you ask mysql() to describe the  
table, then you need to drop
the table and let rasqlinsert() recreate it.

Send email if you're still getting weird behavior after getting the  
new code.

Carter

On Oct 15, 2009, at 8:43 AM, Chance Carroll wrote:

> I didn't reply to the mailing list last time, so I'm sending it  
> again......
> Also, I'm running freeBSD 7.2
>
> On Thu, Oct 15, 2009 at 8:39 AM, Chance Carroll <carroll.chance at gmail.com 
> > wrote:
> I'm not purposely using any cache options, here is the command I'm  
> running:
> rasqlinsert -S localhost:561 -w mysql://root@localhost/argusData/argusTable 
>  -s "srcid proto saddr sport smac sbytes daddr dport dmac dbytes  
> bytes stime dur record" -d
>
> I'll be trying the auto ID option as well, for whatever reason when  
> I installed from the source it did not install the man pages, so  
> this is being a little tricky, I graped the html docs from the  
> source, but there was not a rasqlinsert man page...
>
> Thanks for the help!
> Chance
>
>
>
> On Wed, Oct 14, 2009 at 5:31 PM, Carter Bullard <carter at qosient.com>  
> wrote:
> Hey Chance,
> What are the command-line arguments that you are using?
>
> Sounds like you are running rasqlinsert() in a mode where the database
> table is sync'd with the internal cache state of rasqlinsert().   
> Means that when
> rasqlinsert() timesout a flow and deletes its internal cache entry,  
> it DELETES
> the entry in the database table as well.
>
> Are you using the "-M cache" option?
> Keys are specified using the "-m option".
> You can have rasqlinsert() generate an autoid field in the schema by  
> specifying
> it with a "-s +autoid" like command line option.
>
> Carter
>
>
> On Oct 14, 2009, at 3:54 PM, Chance Carroll wrote:
>
> I have Argus and radium setup and happily logging away our traffic,  
> but I'm having problems with rasqlinsert, when trying to capture  
> live data it starts replacing rows, the total row rarely rises above  
> 1700, and can decrease down to 1100. Also, the table does not show a  
> primary key, could that be part of the problem? Is it possible to  
> set an auto-incrementing key through rasqlinsert?
>
>
> Thanks,
> Chance
>
>
>
>
>
>
>

Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York  10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20091015/7b540209/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20091015/7b540209/attachment.bin>