Help needed with rasqlinsert

[Carter Bullard]

Can't tell what's wrong if you don't tell us what you did.
Seems like you finally figured out how to insert data?

What command line options did you use?

Carter


On Jul 12, 2009, at 3:30 PM, Dave Edelman wrote:

> Significant progress (sort of)
>
> Figured out the debug had to touch .devel as well
>
> I dropped almost everything from the command now I have no index. I  
> still have no data. Increase to D 6 and it looks like I have no  
> source data
>
> Move to earlier data also radium from netflow but before I started  
> to use racluster no dice
>
> Moved back to real argus non netflow data and tons of data in the  
> table.
>
> Still no autoid but are we on to something?
>
> I can provide tons of data if you need it
>
> Dave
>
>
> Dave Edelman
> +1 917 331-0112 cell
>
> On Jul 12, 2009, at 11:33 AM, Carter Bullard <carter at qosient.com>  
> wrote:
>
>> Dave,
>> Need to turn on debug to see what is going on with why you don't  
>> have data.
>>
>> Use the "-D 4" option, assuming you turned on debugging when you  
>> configured
>> the clients package.
>>
>> Hmmm, if you want to just load data without aggregation, you need  
>> to use the
>> "-m none" option.  Without this, you are invoking the default  
>> aggregation key, which
>> contains the srcid, saddr, daddr, proto, sport and dport, and those  
>> keys are
>> propagated into the table.
>>
>> If you want an autoincrement  use the "-M autoid" as the usage  
>> statement sez to do.
>>
>> If you don't want the record (you really want the record, but  
>> you'll get to that
>> as you use it more), use the "-M norec" option.
>>
>> Send mail if you continue to have problems.
>>
>> Carter
>>
>>
>> On Jul 11, 2009, at 6:33 PM, Dave Edelman wrote:
>>
>>> I have MySql installed and working, and I have the latest version  
>>> of the argus clients created with the sql stuff so that I create  
>>> rasqlinsert.
>>>
>>> RaSqlInsert Version 3.0.2.beta.9
>>> usage: rasqlinsert
>>> usage: rasqlinsert [ra-options] [specific-options] [- filter- 
>>> expression]
>>>          -M <mode>          specify modes
>>>              autoid         use auto-increment for the record id
>>>              cache          use the database table contents as cache
>>>              nodrop         do not delete the table if it exists
>>>          -R <directory>     recursively process argus data files  
>>> in directory.
>>>
>>>          -r <dbUrl>         read argus data to mysql database.
>>>          -w <dbUrl>         write argus data to mysql database.
>>>                             Rasqlinsert will create the database  
>>> and table
>>>                             if they do not exist.
>>>
>>>                dbUrl:       mysql://[user[:pass]@]host[:port]/db/ 
>>> table
>>>
>>>          -s [-][+[]]field   specify fields to print.
>>>                fields:      record
>>>
>>>
>>> Starting without the database and table I utter the incantation
>>>
>>> rasqlinsert -r argus.2009.07.08.09.00.01.gz  -w mysql:// 
>>> root:XXXXX at localhost/argusData/flowData -M autoid -n \\
>>> -s saddr daddr sport dport proto pkts bytes srcid
>>>
>>> And MySQL tells me that I now am the proud possessor of both a  
>>> database and a table
>>>
>>> mysql> desc flowData;
>>> +--------+-------------+------+-----+---------+-------+
>>> | Field  | Type        | Null | Key | Default | Extra |
>>> +--------+-------------+------+-----+---------+-------+
>>> | saddr  | varchar(64) | NO   | PRI | NULL    |       |
>>> | daddr  | varchar(64) | NO   | PRI | NULL    |       |
>>> | sport  | varchar(10) | NO   | PRI | NULL    |       |
>>> | dport  | varchar(10) | NO   | PRI | NULL    |       |
>>> | proto  | varchar(16) | NO   | PRI | NULL    |       |
>>> | pkts   | bigint(20)  | YES  |     | NULL    |       |
>>> | bytes  | bigint(20)  | YES  |     | NULL    |       |
>>> | srcid  | varchar(64) | NO   | PRI |         |       |
>>> | record | blob        | YES  |     | NULL    |       |
>>> +--------+-------------+------+-----+---------+-------+
>>> 9 rows in set (0.00 sec)
>>>
>>>
>>> Two small(ish) problems, I really did want the primary key to be  
>>> an auto-increment value and I really did want a bunch of data in  
>>> the table.
>>>
>>> The file that I am reading is fine, I can use ra –r argus. 
>>> 2009.07.08.09.00.01.gz  and it spews forth all sorts of data,  
>>> MySQL tells me that the table is empty.
>>>
>>> The original source of the data is a radium instance that is  
>>> receiving NetFlow data from a Cisco router.
>>>
>>> What am I doing wrong? BTW, rasqlinsert told me that I needed to  
>>> include -s srcid because it was part of the primary key L
>>>
>>> As foolish as this may sound, I want to pop quite a bit of argus  
>>> data into a MySQL database to do a bit of mining.
>>>
>>> I already have made a few simple tweaks to some existing clients  
>>> so that they nicely feed RRDTOOL databases but that’s another story.
>>>
>>> Oh, and if anyone can tell me how to not include the blob, I would  
>>> be quite happy to not include it.
>>>
>>>
>>> --Dave
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090712/156c3feb/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090712/156c3feb/attachment.bin>