argus-clients-3.0.2.tar.gz with mysql support

[carter at qosient.com]

Hey Pablo,
You can't go back to earlier versions.
Maybe I should number it 3.1.0?
Can racluster 3.0.2 read the file?

Carter


Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: "Pablo J. Rebollo-Sosa" <Pablo.Rebollo at ece.uprm.edu>

Date: Tue, 03 Mar 2009 18:22:01 
To: Carter Bullard<carter at qosient.com>
Cc: Argus<argus-info at lists.andrew.cmu.edu>
Subject: Re: [ARGUS] argus-clients-3.0.2.tar.gz with mysql support


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Carter,

There is an output error when using racluster 3.0.0 with the same file
generated by rasplit (3.0.2).

racluster -r argus.2009.03.03.17.00.00 -M rmon -m saddr | head

racluster[5789]: 18:18:56.066547 ArgusReadStreamSocket (0xb7bcca84)
record length is zero

Regards,

Pablo J. Rebollo

Pablo J. Rebollo-Sosa wrote:
> Dear Carter,
> 
> I'm testing the new clients and noticed odd results with ra.  When using
> racluster with a specific file I get certain amount of lines
> 
> server# racluster -r argus.2009.03.03.17.00.00 -M rmon -m saddr | wc -l
> 5555
> 
> The problem is when using racluster with -w option with ra.  When
> running the command I get fewer amount of lines.
> 
> racluster -r argus.2009.03.03.17.00.00 -M rmon -m saddr -w - | ra -r -
> | wc -l
> 2334
> 
> Any suggestions?
> 
> Best regards,
> 
> Pablo J. Rebollo
> 
> Carter Bullard wrote:
>> Gentle people,
>> First pass at the new argus-clients distribution is on the dev server.
>>    ftp:/qosient.com/dev/argus-3.0/argus-clients-3.0.2.tar.gz
> 
>> First pass because there will be modifications before its released,
>> as the user data analysis programs still need a little tweak.
> 
>> This version addresses many problems, particularly those
>> relating to backward compatibility to argus-2.x streams.
>> I have not had a chance to directly test the changes on
>> some of the bugs on the list but I suspect that this version
>> should fix those backward compatibility bugs.
> 
>> If you try the code, and it doesn't have your issue fixed,
>> please, please, please, send email, so that I can get those
>> issues dealt with.
> 
>> I am pleased to say that the database programs, rasqlinsert()
>> and rasql() are mostly ready to go.   I don't have a manpage yet,
>> so hopefully the "-h" option will give you guidance.
> 
>> I will be sending out sometime this week detail on the use of
>> rasqlinsert(), the format of the database url that is needed to
>> access database data, and the concepts of rasql() and why
>> its needed.
> 
>> If you want to give rasqlinsert a run, like loading tables from
>> files, try these types of commands:
> 
>>    rasqlinsert -r file -w mysql://user@host/db/table -m none
> 
>> This will load the table 'db.table' with the records, and the
>> fields will be those that you would expect to be printed if
>> you had run ra against the file.  To modify the schema, just
>> use the "-s field" command.
> 
>> The "-m none" removes any keys that rasqlinsert() may have
>> wanted to use based on your .rarc file, so MySQL won't
>> complain about DUPLICATE inserts into the table.
> 
>> If you then run these programs:
> 
>>    rasql -r mysql://user@host/db/table
> 
>>        or
> 
>>    rasqlinsert -r mysql://user@host/db/table
> 
>> rasqlinsert() will look like ratop(), but its data will come from
>> the MySQL tables.
> 
>> rasqlinsert pokes the actual binary record into the database,
>> along with ascii representations of the attributes.  This is
>> so programs like rasql() can get argus records, rather
>> than ascii text out of the database.  If you want to get rid
>> of the binary BLOBs, use "-s -record".  rasql(), when reading
>> this type of table, will just return, without any data.
> 
>> A set of programs I use a lot are:
> 
>>    rabins -S localhost -M time 30s -B 5s -w - | \
>>       rasqlinsert -r - -w mysql://user@host/ratop/flowTable -m none
> 
>> This reads data from a live stream, holds it for 30s, aggregating
>> common records together, and then pokes it into the database
>> table.  This table will grow forever with argus records, but you can
>> see how something very simple like this can be the base of
>> a large flow system.
> 
>> Hope all is most excellent, and thanks for all the help!!!!
> 
>> Carter
> 
>> Carter Bullard
>> CEO/President
>> QoSient, LLC
>> 150 E 57th Street Suite 12D
>> New York, New York  10022
> 
>> +1 212 588-9133 Phone
>> +1 212 588-9134 Fax
> 
> 
> 
> 
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkmtrYYACgkQxjU5UYZ6K6dQ5wCggIMKU8diqDOebCQPEkxeSX/r
rzsAnAuPNGmYN4UnFW4C0oH7fPMDNEXj
=c2AA
-----END PGP SIGNATURE-----