argus-clients-3.0.2.tar.gz with mysql support
Pablo J. Rebollo-Sosa
Pablo.Rebollo at ece.uprm.edu
Wed Mar 4 18:42:58 EST 2009
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
isCarter,
I ran the the tests with racluster and the outputs are the same.
root at argus:~# racluster -r argus.2009.03.03.17.00.00 -M rmon -m saddr -w
- - | racount
racount records total_pkts src_pkts dst_pkts
total_bytes src_bytes dst_bytes
sum 2334 336945 193790 143155
276763462 239874647 36888815
root at argus:~# racluster -r argus.2009.03.03.17.00.00 -M rmon -m saddr -w
- - | ra -w - | racount
racount records total_pkts src_pkts dst_pkts
total_bytes src_bytes dst_bytes
sum 2334 336945 193790 143155
276763462 239874647 36888815
I still thinking that there is a problem. For example I tried the
following.
root at nsm:~# racluster -r argus.2009.03.03.17.00.00 -M rmon -m saddr -w
- - | rasort -m bytes -w - | ra -N 5
StartTime Flgs Proto SrcAddr Sport Dir
DstAddr Dport TotPkts TotBytes State
17:00:02.870130 e ip 74.125.0.163 <->
0.0.0.0 25377 27162034 CON
17:00:43.147390 e ip 68.142.122.70 <->
0.0.0.0 23169 21108846 CON
17:00:02.635304 e ip 63.251.219.114 <->
0.0.0.0 21325 19133350 CON
17:00:02.075566 e ip 66.90.64.10 <->
0.0.0.0 19430 18250118 CON
17:01:13.549650 e ip 74.125.0.214 <->
0.0.0.0 15158 14123330 CON
- From the previous output the maximum total bytes is 27162034. But if the
"- net 136.145.34.0/24" filter is applied I get a bigger value for total
bytes (686322024).
root at nsm:~# racluster -r argus.2009.03.03.17.00.00 -M rmon -m saddr -w
- - - net 136.145.34.0/24 | rasort -m bytes -w - | ra -N 5
StartTime Flgs Proto SrcAddr Sport Dir
DstAddr Dport TotPkts TotBytes State
17:00:00.584726 e ip 136.145.34.11 <->
0.0.0.0 669192 686322024 CON
17:00:00.584726 e ip 96.7.19.16 <->
0.0.0.0 669185 686321500 CON
17:00:00.099711 eU F ip 136.145.34.80 <->
0.0.0.0 71461 50620443 CON
17:00:43.147390 e ip 68.142.122.70 <->
0.0.0.0 23140 21093742 CON
17:00:01.903740 e ip 136.145.34.81 <->
0.0.0.0 23002 21043963 CON
Any clues?
Best regards,
Pablo J. Rebollo
carter at qosient.com wrote:
> Use racount instead of wc, but I suspect a problem.
> So instead of " | wc -l"
> Try " -w - | racount"
>
> Racluster maybe leaving a few records in its output when it closes terminates. I'll take a look today!!
>
> Carter
>
> Sent from my Verizon Wireless BlackBerry
>
> -----Original Message-----
> From: "Pablo J. Rebollo-Sosa" <Pablo.Rebollo at ece.uprm.edu>
>
> Date: Tue, 03 Mar 2009 17:52:56
> To: Carter Bullard<carter at qosient.com>
> Cc: Argus<argus-info at lists.andrew.cmu.edu>
> Subject: Re: [ARGUS] argus-clients-3.0.2.tar.gz with mysql support
>
>
> Dear Carter,
>
> I'm testing the new clients and noticed odd results with ra. When using
> racluster with a specific file I get certain amount of lines
>
> server# racluster -r argus.2009.03.03.17.00.00 -M rmon -m saddr | wc -l
> 5555
>
> The problem is when using racluster with -w option with ra. When
> running the command I get fewer amount of lines.
>
> racluster -r argus.2009.03.03.17.00.00 -M rmon -m saddr -w - | ra -r -
> | wc -l
> 2334
>
> Any suggestions?
>
> Best regards,
>
> Pablo J. Rebollo
>
> Carter Bullard wrote:
>> Gentle people,
>> First pass at the new argus-clients distribution is on the dev server.
>> ftp:/qosient.com/dev/argus-3.0/argus-clients-3.0.2.tar.gz
>
>> First pass because there will be modifications before its released,
>> as the user data analysis programs still need a little tweak.
>
>> This version addresses many problems, particularly those
>> relating to backward compatibility to argus-2.x streams.
>> I have not had a chance to directly test the changes on
>> some of the bugs on the list but I suspect that this version
>> should fix those backward compatibility bugs.
>
>> If you try the code, and it doesn't have your issue fixed,
>> please, please, please, send email, so that I can get those
>> issues dealt with.
>
>> I am pleased to say that the database programs, rasqlinsert()
>> and rasql() are mostly ready to go. I don't have a manpage yet,
>> so hopefully the "-h" option will give you guidance.
>
>> I will be sending out sometime this week detail on the use of
>> rasqlinsert(), the format of the database url that is needed to
>> access database data, and the concepts of rasql() and why
>> its needed.
>
>> If you want to give rasqlinsert a run, like loading tables from
>> files, try these types of commands:
>
>> rasqlinsert -r file -w mysql://user@host/db/table -m none
>
>> This will load the table 'db.table' with the records, and the
>> fields will be those that you would expect to be printed if
>> you had run ra against the file. To modify the schema, just
>> use the "-s field" command.
>
>> The "-m none" removes any keys that rasqlinsert() may have
>> wanted to use based on your .rarc file, so MySQL won't
>> complain about DUPLICATE inserts into the table.
>
>> If you then run these programs:
>
>> rasql -r mysql://user@host/db/table
>
>> or
>
>> rasqlinsert -r mysql://user@host/db/table
>
>> rasqlinsert() will look like ratop(), but its data will come from
>> the MySQL tables.
>
>> rasqlinsert pokes the actual binary record into the database,
>> along with ascii representations of the attributes. This is
>> so programs like rasql() can get argus records, rather
>> than ascii text out of the database. If you want to get rid
>> of the binary BLOBs, use "-s -record". rasql(), when reading
>> this type of table, will just return, without any data.
>
>> A set of programs I use a lot are:
>
>> rabins -S localhost -M time 30s -B 5s -w - | \
>> rasqlinsert -r - -w mysql://user@host/ratop/flowTable -m none
>
>> This reads data from a live stream, holds it for 30s, aggregating
>> common records together, and then pokes it into the database
>> table. This table will grow forever with argus records, but you can
>> see how something very simple like this can be the base of
>> a large flow system.
>
>> Hope all is most excellent, and thanks for all the help!!!!
>
>> Carter
>
>> Carter Bullard
>> CEO/President
>> QoSient, LLC
>> 150 E 57th Street Suite 12D
>> New York, New York 10022
>
>> +1 212 588-9133 Phone
>> +1 212 588-9134 Fax
>
>
>
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAkmvEf4ACgkQxjU5UYZ6K6dQoACeNpfxh3+lTKklNVz3YDc8fxoN
wdAAnipgJNsG9E31PEjX0766lTjlNnPI
=w/oi
-----END PGP SIGNATURE-----
More information about the argus
mailing list