getting lost among so many digits in large byte count numbers
Carter Bullard
carter at qosient.com
Thu Jan 19 19:10:38 EST 2012
Hey Kevin,
This is very cool !!!! How would you like for me to post it?
We can leave it in the email, as it will be captured in the mail archive,
or we can add a shell script that does this in a single command.
Very cool !!!!
I do apologize, and its my fault that I haven't done more in the documentation.
We do have this kind of support in the clients library, and the function you
provided is available in ratop.1. I will make some changes to make it
available to all the ra* programs, if you like.
I have envisioned ratop.1 as the client of choice to browse files and
to provide a real-time interface. Try your racluster command, but using
ratop.1 instead.
ratop -m saddr -r /argus/today/unt-08.arg
The idea is that ratop will read the data and provide a curses screen
that in real time will display the aggregated output. Its got a real-time
component to it, so you'll see the data accumulating, right before your eyes.
ratop.1 is like 'vi', in that you will have at the bottom of the screen
a status line. If you type '/', you go into search mode, and you can type
any string, then carriage return, and like 'vi', the cursor will bounce to that
string in the developing flow cache display that ratop.1 is printing. 'vi'
navigation commands like 'n', 'N' will go to the next or previous occurence
of the string…..
if you type ':' you will be in command mode and you can type options and
commands. Command 'h', will print out the help screen. Using the
':' command method, you can change the sorting algorithm on the fly
(command 's'), you can change the fields (command 'F'), and command 'H'
will turn abbreviations on and off.
At anytime type command 'H' while ratop is reading data, and most of the
numeric metrics, such as bytes, appbytes, packet counts, rates, loads, etc….
will be converted to the appropriate abbreviations. 'H' is a toggle, so you
can hit it as many time as you like to flip the abbreviations, and when you're
done, carriage return will put ratop.1 back into navigation mode.
I was reluctant to provide this as an option, as I need another letter to turn
it on, and we're running out of letters. Only rahisto() currently uses the -H option,
to provide histogram options. For ra.1, I can use the -H option to print values using
these abbreviations, if that will work for you.
Try ratop.1 to see if you like how I did it. The letters are right next to the numbers,
and the precision is specified using the -p option, so you can change the
numbers right of the decimal. If that works, I'll turn on the "-H" option to
do this for all ra* programs.
Carter
On Jan 19, 2012, at 5:59 PM, The Branches wrote:
> I'm humbly submitting the following one-liner "format-argus" script
> sed 's/$/ /;s/\([^0-9]\)\([0-9]\+\)\([0-9]\)[0-9]\{8\}\([^0-9]\)/ \1\2.\3 GB\4/g;s/\([^0-9]\)\([0-9]\+\)[0-9]\{6\}\([^0-9]\)/ \1\2 MB\3/g;s/\([^0-9]\)\([0-9]\+\)[0-9]\{3\}\([^0-9]\)/\1\2 KB\3/g'
> to the argus community for any others who like me often visually lose track of number place value when the digits start racking up beyond 8 or so. Casual order of magnitude errors caused by this have repeatedly messed up my analysis work so I hacked together something to help me see straight.
>
> By piping the ra- command output to format-argus, this:
>
> root at nids:~# racluster -m saddr -r /argus/today/unt-08.arg -w - | rasort -m bytes | head -n10
> 08:00:00.000000 eU F ip 174.25.157.3 <-> 0.0.0.0 7392139 5870710356 CON
> 08:00:00.000000 e F ip 205.145.81.2 <-> 0.0.0.0 358749 207782657 CON
> 08:00:00.996733 e ip 112.79.40.25 <-> 0.0.0.0 51213 42256964 CON
> 08:00:07.939546 e ip 68.202.49.96 <-> 0.0.0.0 63847 39415243 CON
> 08:00:49.225226 e ip 66.87.71.143 <-> 0.0.0.0 31941 31430729 CON
> 08:00:00.643207 eU ip 174.25.157.81 <-> 0.0.0.0 46676 27878467 CON
> 08:00:11.659829 e ip 99.6.241.61 <-> 0.0.0.0 31779 27756920 CON
> 08:34:16.485343 e ip 119.246.89.188 <-> 0.0.0.0 23186 22061798 CON
> 08:00:10.360478 e ip 13.177.127.226 <-> 0.0.0.0 20570 18453533 CON
> 08:01:02.796361 e ip 99.122.1.237 <-> 0.0.0.0 21647 17419334 CON
>
> ends up looking like this
>
> root at nids:~# racluster -m saddr -r /argus/today/unt-08.arg -w - | rasort -m bytes | head -n10 | format-argus
> 08:00:00.000 KB eU F ip 174.25.157.3 <-> 0.0.0.0 7 MB 5.8 GB CON
> 08:00:00.000 KB e F ip 205.145.81.2 <-> 0.0.0.0 358 KB 207 MB CON
> 08:00:00.996 KB e ip 112.79.40.25 <-> 0.0.0.0 51 KB 42 MB CON
> 08:00:07.939 KB e ip 68.202.49.96 <-> 0.0.0.0 63 KB 39 MB CON
> 08:00:49.225 KB e ip 66.87.71.143 <-> 0.0.0.0 31 KB 31 MB CON
> 08:00:00.643 KB eU ip 174.25.157.81 <-> 0.0.0.0 46 KB 27 MB CON
> 08:00:11.659 KB e ip 99.6.241.61 <-> 0.0.0.0 31 KB 27 MB CON
> 08:34:16.485 KB e ip 119.246.89.188 <-> 0.0.0.0 23 KB 22 MB CON
> 08:00:10.360 KB e ip 13.177.127.226 <-> 0.0.0.0 20 KB 18 MB CON
> 08:01:02.796 KB e ip 99.122.1.237 <-> 0.0.0.0 21 KB 17 MB CON
>
> It just condenses down large numbers to KB, MB, or GB values, chopping off anything after the decimal point, except for GB values from which I retain the first value after the decimal point. It isn't perfect in that it chops off digits without rounding, but it is good enough for my ad hoc analysis needs. If you have a script that does it better, I'd love to head about it.
>
> Kevin Branch
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120119/479b394f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120119/479b394f/attachment.bin>
More information about the argus
mailing list