new client program and strategy

[Carter Bullard]

Gentle people,
I'm starting to add new capabilities to argus-clients-3.0.7.x and would like to get some dialog going.

The theme is more client processing, which to me means more analytics, better / faster searching,
indexing, collection, etc…. and alarm / alerting.  Now my list of priorities maybe different than yours,
but all of this is of course up for discussion and I'm always interested in what the group thinks.

When I think of more analytics / alarm / alerting, I tend to think of real-time processing
of streaming data, rather than batch processing data from files / archives.  And I think
about having to validate and debug analytics, which can be pretty complicated.

So I've been working on some tools that will allow me to look at a lot of argus data processing
on the fly, and to allow me to see different kinds of things.  Basically, I've been beefing up
ratop() to add color, and a lot of embedded analytics to improve knowing a bit more about
flows that its tracking.

I've included a screenshot of what will become the new ratop() (now its called racurses, as
I had to completely rewrite the curses part to get the color to work), and I know that it will
create more questions than answers, but here goes.  So, what you've got is a standard
ratop() screen, with whole flow and address painting, with search highlighting.

So, whole flow painting uses a large set of conditionals to decide what color the whole line
should be, in this case we're using an analytic that is tracking service availability.  The
flows that are failing because of no response are painted orange, all the rest are white.
So we've got two, an IPSec tunnel where the other endpoint has gone, and the other
is a TCP connection attempt from the outside of my firewall that didn't get through.

Once the flows are painted, then we paint the addresses.  Blue is the local host address,
light blue are local network addresses, and dark grey are multicast/broadcast addresses.
ratop() is looking at the local addresses to decide what is the local host and what is the
local network, but you can provide it with a configuration file of what you think is local.
These can be L2 or L3 addresses.

Then, lastly, I typed in " /imaps " , which causes ratop() to search for the literal string,
just like vi, and it highlights those strings in green.

So the concept is that if I can paint them, then I also so other things, like alarm / alert,
move them to another window, whatever.  The painting tells me what the analytic
decided.  Since you can run data through ratop() over and over again, you should be
able to use something like this as a debugger. 

Now the work that needs to be done to the new ratop() to make it useful includes
configuration, lots of embeded analytics and the ability to add analytics on the fly, so
testing new methods etc….. at least in the way that I'm thinking about use this.

Colors are now statically defined, but I have the beginnings of a styles configuration.
The number of objects, and they types of colors they can get can be staggering, so
I'm not sure about the configuration strategy at this point, but hopefully that will become
the topic of good discussions along the way.

OK, I'm going to put this out there in the next few weeks.  Please send some opinions,
comments, whatever, to the list or to me personally.  I look forward to working with you
on this type of stuff in the coming months, fi there is interest.

Hope all is most excellent,

Carter


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20121011/1835419c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: racursesSnapShot.tiff
Type: image/tiff
Size: 894802 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20121011/1835419c/attachment.tiff>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20121011/1835419c/attachment.bin>