Database design concerns

[Carter Bullard]

Hey Mark,
I'm very interested in doing more client efforts, and making it available.
We have a few efforts that either are doing a GUI, or have a GUI, or something
that looks like it could be a GUI, but they are doing lots of different things.

Rather that describing a GUI and developing it, I've been very silent on this
topic, because there is so much one can do, and GUIs are pretty demanding
efforts, if you do it right.

If we can come up with some specific goals, I will be very happy to get involved
and make a go at an official Argus GUI.

The project needs archive establishment, management and ops and status
for the complete chain of sensors to archive.  I imagine that we need an analytics
manager for starting, and maintaining streaming analytics as well as periodic
archive analytic management.  

But we have projects like Isis, from Stanford, and Tiet that have GUIs or images
that are interesting, and the obvious social network presentations, like AfterGlow
did, that would make the GUI powerful.  We have the idea's coming from Gloriad,
where they have an Earth with ballistic arc's for activity, and there are forensics
analytics like the Cube.  Lots of things to think about.

Carter


On Oct 26, 2012, at 8:42 PM, Mark Bartlett <mabartle at gmail.com> wrote:

> Hey Paul,
> 
> I wrote a web gui for argus in php before Carter had added the db component to argus (I had created my own mysql db and was inserting batches of argus data).  I would be more than willing to offer up what I have done and you can change the select statements to match the db schema which argus resides now.  Let me know if you are interested.
> 
> Bartlett
> 
> On Oct 26, 2012 6:05 PM, "Paul Schmehl" <pschmehl_lists at tx.rr.com> wrote:
> --On October 26, 2012 3:11:25 PM -0400 Carter Bullard <carter at qosient.com> wrote:
> 
> Hey Paul,
> No problem at all !!!!!  If there is a problem, its that we don't have
> enough
> documentation on this topic for you to read.  I was excited that you
> brought
> up the topics that you did, as I'm hoping that it will generate some
> thoughts
> on the " first pass " support we put into argus-3.0.4+.
> 
> 
> I think the best thing for you to do is to keep doing what you're doing,
> and
> if there is anything I can help you with, I'm happy to do so.  I still
> think you're
> onto something with the partitions, but I don't know enough about them to
> know if they are a distraction, or something useful.
> 
> 
> Here's some basics I've learned over the past 72 hours.  Partitions break up a table into lots of mini-tables.  They don't recommend having more than 50 partitions on a table.  Partitions make sense if the bulk of the selects will only search one partition.  In the case of argus data this would make sense because a lot of queries are done against time frames less than 24 hours in size.  Even if a search crossed daily boundaries, it would usually only require searching two partitions.
> 
> However, it sounds like your approach has already solved the problem of searching massive amounts of data quickly, so I'm not sure partitions would improve performance.  In fact they may be more trouble than they're worth.
> 
> My problem is I don't yet understand what you're doing and how all the various utilities you've written tie into the greater picture.
> 
> The other issue I have is that our department is expanding, and some of the newer analysts don't have the same level of expertise that I do with unix commandline apps.  So I was looking for a way to "webify" argus data and searches so that the analysts can use an interface they're more familiar with than bash.
> 
> I just don't want you to think that argus is half done.
> 
> You and I both know that many open source apps leave much to be desired. You often have to cobble together several different apps to create something worthwhile for daily use by graphics-demanding younger folks.
> 
> I realize now, after your explanations, that is not the case with argus. I'm going to have to pore through the docs to figure out how I can do what you're doing without bugging the daylights out of you.  If I can put a web front-end on it, so much the better.
> 
> I'm thinking its getting done, but you never know how people like their 
> software cooked ;o)
> 
> 
> There's often a dichotomy between what the users want and what software gives them.  I frequently wonder why some software is so counter intuitive and why features you would think would be there aren't.  I think the gap is at least partly explained by the different thought patterns of people who design software and those who use it.
> 
> Paul Schmehl, Senior Infosec Analyst
> As if it wasn't already obvious, my opinions
> are my own and not those of my employer.
> *******************************************
> "It is as useless to argue with those who have
> renounced the use of reason as to administer
> medication to the dead." Thomas Jefferson
> "There are some ideas so wrong that only a very
> intelligent person could believe in them." George Orwell
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20121031/40cea042/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2589 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20121031/40cea042/attachment.bin>