Compiling argus with geoip dynamic shared object/ Visualizations

[Carter Bullard]

Hey Charles,
Remind me during the 3.0.3 cycle to be sure and add the changes you  
need for
GeoIP library searching.

Human vision is a very sophisticated sensor, and the only way to use  
is as a
part of a detection/response system is to present a visualization that  
is useful,
familiar, etc.....  Maybe a weird way of putting it, but the primary  
goal is to bring
the "human in the loop", regardless of whether you're supporting  
detection, tracking,
response, mitigation, resolution, whatever.

I personally like molecular modeling systems, as they are cool looking  
and the
provide massive utility for scientific analysis/research, education  
etc...  GIS systems
are very useful for lots of things, so I'm thinking we need to use  
both of these types
of visualization methodologies to get something useful

One of the problems with visualizations is that sometimes you can't  
get past the
image.  So the realtime visualization I'm working now, looks like an  
earth globe with pin
cushions and clouds.  I'll add links (probably the standard ballistic  
lines between
pin cushions) and have some flexibility on colors, shapes (maybe not  
pin cushions).
The goal is to have some simple geo-spatial orientation for the real  
information that
will be displayed for a demo at the SC'09 conference in Portland Ore  
in Nov.   We
need a geo-spatial reference for what we're going to measure.  The  
graphic also has
2D/3D strip charts, (pick any number of push-pins and look at the  
activity between them),
and I'm doing the frequency distributions this week for the real  
metrics that we're
going to demo.

The important thing here, is that the program is an Argus client, that  
can read real-time
data and display some aspects of it in a dynamic OpenGL  
visualization.  A lot of work to
do that.   In graphics terms, its just a mapping of address data  
transformed to geospatial
coordinates plotted onto a 3D quadric, with texture mapping and  
animation.  For smaller
sites, the quadric doesn't have to be a sphere.  A map of the campus  
would be useful,
and for wireless network monitoring/tracking real-time displays would  
be nice.

So, if we're interested in detection, what would be an interesting 3D  
thing to do?
I'm not worried so much about scan/worm like detection, as its pretty  
straight
forward.  I think we can do something like MERITs Flamingo very  
simply, and I might
have it ready by  Nov.

Is there something a little more meaty to do?

Carter

On Sep 26, 2009, at 11:53 AM, Charles Smutz wrote:

>
> Carter,
>
> What you proposed for ./configure options would be awsome. Something  
> akin to what you are doing for SASL would work for me, and for most  
> people I would assume. The maxmind backend for lookups is best for  
> me, even for things like ASN where you have other good alternatives,  
> because I already have infrastructure in place to keep the maxmind  
> dats up to date.
>
> What I'm currently doing woks for me so waiting for 3.0.3 doesn't  
> hurt me any.
>
> On the subject of visualizations,
>
> I looked at where you are heading with your visualization. When you  
> get it finished, I would likely use it.
>
> While these sort of visualizations definitely look cool, I struggle  
> to see their value for true analytics. If "situational awareness"  
> means having eye candy that maps abstract concepts back to the  
> "real" world, especially for management or other non-technical  
> people, then this sort of thing fits the bill. If you're looking to  
> show a big red line that is DoS, exfiltration, etc then maybe this  
> is a decent way of expressing it--at least when you can trace  
> attacks back to same geographic location as attacker. For this sort  
> of thing, the direction you are heading makes sense. The better it  
> looks, the better it is. I'm not sure why, but this type of audience  
> actually does like things like storms and clouds overlaid also.  
> Right or wrong, I've spent a fair amount of effort on this sort of  
> thing.
>
> I have a strong interest in visualizations that aid in analysis.  
> Predominately, I'm talking about visualizations that allow CND  
> analysts to detect new attacks, or even visualize current attacks,  
> not the sort of thing you would show to the same audience as above.
> I see a lot of stuff that attempts creative ways to view data, but I  
> still have a hard time applying it to useful analyis. I've read  
> Conti's stuff and played with things like RUMINT, and while there  
> are some interesting visualizations, I don't see how most help an  
> analyst do true analysis. Ditto for things like the IP address  
> cubes, geolocation, etc. I do see value in things like afterglow,  
> I2, et al but these are pretty limited in scope.
> I've seen parallel coordinate plots mentioned on this distro list. I  
> see great potential in using visualizations that are actually  
> intended to be used for analysis, like parallel coordinate plots/ 
> grand tour. The main limitation I've had in this is 1. graph  
> saturation, 2. inadequate features.
> 1. So many of us that use argus deal with lots of data. The obvious  
> data reduction mechanisms here are flow aggregation (racluster) and  
> data partitioning (one graph per protocol, sensor, etc). Has anyone  
> else found any others that work well?
> 2. As attacks move farther up the protocol stack, our detections  
> need to move up also. It's easy to think of cases where an attack is  
> a total mimicry it you are only looking at packet headers,  
> especially if attacker uses a third party's infrastructure. One  
> thing that I think would help with this sort of thing and would add  
> more features that would aid in data mining/visualization is if we  
> had the option to put some characterization of the payload data in  
> the flow record. I've discussed this with Carter privately, but I  
> think adding some sort of payload data characterization, similar to  
> what the ForNet guys were doing in the 2003-2005 timeframe would be  
> very useful. My own thoughts and studies on this are that simple  
> histograms of values in the payloads would allow one to derive a lot  
> of interesting statistics, would fit well in the argus flow model,  
> doesn't significantly compromise confidentiality, is computationally  
> feasible for for links up to say 1 Gbps, and would facilitate  
> detecting a lot of badness. This sort of visibility would be huge  
> compliment to traditional signature matching IDS which is impotent  
> against trivial encoding/obfuscations schemes. Again, I write this  
> from the viewpoint of someone trying to defend a single enterprise/ 
> campus network and who only really has visibility out to the edges  
> of that enterprise/campus, not someone trying to defend a huge/ 
> spawling worldwide network who also has significant network  
> visibility throughout the internet.
>
> Thanks again for the hard work on argus,
>
> Charles Smutz
>
>
>
>
>
>
>
>
>
>
>
>
>
> Carter Bullard wrote:
>> Hey Charles,
>> Thanks for the email!!!!   There are a lot of strategies that we  
>> can use in ./configure
>> to find a particular library.  The support I put in for GeoIP was  
>> fast and dirty.  I'll take
>> a look at making the support more generic, like the support we have  
>> for SASL.
>> That strategy will return shared first then static, and if you want  
>> to force static, there
>> is a ./configure switch for that.   './configure --help' will show  
>> the SASl options.
>>
>> Would that type of approach work for you?  I can try to put this in  
>> in a few weeks.
>> I've decided to release 3.0.2, and so this will be in the 3.0.3  
>> development thread.
>>
>> Thanks again!!!!!
>> Carter
>>
>>
>> On Sep 25, 2009, at 8:14 AM, Charles G. Smutz wrote:
>>
>>> Carter,
>>>
>>> Thanks for making such a great tool. The recent updates to the  
>>> argus website are also nice.
>>>
>>> I wanted to report a minor issue I had compiling argus with geoip  
>>> support and what I did to deal with it. I realize I’m really  
>>> getting nit picky but I figured providing feedback to this distro  
>>> might help next guy.
>>>
>>> The problem I encountered involves compiling argus clients against  
>>> the maxmind geoip dynamically loaded shared object. The current  
>>> configure script doesn’t support this directly, but it can still  
>>> be successfully built with a little hacking.
>>>
>>> I run argus on RHEL/centos 5 on x86_64. I have maxmind GeoIP  
>>> installed as dynamic shared object (.so) but not as a static  
>>> shared object (.a) as is common for packages for this platform  
>>> such as those found on EPEL (http://download.fedora.redhat.com/pub/epel/5/x86_64/repoview/geoip.html 
>>> ). Regardless, I prefer to compile argus with things like geoip  
>>> linked dynamically rather than statically linked.
>>>
>>> The current configure script (3.0.2) for argus-clients doesn’t  
>>> handle this case well. First of all, from what I can tell, it only  
>>> looks for the static object (GeoIP.a). Beyond that, I seem to  
>>> remember having some issues overriding the directory in which it  
>>> was searching even thought ld has no problem finding my GeoIP.so.  
>>> If I remember correctly, it was a “lib” vs. “lib64” issue combined  
>>> with a .h and .so in different directory issue.
>>>
>>> I overcame this problem my hacking my specfile such that it  
>>> circumvents the configure scripts’ geoip checks and just forces  
>>> geoip to be enabled directly. This is ugly for many reasons but  
>>> works for my deployment. More robust options for dynamic/static  
>>> shared objects and library paths would be nice. The following is  
>>> an excerpt from my specfile for argus-clients:
>>>
>>> export CFLAGS="$CFLAGS -DARGUS_GEOIP=1"
>>> export LIBS="-lGeoIP"
>>> %configure
>>>
>>> Hope this information is useful to you or to others.
>>>
>>> Thanks again for the great tool,
>>>
>>> Charles Smutz
>>>
>>>
>>
>> Carter Bullard
>> CEO/President
>> QoSient, LLC
>> 150 E 57th Street Suite 12D
>> New York, New York  10022
>>
>> +1 212 588-9133 Phone
>> +1 212 588-9134 Fax
>>
>>
>>
>

Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York  10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20091006/2a491ee6/attachment.bin>