radark discovery detector description

[Carter Bullard]

Gentle people,
Here is a brief description of the radark discovery effort, of which
radark.pl will be the implementation.  The script is going to be,
or could be, the focus of a long winded discussion on network
discovery detection.  Some people on the list have developed
their own tools to do some of this stuff, and I want them to know
that I am not trying to compete with their efforts (i'm hoping they
will contribute to the discussion).  This is an effort to try to  
describe
aspects of the argus tool set, and what you can do with it.

Radark is a snipit of some of the research work that I've done
in attack detection.   To get right into it, one of the biggest issues
in this area is discovery detection.  All security events are preceded
by some form of discovery.  Can't attack something if you don't know
that its there (and if you don't have discovery, maybe its an insider
attack).  Being able to detect discovery is a way of getting
a heads up on trouble.   There are 2 fundamental types of discovery.
Active and passive (from the perspective of the discoverer). The  
security
market has started to abuse these terms, so it maybe better to
call them stimulated and non-stimulated discovery strategies, but I'll
stick with active and passive for the moment.

Traditional active discovery techniques fall into the challenge
categories.  Actively transmit/stimulate a blackbox network and see
what comes out.  Argus provides many simple tools to cover the basics
in active discovery detection (ie port and address scanning).  The 2
primary techniques, such as snort's IP address/sec threshold and
Threshold Random Walk strategies can be implemented with the
existing tools that are in argus-3.0.  Some tools, like rahosts.pl
and raports.pl, are provided to specifically support these types of
threshold strategies, and general programs like rabins() and
racluster() have the flexibility to provide direct support as well.

These tools, when run periodically, can provide scan detection windows
in the months/year range, which overcome the limitations of other
simple tools.  I will describe how to use these tools in another
submission later this week/early next week, if there is interest.

Threshold based scan detection is notoriously faulty.  Scan a little
slower, is all you have to do to get around snort and others, and
you will never know that they are all over your network.  What I'm
proposing in radark() is not rocket science, just a more practical
approach.  The idea is to track all activity in the dark space of a  
site,
the dark IP addresses, and once an external IP address is identified
as attempting to access the dark space, go back through the complete
argus data set to find out if that host  had an interaction with  
anything
in the active address space.   A simple interaction is not alarming,
however, an interaction where there is a response with data is.

Radark() is designed now to identify the scanners, through automatic
active address identification, and then go back and track any activity
that scanner may have had with any active addresses.  The big deal
is to realize what active addresses interacted with the scanner, and
if there was any data exchanged between the two machines.  Radark
should be able to actually show the content of the data if a local  
active
address responded with data.  This way you get a report of relevant
scanner activity, not just that the scan occurred.

The challenge is defining what the dark space is.

The way I suggest we go about it is to track the active address/port
space.  Argus is very good at realizing if a flow represents active
traffic or not.  In this first round of implementation, Radark will use
perl and standard argus client programs to track the "lit" network  
entities,
and then use the active addresses as a filter to identify traffic  
that is
going to the dark address space.

Currently radark() needs to know what is the CIDR address for local
traffic and a host number threshold for generating scanner reports,
but you don't have to use the threshold.  I think you're better off  
keeping
track of all addresses (threshold 1).  This can get in the 1M address  
space,
but with perl, its manageable.

We're not alone in going for dark space tracking.  An example of a
formal security relevant effort built around tracking dark
address activity can be found here:

    http://www.switch.ch/security/services/IBN/index_log.html

In many ways, this script, and follow on work should be seen
as an effort to bring this type of monitoring to any site.

I think the only configuration item should be a description of
the local address space.  The CIDR address of the local network
should be enough, but I also think the program should work without
any configuration.


OK, does this sound interesting?   Does this type of dialog that can
help us to improve the documentation?

Thanks, and I hope this path is helpful!!!!!!

Carter