Argus 2.0 features

[Carter Bullard]

Gentle People,

I've been away on vacation, so sorry for the late reply!

Now that I am free of corporate entanglements, I can now
start to talk about what I would like to work on in Argus-2.0.
The way I see it, Argus is a unique monitor, even after
all these years.  No other network activity monitor generates
comprehensive IP traffic audits like Argus.  I believe that this
is Argus's strength and I want to emphasize this in the
Argus-2.0 effort.  My goal is to make Argus-2.0 a complete IP
transaction audit system.  What that actually means will
hopefully come out in the next weeks.

In this mail, I mention some topics that are at the top of
my list to work on.  This doesn't make them the highest priority
items, these are just the ones that came to mind while I was
writing this piece of mail.  Hopefully, we'll get a dialog
started so we can find the right list of items to work on.

As an audit system, I believe that there are three fundamental
issues that we need to work on in Argus-2.0.  The quality/utility
of Argus data, the management of that data, and applications
that use Argus data to do real work.

The quality/utility of Argus data is demonstrated by the applications
that use the data, so 1 and 3 are related.  So far, the Argus
project has presented proof-of-concept clients.  In Argus-2.0,
we are going to provide more real tools, rather than "here's
an example of .....".  The tools can be, but not limited to,
security, operations and network management, and accounting,
to name a few.  Suggestions in this area are very much encouraged.

Argus data can already support a large number of applications
that we don't have support for.  Things such as service availability
reporting and network delay determinations, say using
the ping data that Argus generates, are real world applications
that are easy.  Simple, straight-forward ....

But we do need to improve the support we currently have.  Russell's
suggestions on TCP protocol reporting is well said, and definitely
will be a part of 2.0!

The kind of new applications that I want to support are in the
area of network service quality, which involve service quality
measurement and assessment.  We already have a good start on
this with Argus-1.8, but there is more that needs to be done.
The trend in QoS is IP telephony performance measurement which
makes things like packet jitter important.  There are serious
security issues in the area of service quality, and so for those
that have a security side, I believe that this will make for
great conversation!!

But in the area of data management, there is opportunity to do
a lot of really good stuff.

Version 1.x of Argus had 2 record types, data and management.
I would like to enhance/extend the management records to improve
file management and data processing.  Adding some database concepts
to Argus output files, such as indexes and summary information
in the front of the files using special management records, should
support faster lookups.  Also developing efficient compression
techniques for Argus data is something that all of us would
appreciate.

A big one will be direct support for anonymity in Argus data.
This could be accomplished through tools that alias addresses,
and put the alias lookup table in a special section of the Argus
data file.  So if you want to share Argus data,  without
giving the real identities away, you just strip away the
alias table.  This could also be a source of really efficient
compression, by the way.
 
Sounds already like a HUGE amount of work, but the file formats
come first, and the functionality then follows.

Most on the list have not been vocal about the binary formats
of Argus streams/files, probably because most have been working
with ra() output for the most part.  And I have not heard of any
problems with the core concepts of Argus itself, which I am not
planning on changing in the 2.0 effort.  Hopefully in the next
month or so, we'll get everyone interested in some aspect of
the theory of Argus, so we can decide if its doing the job and
of course the actual format of its data records.

OK, down to brass tacks.

I am going to propose a 192 byte fixed length record as the basic
Argus data element, instead of the 64 byte record that we currently
are using.  The additional 128 bytes will be available for a lot
of stuff!  Some things we will be adding to support better data
management.  Simple, boring things like an Argus monitor ID so
we'll know which Argi generated the record, an Argus transaction ID,
so we can correlate events and records better, and data fields to
directly support record aggregation, such as an aggregation
counter, which will indicate how many records were merged together.
Any suggestions along these lines would be great!!!!

I am also going to propose that we capture the first 32 bytes of
user data that is sent by both sides of the IP transaction.
This will aid in so many things that its hard to enumerate
them all, but it should solve a lot of problems that people have
mentioned.  My biggest use of this is in covert channel detection
and service detection.

Any comments?


Carter