Most efficient way to generate summary traffic report

[Karl Tatgenhorst]

   I do similar things and it can be done in one pass. You can do it in RAM or
if it is large enough you might database it first and then do analysis against the db.

Here is a non databased walkthrough:

First set a couple of variables:


$office1 = "192.168.1."
%ofc1 = ()
$office2 = "192.168.2."
%ofc2 = ()
$datacenter1 = "172.16.10."
%dc1 = ()
$datacenter2 = "172.16.20."
%dc2 = ()

my @arg_line = ()

while (<>) {
  @arg_line = split(~/\|/,$_)
  # This arrays structure will be dependant on how you set up your argus query
  # determine which (src or dst) is your local
  # determine what the other is sourced or destined to
  # for purpose of example we will say that office1 is local and this is destined to datacenter1
  # in reality you would have a switch statement with regexs to determine which it is
  $dc1{ $srcaddr.$dstaddr } = @arg_line
  # this is a hash of arrays the array contains all the argus fields
}

# if you do keys on each of these arrays each number will be the number of flows for that bucket
# add those numbers for total  and do the stats on that. Also you can unwind the hashes and pull
# pkt counts and byte counts and do that math as well


Sorry this is only skeleton code, but you should be able to make it work.


Oops, BTW I did the split based on a pipe as the delimiter. Pipe | is my favorite delim as it rarely
shows up in the packet captures. You can set that by adding -c"|" to your argus (ra) command.


Karl


On Thu, Apr 12, 2007 at 01:03:28PM -0700 toddmichael said:
> Just started working with Argus after reading about it in Richard Bejtlich's
> Tao of NSM book.  He's right - this tool is fantastic!  To get me started I
> am running argus on a span port to gather everything I need.  At this point,
> my plan is to let it run for about a week for the purpose of gathering some
> statistics regarding Internet usage.  I have an Internet pipe that is used
> for standard Internet usage and also for inter-office vpn connectivity.  I'm
> trying to generate a report which shows the percentage of our Internet
> traffic which is to/from:
> 
> office1
> office2
> datacenter1
> datacenter2
> all other
> 
> so I can report to management on how much is inter-office VPN vs all-other.
> I know I can do this using racount with perl or shell, but it requires many
> passes over the same file ( i.e. using filter of -t -24h net 192.168.x.x to
> find out traffic to/from 192.168.x.x in the past 24 hours) which is quite
> large and thus would be pretty inefficient.  Before going this route I'm
> wondering if there's a way to gather this info in a more efficient way
> (using 1 or 2 passes rather than 5+) using either racount or the numerous
> other utilities.  I appreciate your guidance.
> 
> toddmichael