Graphing traffic of the top 10 ports

[Carter Bullard]

Hey Jeffrey,
You just need to make a few modifications, and currently you'll have to
run multiple sets of commands to accomplish your graph.  This may
seem a bit complicated, but its not really.

You are trying to draw a time series graph for the last 24 hours of the
top 10 ports in your file, with a granularity of 30 seconds.  This graph
will have  (24 * 60 * 2) slots in the X axis, and up to 10 values in the Y axis.

The first question is, do you want to track the same "top 10 ports" from the
entire file, or are you interested in the "top 10 ports"  for each 30 second
graph interval?  The simpler graph is to track 10 ports over your entire
24 hour period.  That way you're looking at only 10 ports.

If you are interested in the top 10 ports for each 30 second period
that is another story, and I can answer that in another email.

The first thing you'll need for your graph, is the list of  " top 10 " ports.
This is not easy to do with a single pass through the data, so we'll have
to generate the port list, and then use it to filter the data to generate the graph.
I'll break it down such that we can do this efficiently, with limited passes
against the original file.

Because you are looking to draw with 30 seconds granularity, we 
should aggregate the data to dst ports, preserving the time resolution
for the graph.  To do this, use rabins().  rabins() is basically racluster(), but
confined to the bin strategy and size specified on the command line.

   rabins -r file -t -24h -m proto dport -M hard time 30s -w data.file - \(tcp and syn and synack\)  or udp

OK, aggregate data for the last 24 hours from file, using the "proto dport"
as the aggregation key, into 30 second hard time bins, and write the data to
the the file " data.file ", this will generate a file that has all the ports in them
aggregated in 30 second bins.  The filter is there to ensure that the "dst port" is
actually the service port.  When dealing with TCP, the dst port is the service
port only when we've seen the connection setup (syn and synack).

OK now we have an aggregated intermediate file, " data.file ", from which we can
determine the top 10 ports.

   racluster -r data.file -w - - not frag | rasort -m pkts -w - | ra -N 10 -s dport -n -L -1 > /tmp/ra.top.10.ports

OK, just aggregate up the data using racluster().  Pipe the output into rasort(), like you did
before, and let ra() trim the output to the top 10.  If you are interested in the next 10, you
will use another -N option, such as "-N 10-20".

Write the ascii port numbers (-n) to an intermediate file, without the column label (-L -1).

This will generate the list of top 10 ports.  To turn this list into a port filter, we'll have to use
some sed.1 magic, such as.

   % sed -e "2,\$s/^/or /" /tmp/ra.top.10.ports

This just prepends a "or " in front of all the lines except the first one, so putting it all together,
using the intermediate data file, and the port list, you can run ragraph() like so:

   ragraph pkts dport -M time 30s -r /tmp/data.file -w graph.png ……  - port \(`sed -e "2,\\$s/^/or /" /tmp/ra.top.10.ports` \)

Then when you're done, delete the temporary files.

OK, I know that was a lot, but hopefully that helps you out.  If there are problems,
don't hesitate to send more email !!!!!!


Carter


On Dec 29, 2011, at 7:01 AM, Jeffrey Everling wrote:

> Hello,
> 
> Can anyone point me in the right direction?
> 
> So far I have:
> 
> racluster -r /var/log/argus/argus-id5-log -t -24h -m proto pkts dport -w
> - | rasort -m pkts -r - -w - | ra -N 9 -r - -w - | ragraph pkts dport -M
> 30s -r - -w /usr/local/www/data/img/top10ports-24h.png -nn -title "Top
> 10 destination ports
> 
> But I think I need a replacement for racluster, because he graphs the
> clustered data.
> 
> What I want:
> 
> A graph that shows the traffic of the top 10 busiest ports, But not
> summarized like the above command does.
> 
> -- 
> Kindly regards,
> 
> 
> Jeffrey Everling

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20111229/03ef76da/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20111229/03ef76da/attachment.bin>