argus and 95th percentile
Carter Bullard
carter at qosient.com
Mon Mar 23 21:10:18 EDT 2009
Hey Rodney,
This should be very simple to do. rabins() is the client you should
use.
It is designed to give you the timed 'bins' of data, and from those you
should be able to calculate your 95th percentile. (rabins() is what
ragraph()
uses to structure its data).
So, do you want to do it for the whole link? The totals seen for a
specific
ethernet address every 5 minutes would be the best way to calculate
the metrics.
Here is a run with a little data I just grabbed.
rabins -M rmon hard time 5m -m smac -r hourly.file -w /tmp/data.out
rasort -r /tmp/data.out -m load -s stime dur smac sload:16 dload:
16 - ether src host 0:a0:c5:e1:7a:fa
My file has a few ethernet addresses, so I put a filter on the
rasort() to pick
the interface I was most interested in. Run yours the first without
the filter, so
you can see all the ethernet addresses that are included in your data
file.
OK, just as a reminder, the "SrcLoad" is the load where the ethernet
address is
the source. This converts to the "OutBound" metric for that specific
interface.
The "DstLoad" is the bits per second where the ethernet address is the
destination,
so this value is the "InBound" metric. If its confusing, I can
change the column labels
pretty easily.
So, here's my output.
StartTime Dur SrcMac
SrcLoad DstLoad
2008/01/30 12:10:00.000000 300.000000 0:a0:c5:e1:7a:fa
173847.312500 5741739.500000
2008/01/30 12:25:00.000000 300.000000 0:a0:c5:e1:7a:fa
173845.687500 5711335.500000
2008/01/30 12:50:00.000000 300.000000 0:a0:c5:e1:7a:fa
170789.921875 5646871.500000
2008/01/30 12:15:00.000000 300.000000 0:a0:c5:e1:7a:fa
169014.750000 5558620.000000
2008/01/30 12:20:00.000000 300.000000 0:a0:c5:e1:7a:fa
166964.312500 5483918.000000
2008/01/30 12:35:00.000000 300.000000 0:a0:c5:e1:7a:fa
163149.328125 5415570.000000
2008/01/30 12:00:00.000000 300.000000 0:a0:c5:e1:7a:fa
155537.250000 5008136.500000
2008/01/30 12:45:00.000000 300.000000 0:a0:c5:e1:7a:fa
152479.953125 4992304.500000
2008/01/30 12:40:00.000000 300.000000 0:a0:c5:e1:7a:fa
140724.562500 4634209.000000
2008/01/30 12:05:00.000000 300.000000 0:a0:c5:e1:7a:fa
142387.859375 4568126.000000
2008/01/30 12:30:00.000000 300.000000 0:a0:c5:e1:7a:fa
137727.046875 4559280.000000
2008/01/30 11:45:00.000000 300.000000 0:a0:c5:e1:7a:fa
135793.250000 4156575.750000
2008/01/30 11:55:00.000000 300.000000 0:a0:c5:e1:7a:fa
127361.468750 3767734.500000
2008/01/30 11:50:00.000000 300.000000 0:a0:c5:e1:7a:fa
105753.250000 3172967.750000
2008/01/30 11:30:00.000000 300.000000 0:a0:c5:e1:7a:fa
82726.773438 2935927.250000
2008/01/30 12:55:00.000000 300.000000 0:a0:c5:e1:7a:fa
59120.800781 1962049.500000
2008/01/30 11:35:00.000000 300.000000 0:a0:c5:e1:7a:fa
47501.199219 1717744.000000
2008/01/30 11:10:00.000000 300.000000 0:a0:c5:e1:7a:fa
52015.308594 612487.437500
2008/01/30 10:40:00.000000 300.000000 0:a0:c5:e1:7a:fa
35931.066406 490524.031250
2008/01/30 11:40:00.000000 300.000000 0:a0:c5:e1:7a:fa
13628.639648 362009.125000
So I just processed 100 minutes of data, for the example, but you
would want to do a full day, or a week or a
months worth of data, to get a really good number. But in hist
example, I have 20 samples, and you would get
the 95th percentile by tossing just the first entry. So the answer
would be:
InBound 95th Percentile = 5711335.500000 bps
OutBound 95th Percentile = 173845.687500 bps
I sorted them based on "load" which is the combined ('sload' +
'dload'), and so you would probably want to
sort them on the specific direction that you want to calculate the
metric for (in this case we got away with it).
If you try rabins() and it doesn't generate any output (but there
should be data), you may
be experiencing a bug. I have uploaded a new image of argus-
clients-3.0.2.beta.3
that I know works, so please download the new client software before
starting.
ftp://qosient.com/dev/argus-3.0/argus-clients-3.0.2.beta.3.tar.gz
So, I know that seems a bit complicated, but the process is hopefully
simple.
You need to aggregate all the flow data so that the metrics represent
values associated
with just the link in question. To do this, you use the "-M rmon"
option. The "rmon" option
converts bi-directional flow data that has metrics for 2 entities (the
sender and receiver),
into RMON style data, where you want metrics for a single entity with
"in" and "out"
oriented stats.
The "hard" option forces the metrics to be bound to the hard time
boundaries specified
(5 minute time boundaries), and the "time 5m" is generating our 300
second bins.
We use the "-m smac" option, to aggregate on one of the ethernet
addresses (pick one for
the rabins() and always use that same one (smac or dmac) throughout
the entire operation).
At this point, we can either write the data to a file, or to a pipe.
I wrote it to a file, just
for convenience.
rabins() does do some sorting, but its key is oriented around
aggregation. To
order the data on "load" so you can do the simple exclusion, you need
to send the output to
rasort(), using the "-m sload" option when you want to determine the
load sent from
the ethernet address (outbound) (where the ethernet address was the
'src') and "-m dload"
when you want to sort based on the load sent to the ethernet address
(inbound).
If you have any questions at all, or if something doesn't jive, just
send more email!!!!
Carter
On Mar 23, 2009, at 3:16 PM, Rodney McKee wrote:
> Carter,
>
> The value I'm after is based on 5 minute samples of user uploads
> (inbound traffic) that are sorted highest to lowest then the value
> at the 95th percent point is then used for our volume calculation.
> Is their a way to pull the 5 minutes samples like "ragraph -M 5m"
> using the text "ra" tools?
> I just see the rrdgraph is able to do it but not sure yet how to use
> it.
>
> More info on the billing scheme here:
> http://en.wikipedia.org/wiki/Burstable_billing
>
> ----- "Carter Bullard" <carter at qosient.com> wrote:
> > Hey Rodney,
> > When I see '95th percentile', i immediately ask, "of what?".
> > Argus can do a lot of simple stats, and means with standard
> > deviations are really simple for any of the metrics argus
> > supports, but....., there are a lot of them, so are you looking
> > for 95th percentile of flow duration? (just as an example)
> >
> > Carter
> >
> > On Mar 22, 2009, at 7:07 PM, Rodney McKee wrote:
> >
> > > Just wondering what might be the best way to calculate the 95th
> > > percentile out of argus?
> > >
> > > We use several data centers that do billing calculations using
> this
> > > process.
> > >
> > > Rgds
> > > Rodney
> > >
> >
> >
>
> --
>
>
>
> Rodney McKee
> Linux systems administrator
> Aconex
> The easy way to save time and money on your project
>
> 696 Bourke Street, Melbourne
> Tel: +61 3 9240 0200 Fax: +61 3 9240 0299
> Email: rmckee at aconex.com www.aconex.com
>
>
> This email and any attachments are intended solely for the
> addressee. The contents may be privileged, confidential and/or
> subject to copyright or other applicable law.
> No confidentiality or privilege is lost by an erroneous
> transmission. If you have received this e-mail in error, please let
> us know by reply e-mail and delete or destroy
> this mail and all copies. If you are not the intended recipient of
> this message you must not disseminate, copy or take any action in
> reliance on it. The sender takes no
> responsibility for the effect of this message upon the recipient's
> computer system.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York 10022
+1 212 588-9133 Phone
+1 212 588-9134 Fax
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090323/953203e0/attachment.html>
More information about the argus
mailing list