argus and 95th percentile

Carter Bullard carter at qosient.com
Wed Mar 25 15:02:59 EDT 2009


Hey Rodney,
I found a bug in the code for rabins() when testing out this new version
of the clients that affects the examples that I sent earlier. Please  
grab
the latest code before trying out these new features!!!!

    ftp://qosient.com/dev/argus-3.0/argus-clients-3.0.2.beta.4.tar.gz

I added 95th percentile reporting to rahisto(), which is our frequency  
distribution
tool.  This tool calculates mean, stddev, max, min, and the median (50th
percentile), and so it was very easy to add 95th percentile to the  
report.

You feed rahisto() the output of your 5 minute rabins() aggregations,
and it will give you a little stats report of the specific variable, and
a frequency distribution of where the data falls.  If you don't know  
what
the range is, run it with just a small number of bins, and rahisto()  
will start
to show you where the data lies in its range.

Using the examples I used before:

        rabins -M rmon hard time 5m -m smac -r hourly.file -w /tmp/ 
data.out

Now, run rahisto() instead of rasort(), this way to generate your  
OutBound data
for the specific ether address ('sload') :

    rahisto -r /tmp/data.out -H sload 10 - ether src host  
0:a0:c5:e1:7a:fa
  N = 31      mean =  80407.974516  stddev =  67795.873860  max =  
174059.203125  min = 172.133331
            median =  82742.617188     95% = 173895.046875
  Class           Interval                Freq    Rel.Freq     Cum.Freq
      1   0.000000e+00-1.740600e+04         12    38.7097%     38.7097%
      2   1.740600e+04-3.481200e+04          0     0.0000%     38.7097%
      3   3.481200e+04-5.221800e+04          3     9.6774%     48.3871%
      4   5.221800e+04-6.962400e+04          1     3.2258%     51.6129%
      5   6.962400e+04-8.703000e+04          1     3.2258%     54.8387%
      6   8.703000e+04-1.044360e+05          0     0.0000%     54.8387%
      7   1.044360e+05-1.218420e+05          1     3.2258%     58.0645%
      8   1.218420e+05-1.392480e+05          3     9.6774%     67.7419%
      9   1.392480e+05-1.566540e+05          4    12.9032%     80.6452%
     10   1.566540e+05-1.740600e+05          6    19.3548%    100.0000%

And, run rahisto() this way to generate your InBound data for the  
specific ether address ('dload') :

    rahisto -r /tmp/data.out -H dload 10 - ether src host  
0:a0:c5:e1:7a:fa
  N = 31      mean = 2520065.831098  stddev = 2286667.779977  max =  
5742157.000000  min = 335.946655
            median = 2935971.500000     95% = 5711441.500000
  Class           Interval                Freq    Rel.Freq     Cum.Freq
      1   0.000000e+00-5.742160e+05         13    41.9355%     41.9355%
      2   5.742160e+05-1.148432e+06          1     3.2258%     45.1613%
      3   1.148432e+06-1.722648e+06          1     3.2258%     48.3871%
      4   1.722648e+06-2.296864e+06          1     3.2258%     51.6129%
      5   2.296864e+06-2.871080e+06          0     0.0000%     51.6129%
      6   2.871080e+06-3.445296e+06          2     6.4516%     58.0645%
      7   3.445296e+06-4.019512e+06          1     3.2258%     61.2903%
      8   4.019512e+06-4.593728e+06          3     9.6774%     70.9677%
      9   4.593728e+06-5.167944e+06          3     9.6774%     80.6452%
     10   5.167944e+06-5.742160e+06          6    19.3548%    100.0000%


The numbers are slightly different from the last time, because of the  
bug in rabins().

So you see, the data I'm using is multi-modally distributed, and while  
very low in samples,
it does suggest  an SLA for two tiers, one above 1M bps and one below  
1M bps.
You can calculate a 95th percentile for the two regions, by adjusting  
the range on the
histogram option field like this (just do dload for this example):

Traffic Below 1M bps
  rahisto -r /tmp/rabins.5m.out -H dload 10:0-1M - ip and ether src  
host 0:a0:c5:e1:7a:fa
  N = 14      mean = 220514.849217  stddev = 168862.596105  max =  
612549.625000  min = 335.946655
            median = 157696.734375     95% = 612549.625000
  Class           Interval                Freq    Rel.Freq     Cum.Freq
      1   0.000000e+00-1.000000e+05          2    14.2857%     14.2857%
      2   1.000000e+05-2.000000e+05          7    50.0000%     64.2857%
      3   2.000000e+05-3.000000e+05          1     7.1429%     71.4286%
      4   3.000000e+05-4.000000e+05          2    14.2857%     85.7143%
      5   4.000000e+05-5.000000e+05          1     7.1429%     92.8571%
      6   5.000000e+05-6.000000e+05          0     0.0000%     92.8571%
      7   6.000000e+05-7.000000e+05          1     7.1429%    100.0000%
      8   7.000000e+05-8.000000e+05          0     0.0000%    100.0000%
      9   8.000000e+05-9.000000e+05          0     0.0000%    100.0000%
     10   9.000000e+05-1.000000e+06          0     0.0000%    100.0000%

Traffic Above 1M bps
  rahisto -r /tmp/rabins.5m.out -H dload 10:1-6M - ip and ether src  
host 0:a0:c5:e1:7a:fa
  N = 17      mean = 4413813.698529  stddev = 1253167.017151  max =  
5742157.000000  min = 1717778.125000
            median = 4992350.500000     95% = 5742157.000000
  Class           Interval                Freq    Rel.Freq     Cum.Freq
      1   1.000000e+06-1.500000e+06          0     0.0000%      0.0000%
      2   1.500000e+06-2.000000e+06          2    11.7647%     11.7647%
      3   2.000000e+06-2.500000e+06          0     0.0000%     11.7647%
      4   2.500000e+06-3.000000e+06          1     5.8824%     17.6471%
      5   3.000000e+06-3.500000e+06          1     5.8824%     23.5294%
      6   3.500000e+06-4.000000e+06          1     5.8824%     29.4118%
      7   4.000000e+06-4.500000e+06          1     5.8824%     35.2941%
      8   4.500000e+06-5.000000e+06          4    23.5294%     58.8235%
      9   5.000000e+06-5.500000e+06          3    17.6471%     76.4706%
     10   5.500000e+06-6.000000e+06          4    23.5294%    100.0000%

Hope this is helpful,

Carter

On Mar 23, 2009, at 3:16 PM, Rodney McKee wrote:

> Carter,
>
> The value I'm after is based on 5 minute samples of user uploads  
> (inbound traffic) that are sorted highest to lowest then the value  
> at the 95th percent point is then used for our volume calculation.  
> Is their a way to pull the 5 minutes samples like "ragraph -M 5m"  
> using the text "ra" tools?
> I just see the rrdgraph is able to do it but not sure yet how to use  
> it.
>
> More info on the billing scheme here:
> http://en.wikipedia.org/wiki/Burstable_billing
>
> ----- "Carter Bullard" <carter at qosient.com> wrote:
> > Hey Rodney,
> > When I see '95th percentile', i immediately ask,  "of what?".
> > Argus can do a lot of simple stats, and means with standard
> > deviations are really simple for any of the metrics argus
> > supports, but....., there are a lot of them, so are you looking
> > for 95th percentile of flow duration? (just as an example)
> >
> > Carter
> >
> > On Mar 22, 2009, at 7:07 PM, Rodney McKee wrote:
> >
> > > Just wondering what might be the best way to calculate the 95th
> > > percentile out of argus?
> > >
> > > We use several data centers that do billing calculations using  
> this
> > > process.
> > >
> > > Rgds
> > > Rodney
> > >
> >
> >
>
> -- 
>
>
>
> Rodney McKee
> Linux systems administrator
> Aconex
> The easy way to save time and money on your project
>
> 696 Bourke Street, Melbourne
> Tel: +61 3 9240 0200               Fax: +61 3 9240 0299
> Email: rmckee at aconex.com      www.aconex.com
>
>
> This email and any attachments are intended solely for the  
> addressee. The contents may be privileged, confidential and/or  
> subject to copyright or other applicable law.
> No confidentiality or privilege is lost by an erroneous  
> transmission. If you have received this e-mail in error, please let  
> us know by reply e-mail and delete or destroy
> this mail and all copies. If you are not the intended recipient of  
> this message you must not disseminate, copy or take any action in  
> reliance on it. The sender takes no
> responsibility for the effect of this message upon the recipient's  
> computer system.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York  10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090325/d83808f1/attachment.html>


More information about the argus mailing list