Chebyshev's Inequality

Introduction :

Let X be a random variable which may be either discrete type or continuous type. If the probability distribution of the random variable X is known, then we can compute its mean E (X) and variance V (X) if these exist and hence we can make the probability statements regarding the E (X) and V (X). The probabilities are given by the area under a frequency curve of the distribution. If the frequency curve is approximately normal curve i.e. bell shaped curve, then the approximate probabilities over certain intervals such asμ±σ,μ± 2σ andμ± 3σ can be obtained as,

$$P (\mu \ - \ \sigma \ < \ X \ < \ \mu \ + \ \sigma) \ = \ 0.6826$$

$$P \ (\mu \ - \ 2 \sigma \ < \ X \ < \ \mu \ + \ 2 \sigma) \ = \ 0.9544$$

$$P \ (\mu \ - \ 3 \sigma \ < \ X \ < \ \mu \ + \ 3 \sigma) \ = \ 0.9974$$

$$Where, \ E \ (X) \ = \ \mu \ and \ V \ (X) \ = \ \sigma^2$$

But in many situations, probability or frequency curve differ markedly from a bell - shape. In this case, even if E (X) and V (X) are known, we can not compute the probability such as P ( | X - E (X) |≤ k ). However, we can give a very useful upper or lower bound to such probabilities. Such upper or lower bounds to the probabilities are given by an important theorem in probabilities known as Tchebyshev (1821 - 1894). In fact, the Chebyshev's inequality indicates how the variance V (X) measures the degree of concentration of probability near E (X) =μ. In other words,Chebyshev's inequality provides an upper bound (or lower bound) to the probability of getting a value that deviated from the mean by more than (or less than) k times standard deviations.

Statement :

Let X be a random variable of discrete type with probability mass function (pmf) p(x) or continious type with probability density function (pdf) f(x) with finite meanμ = E (X) and variance V (X) =σ². Then for any positive constant K > 0,

$$ P \ ( \ | \ X \ - \ \mu \ | \ ≥ \ k \sigma \ ) \ ≤ \ \frac{1}{k^2}$$

$$or, \ P \ ( \ | \ X \ - \ \mu \ | \ < \ k \sigma \ ) \ ≥ \ 1 \ - \ \frac{1}{k^2}$$

$$Where \ E(X) \ = \ \mu \ is \ finite \ and \ variance \ V(X) \ = \ \sigma^2 \ > \ 0 \ and \ < \ \infty.$$

Proof :

Consider, X is a random variable.

By definition, we have

$$V \ (X) \ = \ \sigma^2 \ = \ E \ [ \ X \ - \ E(X) \ ]^2 \ = \ E \ ( \ X \ - \ \mu \ )^2$$

$$= \ \int_{-\infty}^{\infty} \ ( \ X \ - \ \mu \ )^2 \ f(x) \ dx$$

$$= \ \int_{-\infty}^{\mu \ - \ k\sigma} \ ( \ x \ - \ \mu \ )^2 \ f(x) \ dx \ + \ \int_{\mu \ - \ k\sigma}^{\mu \ + \ k\sigma} \ ( \ x \ - \ \mu \ )^2 \ f(x) \ dx \ + \ \int_{\mu \ + \ k\sigma}^{\infty} \ ( \ x \ - \ \mu \ )^2 \ f(x) \ dx$$

$$\therefore \ \sigma^2 \ ≥ \ \int_{-\infty}^{\mu \ - \ k\sigma} \ ( x \ - \ \mu \ )^2 \ f(x) \ dx \ +\ \int_{\mu \ + \ k\sigma}^{\infty} \ ( \ x \ - \ \mu \ )^2 \ f(x) \ dx$$

$$Because, \ the \ integral \ \int_{\mu \ - \ k\sigma}^{\mu \ + \ k\sigma} \ ( x \ - \ \mu \ )^2 \ f(x) \ dx \ ≥ \ 0$$

Now, for the first integral, x ≤ μ - kσ ⇒μ - x ≥ kσ and for the second integral, x ≥ μ + kσ ⇒ x - μ ≥ kσ. Therefore, (x-μ)²≥ k²σ².

$$\therefore \ \sigma^2 \ ≥ \ k^2 \sigma^2 \ \left [ \ \int_{-\infty}^{\mu \ - \ k\sigma} \ f(x) \ dx \ + \ \int_{\mu \ + \ k\sigma}^{\infty} \ f(x) \ dx \ \right ]$$

$$or, \ \sigma^2 \ ≥ \ k^2 \sigma^2 \ \left [ \ P ( - \infty \ < \ X \ < \ \mu \ - \ k\sigma ) \ + \ P ( \mu \ + \ k\sigma \ < \ X \ < \ \infty ) \ \right ]$$

$$or, \ \sigma^2 \ ≥ \ k^2 \sigma^2 \ \left [ \ P ( X \ - \ \mu \ < \ - \ k\sigma ) \ + \ P ( X \ - \ \mu \ > \ k\sigma ) \ \right ]$$

$$or, \ \sigma^2 \ ≥ \ k^2 \sigma^2 \ \left [ \ P ( | X \ - \ \mu | \ ≥ \ k\sigma ) \ \right ]$$

$$\therefore \ P( | X \ - \ \mu | \ ≥ \ k \sigma ) \ ≤ \ \frac{1}{k^2}$$

$$Also, \ since, \ P( | X \ - \ \mu | \ ≥ \ k \sigma ) \ + \ P( | X \ - \ \mu | \ ≥ \ k\sigma ) \ = \ 1$$

$$We \ get, \ P( | X \ - \ \mu | \ ≥ \ k\sigma ) \ = \ 1 \ - \ P( | X \ - \ \mu | \ ≥ \ k\sigma ) \ ≥ \ 1 \ - \ \frac{1}{k^2}$$

Bibliography

Sukubhattu N.P. (2013). Probability & Inference - II. Asmita Books Publishers & Distributors (P) Ltd., Kathmandu.

Larson H.J. Introduction to Probability Theory and Statistical Inference. WileyInternational, New York.