Multinomial Distribution

An experiment is said to be a multinomial experiment if

1. the experiment consists of n fixed trials.
2. for each trail there are k ( > 2 i.e. 3 or more ) possible outcomes.
3. the trials are independent.
4. the probability for each outcome remains the same from trial to trial.

The examples of multinomial experiment are

1. a number of throws of a fair die in which rach throws can result six different outcomes.
2. the number of selection or drawings of balls ar random with replacement from a box, containing 20 balls of which 2 are white, 4 are black, 6 red and 8 green balls.
3. a number of people, randomly selected to ask them the reaction to a new hydropower project in a location, in which the outcomes could be

(a) like the project

(b) dislike the project

(c) indifferent

Definition :

The random variable X = (X₁, X₂, ..., X_k) denoting the outcome of n trials, whre X_i = frequencyor number of outcome E_i with respective probability p_i ( i = 1, 2, ..., k), is said to have multinomial distribution with parameters (n, p₁, p₂, ..., p_k), if its probability mass function (p.m.f) is given by

$$p(x_1, x_2, ..., x_n) \ = \ \frac{n!}{x_1! x_2! ... x_k!} \ p_1^{x_1} p_2^{x_2} ... p_k^{x_k}$$

$$where, \ \sum_{x=1}^k \ x_i \ = \ n \ and \ \sum_{i=1}^k \ p_i \ = \ 1.$$

Derivation of multinomial distribution

Let us perform n independent trials, each of which may result any one of k (>2) possible outcomes at eah tria. Let E₁, E₂, ..., E_k be the k mutually exclusive and exhaustive events (outcomes) of a trial with corresponding probability p₁, p₂, ..., p_k so thatp₁ + p₂ + ... + p_k = 1. Again, let X₁, X₂, ..., X_k be the frequency or number of times of respective events E₁, E₂, ..., E_k occurs in the n trials so that each X_i can take the values in the set { 0, 1, 2, ..., n } and X₁ + X₂ + ... + X_k = n.

consider an arrangement of outcomes, in which E₁ occurs x₁ times, E₂ occurs x₂ times, ... E_k occurs x_k times as E₁E₁ ... E₁, E₂E₂ ... E₂, ..., E_kE_k ... E_k.

x₁ times, x₂ times, ..., x_k times,

Here, the number of different possible arrangements of n trials of which x₁ are of event E₁, x₂ are of event E₂, ..., x_k aare of event E_k is equal to $$\frac{n!}{x_1! x_2! ... x_k!}.$$

The probability of this sequence of occurance of events is p₁^x₁ p₂^x₂ ... p_k^x_k for each arrangement. Then, in n trials, the probability that E₁ occurs x₁ times, E₂ occurs x₂ times, ..., and E_k occurs x_k times is given by

$$p(X_1 \ = \ x_1, X_2 \ = \ x_2, ..., X_n \ = \ x_n) \ = \ \frac{n!}{x_1! x_2! ... x_k!} \ p_1^{x_1} p_2^{x_2} ... p_k^{x_k}$$

which is the probability mass (pmf) of multinomial distribution with parameters (n, p₁, p₂, ..., p_k).

Since the above probability mass function is the general term in the multinomial expansion of ( p₁ + p₂ + ... + p_k)ⁿ, the distribution with its probability mass function (pmf) is called multinomial distribution. therefore it can easily be proved that

$$\sum_{x} \ p(x_1, x_2, ..., x_n) \ = \sum_{x} \frac{n!}{x_1! x_2! ... x_k!} \ p_1^{x_1} p_2^{x_2} ... p_k^{x_k}$$

= ( p₁ + p₂ + ... + p_k)ⁿ

= 1

If a random vector X = ( X₁, X₂, ..., X_k ) follows multinomial distribution with parameters (n, p₁, p₂, ..., p_k) then we denote it as

(X₁, X₂, ..., X_k) ~ M_k(n, p₁, p₂, ..., p_k).

It is important to note that the multinomial distribution is a ganeralization of binomial distribution. The multinomial distribution is complicated distribution because it is a multi varite distribution involving k variables X₁, X₂, ..., X_k but only k-1 variables are independent, as X₁ + X₂ + ... + X_k = n. However, this distribution has wide applications such as in sampling with replacement of which individuals or observations are classified into more than two categories or groups.

Reduction of multinomial distribution when k = 2 and k = 3

When k = 2, then the multinomial distribution has probability mass function (pmf)

$$p(x_1, x_2) \ = \ \frac{n!}{x_1! x_2! ... x_k!} \ p_1^{x_1} p_2^{x_2}$$

since x₁ + x₂ = n and p₁ + p₂ = 1, we get x₂ = n -x₁ and p₂ = 1 - p₁. Then,

$$p(x_1, n-x_1) \ = \ \frac{n!}{x_1! (n-x_1)! ... x_k!} \ p_1^{x_1} (1-p_1)^{n-x_1}$$

if we write x₁ = x and p₁ = p, then

$$p(x, n-x) \ = \ \frac{n!}{x! (n-x)! ... x_k!} \ p^{x} (1-p)^{n-x}$$

$$\Rightarrow \ p(x) \= \ \frac{n!}{x! (n-x)! ... x_k!} \ p^{x} (q)^{n-x} \ \ \ ; \ x \ = \ 0, \ 1, \ 2, \ ..., \ n$$

Where,

x = Number of successes

n-x = Number of failures

p = probability of successes of an event in a single trial

q = 1 - p = probability of failures in a single trial.

This p(x) is the pmf of binomial distrinution with parameters n anf p.

Thus, when k = 2, the mulltinomial distribution reduces to binomial distribution with parameters n and p. Symbolically,

(X₁, X₂) ~ M₂ (n, p₁, p₂) = B (n, p₁)

$$\Rightarrow \ X_1 \ ~ \ B(n, p_1)$$

Similarly, when k = 3, then the multinomial distribution reduces to trinomial distribution with probability mass function (pmf)

$$p(x_1, x_2, x_3) \ = \ \frac{n!}{x_1! x_2! x_3!} \ p_1^{x_1} \ p_2^{x_2} \ p_3^{x_3}$$

where, x₁ + x₂ + x₃ = n and p₁+ p₂ + p₃ = 1.

We have x₃ = n - x₁ - x₂ and p₃ = 1 - p₁ - p₂. It implies that

$$p(x_1, x_2, n-x_1-x_2) \ = \ \frac{n!}{x_1! x_2! (n-x_1-x_2)!} \ p_1^{x_1} \ p_2^{x_2} \ (1-p_1-p_2)^{n-x_1-x_2}$$

which is the probability mass function of trinomial distribution of two random variables X₁ and X₂.

Symbolically,

(X₁, X₂, X₃) ~ M₃ (n, p₁, p₂, p₃)

$$\Rightarrow \ (X_1, X_2) \ ~ \ M_3 \ (n, p_1, p_2).$$

Bibliography

Sukubhattu N.P. (2013). Probability & Inference - II. Asmita Books Publishers & Distributors (P) Ltd., Kathmandu.

Larson H.J. Introduction to Probability Theory and Statistical Inference. WileyInternational, New York.