Poisson Distribution

In this post, I explain how to derive the Poisson distribution from the binomial distribution.

The Poisson distribution is a discrete probability distribution. It expresses the probability that a specified number of events occur in a fixed interval (in time and/or space). The crucial assumptions here are (1) the events occur independently of each other, and (2) the rate at which they occur is constant.

To understand the Poisson distribution, we first revisit the binomial distribution. This distribution is also discrete, and it gives the probability that a specified number $x$ out of $n$ independent Bernouille trials, each with success probability $p$, are successes. This frequency function is given by,

$$\mathbb{P} = \binom{n}{x} p^x (1-p)^{n-x}$$

Now imagine that by some strange turn of fate, I am abandoned on a remote island and forced to drink stagnant water from a broken coconut shell. There are bacteria randomly distributed in this water, so the more of it I drink, the more bacteria I ingest, and the larger the risk that I succumb to some infection and die!

Being a statistician, I would like to quantify my risk. Assume that this risk directly corresponds to the number of bacteria I consume, I could model the number of bacteria as a binomial random variable. A trial would be 1ml of water, and a success is signified by the presence of at least one baterium in that 1ml of water. The probability of success is roughly the concentration of bacteria per ml.

If we assume that the bacteria are evenly distributed in the water, then we can assume that the number of bacteria in the next ml of water I drink is independent of the ones I have drunk. This satisfies the independent trials assumption of the binomial distribution.

Now there’s a problem with the binomial model. The bacteria count will not be very accurate if there are often more than one bacteria in a ml of water, since these additional bacteria will not be counted in the $1/0$ success/failure counting method.

Since the amount of water is finite, increasing the number of trials means decreasing the amount of water in a trial (say 0.1 ml, 0.01 ml, etc). Consequently, the bacteria concentration per unit of water will also decrease.

$$\frac{\text{number of bacteria}}{\text{number of larger units}} > \frac{\text{number of bacteria}}{\text{number of smaller units}}$$

because

$$\text{number of units} = \frac{\text{amount of water}}{\text{unit size}}$$

so smaller unit size results in larger number of units, and consequently smaller concentration.

Therefore to make our model more accurate, we need to find the limit of the binomial distribution as the number of trials approach infinity. As $n \rightarrow \infty$, $p \rightarrow 0$ but $n p = \mathrm{number of bacteria}$ stays the same and is finite.

Let $\lambda$ denote the bacteria concentration in ml water, and $n$ the number of trials, each with $t$ ml of water ($t$ could be $0.1, 0.01$, etc). So

$$\lambda t = np$$

and we have

$$\begin{align} \mathbb{P}(X = x) &= \lim_{n \rightarrow \infty} \frac{n!}{x!(n-x)!} p^x (1-p)^{n-x} \\ &= \lim_{n \rightarrow \infty} \frac{n!}{x!(n-x)!} \left( \frac{\lambda t}{n}\right)^x \left( 1 - \frac{\lambda t}{n}\right)^{n-x} \\ &= \lim_{n \rightarrow \infty} \frac{n(n-1)(n-2)...(n-x+1)}{n^x} \frac{(\lambda t)^x}{x!} \frac{(1-\lambda t / n)^n}{(1-\lambda t / n)^x} \\ \end{align}$$

Because

$$\begin{align} \lim_{n\rightarrow \infty} \frac{n(n-1)(n-2)...(n-x+1)}{n^x} &= 1, \\ \lim_{n\rightarrow \infty} \left( 1- \frac{\lambda t}{n}\right)^n &= e^{-\lambda t}, \quad \text{and} \\ \lim_{n\rightarrow \infty} \left(1 - \frac{\lambda t}{n} \right)^x &= 1 \end{align},$$

the previous limit becomes

$$\mathbb{P}(X = x) = \frac{e^{-\lambda t} (\lambda t)^x}{x!},$$

which is the frequency function of Poisson$(\lambda t)$.

Thus if $X ~$ Poisson$(\lambda t)$ then $\mathbb{P}(X=x)$ gives the probability that $x$ events will occur in an interval.