Gamma Distribution

The exponential distribution gives the distribution of waiting time until the first event occurs. What if we are interested in the waiting time until the third event? The distribution of such a variable is given by the Gamma distribution.

As with the exponential distribution, the Gamma distribution can be derived from the Poisson distribution. Say we are interested in the waiting time $T$ until the $\alpha$-th event. The probability that we have to wait at least $t$ units of time is

$$F(t) = \mathbb{P}(T <= t) = 1 - \mathbb{P}(T > t)$$

The event $\{T > t\}$ implies that fewer than $\alpha$ events occurred in the interval $[0,t]$. Therefore,

$$\begin{align} \mathbb{P}(T > t) &= \sum_{k = 0}^{\alpha - 1} \text{Poisson}_{\lambda t} (X = k)\\ &= \sum_{k = 0}^{\alpha - 1} \frac{(\lambda t)^k e^{-\lambda t}}{k!}\\ &= \frac{(\lambda t)^0 e^{-\lambda t}}{0!} + \sum_{k = 1}^{\alpha - 1} \frac{(\lambda t)^k e^{-\lambda t}}{k!} \end{align}$$

Putting this back in $F(t)$ and differentiating gives

$$\begin{align} f(t) &= 0 - (-\lambda)e^{-\lambda t} - \sum_{k = 1}^{\alpha - 1} \frac{k(\lambda t)^{k-1} \lambda e^{-\lambda t} - (\lambda t )^k (-\lambda) e^{-\lambda t}}{k!}\\ &= \lambda e^{-\lambda t} - \lambda e^{-\lambda t} \sum_{k=1}^{\alpha - 1} \left( \frac{(\lambda t)^{k-1}}{(k-1)!} - \frac{(\lambda t)^{k}}{k!} \right)\\ &= \lambda e^{-\lambda t} - \lambda e^{-\lambda t} \left(1 - \frac{(\lambda t)^{\alpha - 1}}{(\alpha - 1)!} \right) \\ &= \frac{(\lambda t)^{\alpha - 1}}{(\alpha - 1)!} \lambda e^{-\lambda t} \\ &= \frac{\lambda^{\alpha}}{\Gamma (\alpha) t^{\alpha - 1} e^{-\lambda t}} \end{align},$$

which is the density function of the Gamma distribution.