Question A.4: Find the maximum likelihood estimator for the parameters α a......
Find the maximum likelihood estimator for the parameters α and β in the Gamma distribution:
\ f(x)=\begin{cases} \frac{\beta^{-\alpha}x^{\alpha -1}e^{-x/\beta}}{\Gamma(\alpha)},& x≥0, \\ 0, & x<0. \end{cases}
Since this is a continuous distribution, use the MLE defined by Equation (A.9) instead of Equation (A.8) as was done for the discrete case in Example A3.
\ L(\theta_1, … , \theta_m)=Pr[X_1=x_1,X_2=x_2, … , X_n=x_n]\\ = Pr[X=x_1] Pr[X=x_2] … Pr[X=x_n]\\=p(x_1;\theta_1, … , \theta_m) p(x_2;\theta_1, … , \theta_m) … p(x_n;\theta_1, … , \theta_m)\\=\prod\limits_{i=1}^{n}{p(x_i;\theta_1, … , \theta_m)}. (A.8)
\ L(\theta_1, … , \theta_m)= \prod\limits_{i=1}^{n}{f(x_i;\theta_1, … , \theta_m)}. (A.9)
There are two parameters: α and β. Therefore, the likelihood is a function of two variables, L(α, β). Assuming data points\ x_1, x_2, . . . , x_n , the likelihood function is given by
\ L(α, β)=(\frac{\beta^{-\alpha}x_1^{\alpha -1}e^{-x_1/\beta}}{\Gamma(\alpha)}) (\frac{\beta^{-\alpha}x_2^{\alpha -1}e^{-x_2/\beta}}{\Gamma(\alpha)}) … (\frac{\beta^{-\alpha}x_n^{\alpha -1}e^{-x_n/\beta}}{\Gamma(\alpha)}) \\[0.3cm] =\beta^{-n\alpha}\frac{(x_1x_2…x_n)^{\alpha -1}e^{-(x_1+x_2+…+x_n)/\beta}}{\Gamma^n(\alpha)}\\[0.3cm]=\frac{\beta^{-n\alpha}(\prod\limits_{i=1}^{n}{x_i})^{\alpha -1}exp(-\frac{1}{\beta}\sum\limits_{i=1}^{n}{k_i})}{\Gamma^n(\alpha)}.
Again, it is easiest to first take the logarithm of each side:
\ln L(\alpha, \beta)=-n\alpha \ln \beta + (\alpha -1) \ln(\prod\limits_{i=1}^{n}{x_i}) – \frac{1}{\beta}\sum\limits_{i=1}^{n}{x_i} – n \ln \Gamma(\alpha).However, this time it is required to set each partial derivative to zero. First, with respect to β,
\ \frac{1}{\Gamma}\frac{\partial L}{\partial \beta }=-\frac{n\alpha}{\beta}+\frac{1}{\beta^2}\sum\limits_{i=1}^{n}{x_i}=0,\\[0.3cm] \alpha \beta= \frac{1}{n}\sum\limits_{i=1}^{n}{x_i}. (A.11)
Next, with respect to α,
\ \frac{1}{L}\frac{\partial L}{\partial \alpha }=-n \ln \beta +\ln(\prod\limits_{i=1}^{n}{x_i})-n\frac{\Gamma'(\alpha )}{\Gamma (\alpha )} =0,\\ \ln \beta=\frac{1}{n}\ln(\prod\limits_{i=1}^{n}{x_i})-\frac{\Gamma'(\alpha )}{\Gamma (\alpha )}\\[0.3cm] = \frac{1}{n}\sum\limits_{i=1}^{n}{\ln x_i}-\Psi (\alpha ), (A.12)
where\ \Psi(\alpha) = \Gamma'(\alpha)/\Gamma(\alpha) is the so-called digamma function.
Combining Equations (A. 11) and (A. 12) to eliminate β,
\ \Psi(\alpha) – \ln \alpha = \ln(\frac{1}{n}\sum\limits_{i=1}^{n}{x_i})-\frac{1}{n}\sum\limits_{i=1}^{n}{\ln x_i}. (A.13)
Unfortunately, no explicit solution of Equation (A.13) for α is known to exist.
However, a very accurate approximation can be shown to be
\ \hat{\alpha}\approx 0.153495+\frac{0.5}{\ln(\frac{1}{n}\sum\limits_{i=1}^{n}{x_i})-\frac{1}{n}\sum\limits_{i=1}^{n}{\ln x_i}}. (A.14)
Once α is known, β is determined by
\ \hat{\beta}\approx \frac{1}{n\hat{\alpha}}\sum\limits_{i=1}^{n}{x_i}. (A.15)