Question

Exercice 6. Let be (Xi,..., Xn) an iid sample from the Bernoulli distribution with parameter θ, ie. I. What is the Maximum Likelihood estimate θ of θ? 2. Show that the maximum likelihood estimator of θ is unbiased. 3. We're looking to cstimate the variance θ (1-9) of Xi . x being the empirical average 2(1-2). Check that T is not unli ator propose an unbiased estimator of θ(1-0).

Answer 1

1.

The likelihood equation is,

$L(\theta) = \prod_{i=1}^{n}f(x_i;\theta) = \prod_{i=1}^{n}\theta^{x_i}(1-\theta)^{1-x_i} = \theta^{\sum_{i=1}^{n}x_i}(1-\theta)^{n-\sum_{i=1}^{n}x_i}$

The log-likelihood function is,

$logL(\theta) = log\theta \sum_{i=1}^{n}x_i + log(1-\theta) \left ( n-\sum_{i=1}^{n}x_i \right )$

The maximum likelihood estimate is,

$\frac{\partial }{\partial \theta}logL(\theta) = \sum_{i=1}^{n}x_i /\theta - \left ( n-\sum_{i=1}^{n}x_i \right ) /(1- \theta) = 0$

$\Rightarrow \frac{1-\theta}{\theta} = \left ( n /\sum_{i=1}^{n}x_i \right ) - 1$

$\Rightarrow \frac{1}{\theta} = \left ( n /\sum_{i=1}^{n}x_i \right )$

$\Rightarrow \theta= \left ( \sum_{i=1}^{n}x_i /n\right ) = \bar{x}$

Thus, maximum likelihood estimate of $\theta$ is,

$\hat{\theta}= \left ( \sum_{i=1}^{n}x_i /n\right ) = \bar{x}$

2.

$E\left [ \hat{\theta} \right ]= E\left ( \sum_{i=1}^{n}x_i /n\right ) =E[\bar{x}] = \theta$ (By central limit theorem and mean of Bernoulli distribution is $\theta$)

Since, $E\left [ \hat{\theta} \right ]= \theta$ ,   $\hat{\theta}$ is unbiased estimate of $\theta$.

3.

$E[T] = E[\bar{x}(1-\bar{x})] = E[\bar{x} - \bar{x}^2] = E[\bar{x}] - E[\bar{x}^2]$

$= E[\bar{x}] - (Var(\bar{x})+E[\bar{x}]^2) = \theta - \left ( \theta(1-\theta)/n + \theta^2\right )$

$= \theta - \left ( (\theta-\theta^2)/n + \theta^2\right ) = (n\theta-\theta+ \theta^2 - n\theta^2)/n$

$= ((n-1)\theta- (n-1)\theta^2 )/n$

$= (n-1)\theta(1 - \theta)/n$

Since, $E[T] \ne \theta(1 - \theta)$ T is not unbiased estimator of $\theta(1 - \theta)$ .

Let $T' = (n/n-1) \bar{x}(1 - \bar{x})$

Then,

$T' = (n/n-1) \bar{x}(1 - \bar{x}) = (n/n-1) T$

$E[T'] = E[(n/n-1) T] = (n/n-1)E[ T]$

$= (n/n-1) (n-1)\theta(1 - \theta)/n = \theta(1 - \theta)$

Thus,  T' is an unbiased estimator of $\theta(1 - \theta)$ .