Question

Let fX(t)) be a Markov chain with state space (o, 1) and consider the state transition matrix 1-? Suppose P(X(0) 0) 0.4 and P(X(0-1) 0.6. Calculate (in terms of ? and ?), (a) (2 pts) P(X(4) 1) (b) (2 pts) Elg(X(4), where g(0) 1 and g(1) 2 You can use that

Answer 1

$u.A^4=\left( \begin{array}{cc} 0.4 & 0.6 \\ \end{array} \right).\left( \begin{array}{cc} \frac{\lambda (-\lambda -\mu +1)^4}{\lambda +\mu }+\frac{\mu }{\lambda +\mu } & \frac{\lambda }{\lambda +\mu }-\frac{\lambda (-\lambda -\mu +1)^4}{\lambda +\mu } \\ \frac{\mu }{\lambda +\mu }-\frac{(-\lambda -\mu +1)^4 \mu }{\lambda +\mu } & \frac{\mu (-\lambda -\mu +1)^4}{\lambda +\mu }+\frac{\lambda }{\lambda +\mu } \\ \end{array} \right)\\ =\left( \begin{array}{cc} 0.4 \left(\frac{\lambda (-\lambda -\mu +1)^4}{\lambda +\mu }+\frac{\mu }{\lambda +\mu }\right)+0.6 \left(\frac{\mu }{\lambda +\mu }-\frac{(-\lambda -\mu +1)^4 \mu }{\lambda +\mu }\right) & 0.6 \left(\frac{\mu (-\lambda -\mu +1)^4}{\lambda +\mu }+\frac{\lambda }{\lambda +\mu }\right)+0.4 \left(\frac{\lambda }{\lambda +\mu }-\frac{\lambda (-\lambda -\mu +1)^4}{\lambda +\mu }\right) \\ \end{array} \right) \\$ $\therefore P(X(4)=1)= 0.6 \left(\frac{\mu (-\lambda -\mu +1)^4}{\lambda +\mu }+\frac{\lambda }{\lambda +\mu }\right)+0.4 \left(\frac{\lambda }{\lambda +\mu }-\frac{\lambda (-\lambda -\mu +1)^4}{\lambda +\mu }\right) \\\\=\frac{0.4 \lambda \left(1-(\lambda +\mu -1)^4\right)+0.6 \left(\mu (\lambda +\mu -1)^4+\lambda \right)}{\lambda +\mu }$

If u = (0.4 0.6) is the initial probability vector and A is the transition probability matrix then, the second entry of the u.A^4 gives the P(X(4) = 1) u.A^4 = (0.4 0.6).(lambda(-lambda - mu + 1)^4/lambda + mu + mu/lambda + mu lambda/lambda + mu - lambda(-lambda - mu + 1)^4/lambda + mu) = (0.4(lambda(-lambda - mu + 1)^4/lambda + mu + mu/lambda + mu) + 0.6(mu/lambda + mu - (-lambda - mu + 1)^4/lambda + mu) 0.6(mu(-lambda - mu + 1)^4/lambda + mu + lambda/lambda + mu) + 0.4(lambda/lambda + mu - lambda?(-lambda - mu + 1)^4/lambda + mu)) therefore P(X(4) = 1) = 0.69mu(-lambda - mu + 1)^4/lambda + mu + lambda/lambda + mu) + 0.4(lambda/lambda + mu - lambda(-lambda - mu + 1)^4/lambda + nu = 0.4lambda(1 - (lambda + mu -1)^4) + 0.6(mu(lambda + mu - 1)^4 + lambda)/lambda + mu g(x) = {1 if x = 0 2 if x = 1 E(g((4))) = sigma_i = 1^2 g(x(4)) \r\n

$g(x)=\begin{cases} 1 & \text{ if } x=0 \\ 2 & \text{ if } x= 1 \end{cases}\\ E(g(x(4)))=\sum_{i=1}^2 g(x(4))P(x(4)=i)=1\times P(x(4)=0)+2\times P(x(4)=0))\\ =\frac{2 \left( \lambda +(0.6 \mu -0.4 \lambda ) (-\lambda -\mu +1)^4\right)}{\lambda +\mu }+\frac{1. \mu +(0.4 \lambda -0.6 \mu ) (-\lambda -\mu +1)^4}{\lambda +\mu }\\ =\frac{2. \lambda + \mu +(0.6 \mu -0.4 \lambda ) (-\lambda -\mu +1)^4}{\lambda +\mu }$