Question

Problem 5. A Markov chain Xn, n probability matrix: 0 with states 1, 2, 3 has the following transition 0 1/3 2/3 1/2 0 1/2 If P(o-: 1)-P(Xo-2-1/4, calculate E(%) (use a computer).

Answer 1

We first find the 3^rd state transition matrix here as:

$P = \begin{pmatrix} 1/2 & 1/3 &1/6 \\ 0 &1/3 & 2/3 \\ 1/2 &0 & 1/2 \end{pmatrix}$

Now the product here is computed as:

$P^3 = \begin{pmatrix} 0.3611 & 0.2037 & 0.4352 \\ 0.4444 &0.1481 & 0.4074 \\ 0.4167 &0.2222 & 0.3611 \end{pmatrix}$

This is the required matrix here for 3^rd transition state.

The probabilities for 2^nd state here is computed as:

$E(X_2)= \begin{pmatrix} 0.25 &0.5 &0.25 \end{pmatrix} \begin{pmatrix} 0.3611 & 0.2037 & 0.4352 \\ 0.4444 &0.1481 & 0.4074 \\ 0.4167 &0.2222 & 0.3611 \end{pmatrix} = \begin{pmatrix} 0.4167 &0.1806 &0.4028 \end{pmatrix}$ This is the required expected states for the 2nd state.