Question

Markov Chains

Consider the Markov chain with transition matrix P = [ 0 1

1 0].

1) Compute several powers of P by hand. What do you notice?

2) Argue that a Markov chain with P as its transition matrix cannot stabilize unless both initial probabilities are 1/2.

Answer 1

0 1 1 0 P =

1) Observe that , hence the order of P is 2. Even powers of P are I and odd powers of P is P.

2) The Markov chain stabilizes when

T

We need to find
T P2
such that $\pi (P-I) = 0$ and $p_1+p_2=1$

-1 1 T 1 0 -1

Hence the solution is
P P
. But $p_1+p_2=1$, therefore
P1 P2 = 1/2
is the only solution.