Question

3.8 Let Pa = (1/2 3/2) and P2 = (145 1/3) Consider a Markov chain on four states whose transition matrix is given by the block matrix (a) Does the Markov chain have a unique stationary distribution? If so, find it. (b) Does lim Pr exist? If so, find it. (c) Does the Markov chain have a limiting distribution? If so, find it.

Can you solve c. I'm done with a and b

Answer 1

Solution » وام +1h at رفع the we see that P and Pe are also atte od itself transition matrix First we find the limiting distribution of P and a seberately. Ergodic Theorem For a finite , irreducible, aperiodic markov chain with TPM P=(bij). The limit th= lim bith exists and no jk are independent of the initial distribution state ŷ. . The limiting distribution {k } is identical with stationary distribution. be in this case limiting distribution are identical with Stationary distribution e

Graph ofor P2 4 0ths @ @ finite Both e and P are irreducible Markov chain we find stationary both Pe and Pa seperately. and aberiodic distribution for Stationary distribution of for p T' P2 = T & T +Tg = where Il = (TI, , Te ) ® - T. +&T2 = T = Te Te - D 3 2 1 1 2 + 2 = Tz 3 / 2 TL = 22 & Ti + Te = 1 = π, = (2-te) Put in equation 3 (1 - 12) = $ Tz = 3 - 3 T2 = 2 2 2 2 2 = 2 et 2 The 34-354-3 TE= T = I-1- -

Herefore the lening & Stationary ļ is '- ( T ) = using the ergodic Thearem, liminting distribution of P is distribution of 3] ما जा lim P معا जा Stationarly distribution of Po I P₂ = A & T + Io = Ty Po = 70 where Title CT3 Ta) - & Mat Tar1 Ś $ r M ] * T3 + 4 TA = To = 1 / 1 4 = 4 T3 1 TA=113 & The + Tast at za tu - I3 = 1-14 = 1-13 Э 9 10 1 5 13 = = = т

P2 is Therefore the stationary distribution of T = [1 Tal= ( 1 2 using ergodic theorem, limiting distribution of P is neopreny -los ho 2 too chain e such that Now we form a markov whose T. p. m is given by o mit -07 at o T io Gilf the Juxa Graph for p LÀ 2 3/4 Period Therefore This the of each state is t. markov Chain is aberiodic. limiting distribution exists for P.

& limiting distribution of Pis lim pr. ho p = P.Po Loo 7 iso o 415 45 ) which is the product of Block diagonal matrix we know that the produest of Block diagonal matrix 18 the product of diagonal their diagonal matrices, le if cis a Block matrix such that matra c=10] where A , & B are square thens = cc = 1*)-( P-PP-1183-18

Similarly, & Slim pr lim pa lim no lim Po nywa Therefore limiting distribution of markov chain is O ho lim pa so o o o Glo Glo -or for To 0 ] J4x4 using D & A*