Question

In the "Choosing Balls rom an Urn” example. Determine the following n-step transition probabilities. (a) After two balls are painted, what is the probability that the state is [O 2 0]? (b) After three balls are painted, what is the probability that the state is [0 1 1]?

Answer 1

Answer:

The data represents a stationary Markov chain. The probability transition matrix is as follows. [011] [020] (002] [200] 10) 1o [00 0.5 0.5 0 00 [020] 0 0 00 [002 0 0 00 [200] 0 0 0 0.5 0.5 [0 0.25 0.25 0000.5 0 0.25 0 0.25 0 0.50

The n-step transition probabilities are to be determined. Let X,-[r b] be the vector denoting the state of the system after t balls are painted. Here u denotes the number of unpainted balls in the urn, r denotes the number of red balls and b denotes the number of black balls in the urn. Hence, when 0 balls are, painted, i.e. the initial state then it is given that

a. The n-step probability of transition from state i to state j (P, (n)) is P, (n)-element ij of matrix P"

After 2 balls are painted, the probability that the state is X2 [0 2 0 is given by the transition probability P(X, =[0 2 0]/x, =[2 0 0])-R2001020 Matrix p2 is obtained by matrix multiplication of P with itself that is p2-PxP. Thus [011] [020] [002] 200] [0 [Io1] [0 000 0 0 [020] 0 0.5 0.5 000 [002] 0 0.50.5 000 [200] 0.25 0.125 0.125 0 0.25 0.25 0 0.375 0.125 0.25 0 0.25 0 0.375 0.25 0.125 00.25

ti. [o 2 0s P(X, [0 2 0]/X [2 0.125

b. After 3 balls are painted, the transition probability that the state is [0 s 200 01 ) Here Matrix p is obtained by matrix multiplication of matrix p2 with itself that is p p x P

The following matrix p is obtained on multiplying matrix p2with p [020] [002] [200] [l10 [10 [0 0 0.5 0.5 0 00 [020] [002] 0 00 [200 0.375 0.1875 0.1875 0 0.125 0.125 10 0.4375 0.25 0.1875 00.125 0 0.4375 0.1875 0.25 00.5 0 So from the above matrix p, the transition probability that after 3 balls are painted, the state is 0 1 is

Now the initial state is X-2 0 0] and the final state is X, -[0 1 1]. To determine a way to go from the state [2 0 0to [0 1 in 3 steps First counting the total probability, along each and every route, the values are [2 0 0 0[0 2 0 16 [2 0 0 16 [2 0 00[0 0 2]-[0

Counting the total probability, we get Total probability11+1 8 16 16 8 0.375

Thus the graphical representation showing the various states and the corresponding probabilities is given below. [011 1/4 [020] [110] 1/2 101 1/4 [011 1/2 1/4 011] [200] [110] 1/2 1/2 [101] 1/4 [002] [011]

1 [011] 1/4 [020] [110] 1/4 [011] 12[101] 1/2 1/4 | [011] [110] 10 [200] 1/2 1/2 [101] 1/4 [0021 (011]