Question

could u help me with this question about the markov chain please?

We have 5 urns which are empty. One ball is placed randomly in one of the urns.The process keep repear until each of the urn have at least 1 ball.Let Xn denoted the number of non-empty urns on day n.

(a) Show {Xn is a Markov chain by giving its transition matrix. (b) Find P(X 3 2) and P(Xs 2 4) (c) Find the expected number of days exactly k of the urns are full for each k = 1, 2, 3, 4.

Answer 1

a) The given process is a markov chain, because the value of X_n for a particular day only depends on what happened on the n^th day and the process value on the (n-1)^th day that is on X_n-1.

This can be represented by the transition matrix as shown below:

$X_n = \begin{pmatrix} 0 &1 &0 &0 &0 &0 \\ 0 &0.2 &0.8 & 0 & 0 &0 \\ 0 & 0 &0.4 & 0.6 & 0 &0 \\ 0 & 0 & 0 & 0.6 &0.4 & 0\\ 0 & 0 & 0&0 &0.8 &0.2 \\ 0 & 0 & 0 & 0 & 0 & 1 \end{pmatrix}$

In the above matrix, the 6 states are: 0, 1, 2, 3, 4, and 5

b) P(X₃ <= 2) is the probability of having 2 or less than 2 is only possible if we are putting 1 ball in each of the 3 days continuously that is 1*0.8*0.6 = 0.48

Therefore 0.48 is the required probability here.

P(X₈ >= 4) is only possible when we have put 1 ball on the first day and then kept putting balls in the same urn on the next 7 days. Therefore probability here is computed as:

= 1*0.2⁷

= 0.0000128

Therefore the required probability here is 0.0000128

c) The expected number of days for exactly k = 1 urn to be full would be 1as we would put 1 ball on day 1 to one of the urn.

Now for k = 2, let the expected number of days be X,
Then X = 1 + 0.8*1 + 0.2*(X + 1)
0.8X = 2
X = 2.5

Therefore 2.5 is the expected number of days here for k = 2

For k = 3, let the expected number of days be Y. Then,

Y = 2.5 + 0.6*1 + 0.4*(Y + 1)
0.6Y = 3.5
Y = 5.833

Therefore expected number of days here is 5.833 for k = 3

For k = 4, let the expected number be Z, then we have here:

Z = 5.833 + 0.4*1 + 0.6*(Z + 1)
0.4Z = 6.833
Z = 17.08

Therefore 17.08 days is the expected number of days here.