Exercise 6 7 Consider An Irreducible Markov Chain That Is Positive Recurrent Recall 2673867

Exercise 6.7. Consider an irreducible Markov chain that is positive recurrent. Recall the technique used to find the expected first passage time from j to i, i.e., Tii, in Section 4.5. The state i was turned into a trapping state by turning all the transitions out of i into a single transition Pii = 1. Here, in order to preserve the positive recurrence, we instead move all transitions out of state i into the single transition Pik = 1. a) Use Figure 6.2 to illustrate that the above strategy can turn an irreducible chain into a reducible chain. Also explain why states i and j are still positive recurrent and still in the same class.
b) Let {7;,; k > 0} be the steady state probabilities for the positive-recurrent class in the modified Markov chain. Show that the expected first passage time T’ii from j to i in the modified Markov chain is (1/71-2) — 1. c) Show that the expected first passage time from j to i is the same in the modified and unmodified chain.
d) Show by example that after the modification above, two states i and j that were not positive recurrent before the modification can become positive recurrent and the above technique can again be used to find the expected first passage time.

Attachments:

hwquestion.png