Consider a Markov process for which the embedded Markov chain is a birth-death chain with transition probabilities Pi, i+1 = 2/5 for all i ≥ 0, Pi, i-1 = 3/5 for all i ≥ 1, P01 = 1, and Pij = 0 otherwise.
(a) Find the steady-state probabilities {πi; i ≥ 0} for the embedded chain.
(b) Assume that the transition rate νi out of state i, for i ≥ 0, is given by νi = 2i. Find the transition rates {qij} between states and find the steady-state probabilities {pi} for the Markov process. Explain heuristically why πi /= pi.
(c) Explain why there is no sampled-time approximation for this process. Then truncate the embedded chain to states 0 to m and find the steady-state probabilities for the sampled-time approximation to the truncated process.
(d) Show that as m → ∞, the steady-state probabilities for the sequence of sampled-time approximations approach the probabilities pi in (b).
Text Book: Stochastic Processes: Theory for Applications By Robert G. Gallager.