Consider the three-state Markov process below; the number given on edge (i, j) is qij, the transition rate from i to j. Assume that the process is in steady state.
(a) Is this process reversible?
(b) Find pi, the time-average fraction of time spent in state i for each i.
(c) Given that the process is in state i at time t, find the mean delay from t until the process leaves state i.
(d) Find πi, the time-average fraction of all transitions that go into state i for each i.
(e) Suppose the process is in steady state at time t. Find the steady-state probability that the next state to be entered is state 1.
(f) Given that the process is in state 1 at time t, find the mean delay until the process first returns to state 1.
(g) Consider an arbitrary irreducible finite-state Markov process in which qij = qji for all i, j. Either show that such a process is reversible or find a counter example.
Text Book: Stochastic Processes: Theory for Applications By Robert G. Gallager.