(Renewal theory proof that a class is all recurrent or all transient) Assume that state j in a countable-state Markov chain is recurrent and i communicates with j. More specifically, assume that there is a path of length m from i to j and one of length k from j to i.
(a) Show that for any n > 0, Pm+n+k ≥ Pm Pn Pk .
(b) By summing over n, show that state i is also recurrent. Hint: Use Theorem 6.2.6.
(c) Explain why this shows that all states in a class are recurrent or all are transient.
Text Book: Stochastic Processes: Theory for Applications By Robert G. Gallager.