Consider a finite-state Markov chain with matrix [P] which has κ aperiodic recurrent classes, R 1, ... , R κ and a set T of transient states. For any given recurrent class .e, consider a vector ν such that νi = 1 for each i ∈ R .e, νi = limn→∞ Pr{Xn ∈ R .e|X0 = i} for each i ∈ T , and νi = 0 otherwise. Show that ν is a right eigenvector of [P] with eigenvalue 1. Hint: Redraw Figure 4.5 for multiple recurrent classes and first show that ν is an eigenvector of [Pn] in the limit.
Text Book: Stochastic Processes: Theory for Applications By Robert G. Gallager.