(a) Assume throughout that [P] is the transition matrix of a unichain (and thus the eigenvalue 1 has multiplicity 1). Show that a solution to the equation [P]w - w = r - ge exists if and only if r - ge lies in the column space of [P - I], where [I] is the identity matrix.
(b) Show that this column space is the set of vectors x for which π x = 0. Then show that r - ge lies in this column space.
(c) Show that, with the extra constraint that π w = 0, the equation [P]w - w = r - ge has a unique solution.
Text Book: Stochastic Processes: Theory for Applications By Robert G. Gallager.