1. Prove that, if a 3-by-3 transition matrix has the property that its column sums are 1, then (1/3, 1/3, 1/3) is a fixed probability vector. State a similar result for n-by-n transition matrices. Interpret these results for ergodic chains.
2. Is the Markov chain in Example 11.10 ergodic?
3. Is the Markov chain in Example 11.11 ergodic?
4. Consider Example 11.13 (Drunkard's Walk). Assume that if the walker reaches state 0, he turns around and returns to state 1 on the next step and, simi- larly, if he reaches 4 he returns on the next step to state 3. Is this new chain ergodic? Is it regular?