1. Two players match pennies and have between them a total of 5 pennies. If at any time one player has all of the pennies, to keep the game going, he gives one back to the other player and the game will continue. Show that this game can be formulated as an ergodic chain. Study this chain using the program ErgodicChain.
2. Calculate the reverse transition matrix for the Land of Oz example (Exam- ple 11.1). Is this chain reversible?
3. Give an example of a three-state ergodic Markov chain that is not reversible.
4. Let P be the transition matrix of an ergodic Markov chain and P∗ the reverse transition matrix. Show that they have the same fixed probability vector w.