1. Three white and three black balls are distributed in two urns in such a way that each contains three balls. We say that the system is in state i, i = 0, 1, 2, 3, if the ?rst urn contains i white balls. At each step, we draw one ball from each urn and place the ball drawn from the ?rst urn into the second, and conversely with the ball from the second urn. Let Xn denote the state of the system after the nth step. Explain why {Xn, n = 0, 1, 2, .. .} is a Markov chain and calculate its transition probability matrix.
2. Suppose that whether or not it rains today depends on previous weather conditions through the last three days. Show how this system may be analyzed by using a Markov chain. How many states are needed?