1. Suppose fXng is a stationary Markov chain. Prove that for all n and all xi ;i D 0; 1; :::;n C 2; P.XnC2 D xnC2; XnC1 D xnC1 jXn D xn ; Xn-1 D xn-1;::: ; X0 D x0/ D P.XnC2 D xnC2; XnC1 D xnC1 jXn D xn/.
2. ( What the Markov Property Does Not Mean). Give an example of a stationary Markov chain with a small number of states such that P.XnC1 D xnC1 jXn xn; Xn-1 xn-1;::: ; X0 x0/ D P.XnC1 D xnC1 jXn xn /is not true for arbitrary x0; x1;::: ; xnC1.
3. (Ehrenfest Urn). Consider the Ehrenfest urn model when there are only two balls to distribute.
(a) Write the transition probability matrix P.
(b) Calculate P 2;P 3.
(c) Find general formulas for P 2k ;P 2kC1.