Consider a random walk on a square with its centroid as shown in Fig. 1.6. Suppose that at each State i(i D 0; 1; 2; 3; 4), the transition probabilities to other adjacent states are all equal. While the probability of staying at the same state in the next transition is assumed to be zero.
(a) Show that the Markov chain of the random walk is irreducible and all the states are recurrent.
(b) Find the state-state probability distribution
π =( π0, π1, π2, π3, π4)
of the Markov chain where
