Mr. Smith reads the local newspaper every day after which he puts it on the top of the pile on his desk. He tidies up his desk every afternoon. If there are more than 5 newspapers in the pile then he puts the entire pile into the garbage, otherwise he does so with probability 1/3 (independently of his other decisions). Consider the number of newspapers Xn in the pile in the evening of day n, that is, after he tidied his desk.
(a) Can you model the process with a Markov chain? If yes, what are the states and the transition probabilities?
(b) Sunday evening the pile was empty. What is the probability that Thursday evening there is exactly one newspaper in the pile?