Question: At the beginning of each day, a piece of equipment is inspected to determine its working condition, which is classified as state 1 = new, 2, 3, or 4 = broken. Suppose that a broken machine requires three days to fix. To incorporate this into the Markov chain we add states 5 and 6 and write the following transition matrix:
(a) Find the fraction of time that the machine is working.
(b) Suppose now that we have the option of performing preventative maintenance when the machine is in state 3, and that this maintenance takes one day and returns the machine to state 1. This changes the transition probability to
Find the fraction of time the machine is working under this new policy.