Ethan Sherwood owns two printing presses, on which he prints two types of jobs. At the beginning of each day, there is a .5 probability that a type 1 job will arrive, a .1 probability that a type 2 job will arrive, and a .4 probability that no job will arrive. Ethan receives $400 for completing a type 1 job and $200 for completing a type 2 job. (Payment for each job is received in advance.) Each type of job takes an average of three days to complete. To model this, we assume that each day a job is in press there is a 1/3 probability that its printing will be completed at the end of the day. If both presses are busy at the beginning of the day, any arriving job is lost to the system. The crucial decision is when (if ever) Ethan should accept the less profitable type 2 job. Ethan's goal is to maximize expected discounted profit (use β = .90).
a. Use the policy iteration method to determine an optimal stationary policy.
b. Use linear programming to determine an optimal stationary policy.
c. Perform two iterations of value iteration.