Response to the following problem:
When a tennis player serves, he gets two chances to serve in bounds. If he fails to do so twice, he loses the point. If he attempts to serve an ace, he serves in bounds with probability 3/8 . If he serves a lob, he serves in bounds with probability 7/8 . If he serves an ace in bounds, he wins the point with probability 2/3. With an inbounds lob, he wins the point with probability 1/3 . If the cost is +1 for each point lost and -1 for each point won, the problem is to determine the optimal serving strategy to minimize the (long-run) expected average cost per point.
(a) Formulate this problem as a Markov decision process by identifying the states and decisions and then finding the Cik.
(b) Identify all the (stationary deterministic) policies. For each one, find the transition matrix and write an expression for the (longrun) expected average cost per point in terms of the unknown steady-state probabilities (π0, π1, . . . , πM).
(c) Use your OR Courseware to find these steady-state probabilities for each policy. Then evaluate the expression obtained in part (b) to find the optimal policy by exhaustive enumeration.