G. Ambler has € 10000 available for a second hand car, but would like to buy a fast car that costs € 25000. He needs the money for that car quickly, and would like to increase his capital to € 25000 via a gambling game. To this end, he can play a game in which he is allowed to toss an imperfect (with probability 0.4 for heads) coin three times. For each toss he may bet each amount (in multiples of € 1000 and the amount should be in his possession). He will win the amount (i.e. receives twice the amount of the bet) when he tosses head, and loses his betted amount when he tosses tails. Use stochastic dynamic programming to determine a strategy that maximises the probability of reaching € 25000 after three tosses.
(a) Determine the phases n, states i, decisions d, en optimal valuefunction fn(i) for this stochastic dynamic programming problem.
(b) Give the recurrence relations for the optimal value function.
(c) Determine the optimal policy, and describe in words what this policy does. What is the expected probability of succes?