Joe and Pete each have 2 cents. They have decided to match pennies. They will each take one of their pennies and flip them. If the pennies match (both heads/both tails), Joe gets Pete's penny; else Pete gets Joe's penny. They will keep repeating until one of them has 4 cents and the other one is broke. They do not realize that all 4 pennies are biased. The probability of tossing a head is 0.6 and a tail is 0.4 for each penny. Let X be a Markov chain where Xn denotes the amount Joe has after the nth play.
a) Give the Markov matrix for X
b) What is the probability that Joe will have four pennies after the second toss?
c) What is the probabilty that Pete will be broke after 3 tosses?
d) What is the probability that the game will be over after the third toss?
e) What is the expected amount of money Pete will have after two tosses?