1) A discrete memoryless source has a symbol alphabet of |A| = 10. Determine the upper and lower bounds on its entropy.
2) Let X be a random variable whose entropy H(X) is 8 bits. Suppose that Y(X) is a deterministic function that takes on a different value for each value of X.
What is H(Y)?
What is H(Y|X)?
What is H(X|Y)?
What is I(X,Y)?
Suppose now that the deterministic function Y(X) is not invertible; so that different values of X may correspond to the same value of Y(X). In this case, what can be said about H(Y), and also about H(X|Y)?
3) A dishonest gambler has a loaded die which turns up the number 1 with probability 2/3 and the numbers 2 to 6 with probability 1/15 each. Unfortunately, he left his loaded die in a box with two honest dice and cannot tell them apart. He picks one die (at random) from the box, rolls it once, and the number 1 appears.
Conditional on this result, what is the probability that he picked up the loaded die?
He now rolls the die once more and it comes up 1 again. What is the possibility after this second rolling that he has picked the loaded die?
Repeat the above, but assume that the first outcome was a 3 and the second was a 1.
4) A dishonest gambler has a loaded die which turns up the number 1 with probability 2/3 and the numbers 2 to 6 with probability 1/15 each. Unfortunately, he left his loaded die in a box with two honest dice and cannot tell them apart. He picks one die (at random) from the box, rolls it once, and the number 1 appears.
Conditional on this result, what is the probability that he picked up the loaded die?
He now rolls the die once more and it comes up 1 again. What is the possibility after this second rolling that he has picked the loaded die?
Repeat the above, but assume that the first outcome was a 3 and the second was a 1.
5) A dishonest gambler has a loaded die which turns up the number 1 with probability 2/3 and the numbers 2 to 6 with probability 1/15 each. Unfortunately, he left his loaded die in a box with two honest dice and cannot tell them apart. He picks one die (at random) from the box, rolls it once, and the number 1 appears.
Conditional on this result, what is the probability that he picked up the loaded die?
He now rolls the die once more and it comes up 1 again. What is the possibility after this second rolling that he has picked the loaded die?
Repeat the above, but assume that the first outcome was a 3 and the second was a 1.