Change of base:- What is the general formula for entropy Hb(S) using base-b logarithms in terms of entropy Ha(S) using base-a logarithms?
(A counterfeit coin*) A certain counterfeit half-dollar has a probability p of being heads and 1 - p of being tails, where p = 1/2. John flips the coin and tells Jane the outcome.
(a) What is the entropy associated with the statement that John makes to Jane?
(b) On the next flip, Jane realizes that there is a probability q that after the flip John will reverse the coin before reporting the (altered) outcome. Is the new entropy of John's statement less than, equal to, or greater than that of part (a)? Prove your answer. This says something about the effect of mixing two sources.