A source has an alphabet {a1, a1, a3, a4} with corresponding probabilities {0.1, 0.2, 0.3, 0.4}.
1. Find the entropy of the source.
2. What is the minimum required average code word length to represent this source for error-free reconstruction?
3. Design a Huffman code for the source and compare the average length of the Huffman code with the entropy of the source.
4. Design a Huffman code for the second extension of the source (take two letters at a time). What is the average code word length? What is the average number of required binary letters per each source output letter?
5. Which is a more efficient coding scheme: the Huffman coding of the original source or the Huffman coding of the second extension of the source?