Question

Problem (A1) (20 points): Huffman Coding Consider a message having the 5 symbols (A,B,C,D,E) with probabilities (0.1,0.1,0.2 ,0.2, 0.4), respectively. For such data, two different sets of Huffman codes can result from a different tie breaking during the construction of the Huffman trees. • Construct the two Huffman trees. (8 points) Construct the Huffman codes for the given symbols for each tree. (4 points) Show that both trees will produce the same average code length. (4 points) For data transmission purposes, it is desirable to choose the code set with the minimum variance from the average code length. Compute such variances for the two resulting code sets and determine which of the two sets is preferable according to this criterion? (4 points) Hint: How to compute the variance of the code set for n symbols: Let the probabilities of the symbols be (p1, p2 , ... Pn) and lengths of the symbol codes (number of bits for assigned to each symbol) be (L1, L2 , ...In). The average code length is: <L>=EpL; i=1 The variance of the code length = V =( (L;-<L>) i=1

Answer 1

Data for huffman coding :

Symbols	A	B	C	D	E
Probabilities	0.1	0.1	0.2	0.2	0.4

Construction Huffman trees :

HUFFMAN TREE 1:

First we find the two symbols with least probabilities that is A and B with 0.1 and 0.1 probabilties. We will call it a combined probability AB. We also place 0 on the left branch and 1 on the right branch. These 0's and 1's will help to construct the Huffman codes. Now we have

A: 0.1 B: 0.1

We will again repeat the previous step, dealing only with remaining Probabilities and combines probabilities. In this case now we have 3 minimum probabilities i.e. C:0.2, D:0.2 and AB :0.2. For this first case of Huffman tree we will select C and D as minimum probabilities to break the tie. Now we have

6.2 EA FA A : 0.1 8 : 0.1 2.ܘ : c 5 : 6.2

After repeating the previous steps a few more times until all the probabilities are used we get

E: 0.4 0.6 0.2

HUFFMAN TREE 2 :

First we find the two symbols with least probabilities that is A and B with 0.1 and 0.1 probabilties. We will call it a combined probability AB. We also place 0 on the left branch and 1 on the right branch. These 0's and 1's will help to construct the Huffman codes. Now we have

A: 0.1 B: 0.1

We will again repeat the previous step, dealing only with remaining Probabilities and combines probabilities. In this case now we have 3 minimum probabilities i.e. C:0.2, D:0.2 and AB :0.2. For this first case of Huffman tree we will select C and AB as minimum probabilities to break the tie. Now we have

C: 0.2 0.2 A :0.1 B : 0.1

After repeating the previous steps until all the probabilities are used we get

E: 0.4 0.6 0:0.2 0. 4 C: 0.2 0.2 A :0.1 B: 0.1

HUFFMAN CODES :

Huffman tree 1 : Huffman tree 2 :

A = 1110     A = 100

B = 1111 B = 101

C = 110 C = 110

D = 10 D = 111

E = 0 E = 0

HUFFMAN AVERAGE CODE LENGTH:

Huffman tree 1 :

If we sum the products of the probabilities and the code lengths for this case we get

0.1 *4 + 0.1 *4 + 0.2 *3 + 0.2 *2 + 0.4 *1 = 2.2

Huffman tree 2 :

If we sum the products of the probabilities and the code lengths (<L> = $\tiny \sum_{i=1}^{n}$ piLi) for this case we get

0.1 *3 + 0.1 *3 + 0.2 *33+ 0.2 *3 + 0.4 *1 = 2.2

hence, the average code length of both the trees are same.

MINIMUM VARIANCE :

Huffman tree 1 :

Calculating variance using the given formula:

1/5 [ 0.1(4 - 2.2)2 + 0.1(4 - 2.2)2 + 0.2(3 - 2.2)2 + 0.2(2 - 2.2)2 + 0.4(1 - 2.2)2 ] = 0.272

Huffman tree 2 :

1/5 [ 0.1(3 - 2.2)2 + 0.1(3 - 2.2)2 + 0.2(3 - 2.2)2 + 0.2(3 - 2.2)2 + 0.4(1 - 2.2)2 ] = 0.192

Second set of Huffman codes (i.e. Huffman tree 2 ) is more preferable because it has minimum variance.