Question

Question 2: There are two data classes "+" and "-". The training set is shown in Table 4.2 as follows. Please compute the entropy of al. (hint: compute the entropy of the case al =T, and the entropy of the case al = F; then, the entropy of al is the weighted average entropy of the two cases) + + Table 4.2. Data set for Exercise 3. Instance 01 02 03 Target Class TT 1.0 TT 6.0 TF 5.0 IF F 4.0 FT 7.0 FT 3.0 F F 8.0 T F 7.0 FT 5.0 + +

Answer 1

The standered formula for calculation of entropy is used. The formulas are mentioned in the images below. I have explained all the steps in detail. Furthur clarification can be asked via comments.

Total ho. of positive outputs p = 4 Totd no. of nogetue culpets, h=6 [culpat is the Target Class value] For entropy of ate, hete make the folloing table a al Pi hi 7 3 Entropy of at at ginen value T -3 109/2) - 1 log ( 4 = Calculated F . 1 1 4 1 - log(+4) - 4 log (4) below For getting pi for a] = I, me count number of 't freeQIPT. Similazilo, all othes are calculated Formula for entropy of an alterbutt at a quan value t ptn E (A2v). In logolain) ve en bog I Nele that all log is in base et E(92= 7) = -3 log (2) - loge (4) = 0.3258 +0.5 = 0.8258

Here the formula E(A=v) just has two terms as A just takes two distinct values. Like here A=a1 takes two values, that is , T and F. If A takes more values in some other question, then number of terms will be equal to number of distinct vaues taken by A

E[a] = F) = -log(1) - 4 log (45) 00464+00257 - 00721 ECal) = { Pithu. Ela2 = 0) -1 pth VE no. of olistinct values of al. Ea1) = 3+1 0.8258 + 1 + 4 x 0.72) 4+5 - 4+5 + 5x0.7.26 2 4x0.8268 00400 2 0.367 F = 0.767 py of a 1 = 0.767

E(a1) is the required weighted average, Again, here also the number of terms in the formula is will be equal to number of distinct vaues taken by A. Here just two ( Same reason as above).

The final answer is 0.767.