Question

Suppose you are given the true data generating distribution, D, for a binary classification problem (so the true output y is either +1 or 1).

(a) If your loss is absolute loss (|y f (x)), is the Bayes optimal classifier still optimal? Explain your reasoning or show a counterexample.

(b) If your loss is squared loss ((y f (x))2), is the Bayes optimal classifier still optimal? Explain your reasoning or show a counterexample.

Answer 1

(Assuming that true output is either 1 or -1)

As you are doing binary classification, loss functions like absolute loss and squared loss should not be chosen because of the fact that these functions are used to minimize the difference between true values and predicted values and thus should be used in the regression problems. If you are going to use these loss functions for classification purposes they will not give an accurate measure about how your model is performing on the given dataset.