Question

1. With reasons, explain if following are examples of good or bad machine learning practice:

   1. Ammar is solving a regression problem. He has a dataset of 500. He splits the dataset into training (60%), validation (20%) and testing (20%). Ammar fits multiple models on the dataset. The quadratic and cubic models are very close in their validation errors with cubic having smaller validation error so Ammar chooses the cubic model.

Answer 1

Yes it is a good machine learning practice for the following with some reservations

• It is better to use 80-20 rule and not 60-20-20 .Amar has partitioned the data set in 60% training, 20% validation and 20% testing data set. This is also a good practice but since the data set is very small (only 500) so it would have better to keep only two partitions as 80% training and 20% validation data set
• this is a regression problem where outcome is quantitative but non linear. Which means its not a straight line between x and y. So here we need to fit the curves.

Fitting the curves means you need to increase the degree of your polynomial equations. Which means now:

Y = mX + c will be changed to something like
Y = m1X1 + m2X^2 + m3X^3 +m4X^4 ...... + mnX^n

So Y = mX + c is linear model
Y = m1X^1 + m2X^2 is squared model

Y = m1X^1 + m2X^2 + m3X^3 is a cubic model and so on.

So consider an assumption duing linear model amaar got 95% accuracy, in square he got 96% accuracy in cubic model he got 99% accuracy. But after cubic model there no change in accuracy even after increasing the degree of independent variables to fit the curve. Hence he used cubic model.