Question

Explain the differences among training sets, validation sets, and test sets.

Please explain the answer in detail and in good hand writing! Thanks a lot!

Answer 1

Solution:

Once we have the training data, you need to split it into three sets:

• Traning set: The data you will use to train your model. This will be fed into an algorithm that generates a model. Said model maps inputs to outputs.
• Validation set: This is smaller than the training set, and is used to evaluate the performance of models with different hyperparameter values. It's also used to detect overfitting during the training stages.
• Test set: This set is used to get an idea of the final performance of a model after hyperparameter tuning. It's also useful to get an idea of how different models (SVMs, Neural Networks, Random forests...) perform against each other.

Now, some important considerations:

• The validation and test sets are usually much smaller than the training set. Depending on the amount of data you have, you usually set aside 80%-90% for training and the rest is split equally for validation and testing. Many things can influence the exact proportion of the split, but in general, the biggest part of the data is used for training.
• The validation and test sets are put aside at the beginning of the project and are not used for training. This might seem obvious, but it's important to remember that they are there to evaluate the performance of the model. Evaluating a model on the data used to train it will make you believe it's performing better than it would in reality.
• All 3 sets need to be representative. This means that all the sets need to contain diverse examples that represent the problem space. For example, in a multiclass classification problem, you want to ensure that all 3 sets contain enough examples of each class. Otherwise, you run the risk of training a model with just a non-representative subset of the data or performing poor validation and testing.

Please give thumbsup or do comment in case of any query. Thanks.