Question

I have some binary classifcation problem with ~30 features and 200 observations. I've used L1 (Lasoo) and L2 (Ridge) to run logistic regression and accuracy, prescion and confusion matrix on both training and test data. From there, have found the best hyper parameter from a selection and have the same output (precision, accuracy and confusion matrix) from this. How do I comment on signs of overfitting or underfitting? Is there more analysis or graphing that is required to answer this?

Answer 1

Comment on signs of overfitting or underfitting:

Overfitting: Normally overfiitting happens when it learns about details and noise related to it in the training to an extent that it negative impact (performance) on new data.

overfitting is more likely that have more flexibility on learning a target function,it includes various parameters and techniques to limit and constrain how much it will learn.

For example:Decision trees which are very flexible and is related to overfitting data.This problem can be address by pruning a tree.

Underfitting: A model that can neither model the training data nor generalise to new data refers to underfitting. underfitting is not suitable for any purpose due to poor performance.However it does provide a good contrast to overfitting.

we can limit overfitting through following techniques:

1:use resampling technique to estimate accuracy.

2.store validation dataset.

yes few more analysis that are present to answer this that are as follows :

1.Regression Analysis.

2.Validation Set.

Kindly rate my answer .Thankyou