Question

Which type of ensemble learning is likely to be more susceptible to noise – bagging or boosting? Explain your reasoning

Answer 1

In machine learning ,Ensemble learning involves using multipe algorithms to obtain bettre performance than could be from any of the contituent learning algoeithms alone.Ensemble learning contructs a set of multiple hypothesis as compared to ordinary machine learning approach which try to learn one hypothesis from the training data.Ensemble laerning is used to improve the performance of a model or reduce the chance to choose a poor model.An Ensemble based system is obtained by combining diverse models(classifiers),therefore they are called multiple classifier system or ensemble system.The advantage of using the ensemble based system is that by ensembling all the classifiers that are having the same performance on the validation data instead of choosing a particular one and by combining their output can reduce the risk of selection of poorly performing indentifier.An Ensmeble contains numbers of learners called base learners.

An Ensemble is created in two steps: first a number of base learners are produced, the generation of base learner has an effect on the generation of sebsequent learners.

Second the base learners are combined to use.to get a good ensemble the base learners should be as accurate as possible

BAGGING AND BOOSTING

Bagging and Boosting are both ensemble learning Algorithms.

Bagging is also called bootstrap aggregating.In bagging different training data subsets are obtained from the entire training dataset.Each training data subset is used to train a different classifier of the same type.Individual Classifiers are then obtained by taking majority vote of their decisions.

Boosting also creates ensemble of classifiers by resampling the data which are then combined by majority voting.However in boosting resampling is strategically done in order to provide most informative training data for each consecutive classifier.

Among the two approaches boosting is more likely to affected by noise since boosting involves involves over fitting the training sets since the training set may be over emphasizing examples that are noise. boosting is affected by noise because of two reasons:

• Their method for updating the probabilities may be over emphasizing noisy examples.
• The classifiers are combined using weighted voting, optimizing the combining weights can lead to over fitting while an unweighted voting scheme is often resilient to over fitting

Although in some cases boosting has yielded better results than bagging but it tends to more likely over fit the training data because of which it more likely to be affected by noise.