Question

Implement Knn+ ensemble method using java or python by taking a data set of your choice.
please provide the data set used and the program with comments explaining steps.

Answer 1

Dataset :- We are using iris dataset for this task . This dataset suitable for classification task so our model KNN. It contains various features like petal_length, petal_width, sepal_length and sepal_width and class of that flower like setosa etc...,

Bagging is nothing but bootstrap aggregating . It averages all predictions of our models and thus reduces variance i.e.. overfitting.

Code:-

# KNN + Bagging

# importing all necessary libraries

from sklearn import model_selection

# we are going to use bagging ensemble method

from sklearn.ensemble import BaggingClassifier

from sklearn.neighbors import KNeighborsClassifier

import pandas as pd

from sklearn import datasets

import matplotlib.pyplot as plt

# Using Kfold cross validation technique to work on different subsets of data

from sklearn.model_selection import KFold

# loading built in dataset from sklearn that is iris dataset

iris = datasets.load_iris()

# feature matrix

X = iris.data

# target vector

Y = iris.target

# initializing the base classifier

base_classifier = KNeighborsClassifier()

# no. of base classifiers to be used for bagging technique

numberOf_trees = 500

# splitting data into 10 parts

kf = KFold(n_splits=10)

# bagging classifier

model = BaggingClassifier(base_estimator = base_classifier, n_estimators = numberOf_trees) # it will run all 500 models and aggregate them .

result = model_selection.cross_val_score(model, X, Y,cv=kf)

print(result.mean()) # printing accuracy

output:-