1. We import BaggingClassifier and DecisionTreeClassifier from the scikit-learn library. We also import the other required libraries as follows:

from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split

2. Next, we read out the data and take a look at the dimensions:

df_winedata = pd.read_csv('winedata.csv')
df_winedata.shape

3. We separate our features and the response set. We also split our data into training and testing subsets.

X = df_winedata.iloc[:,1:14]
Y = df_winedata.iloc[:,0]

X_train, X_test, Y_train, Y_test = train_test_split(X, Y, random_state=1)

4. We create an instance of the DecisionTreeClassifier class and pass it to the BaggingClassifier():

dt_model = DecisionTreeClassifier(criterion='entropy')
bag_dt_model = BaggingClassifier(dt_model, max_features=1.0, n_estimators=5, 
                                 random_state=1, bootstrap=True)

5. We fit our model to the training data as follows:

bag_dt_model.fit(X_train, Y_train)

6. We can see the score after passing the test data to the model:

bag_dt_model.score(X_test, Y_test)

7. We use the predict function to predict the response variable as follows:

predictedvalues = bag_dt_model.predict(X_test)

8. We will now use a code to plot the confusion matrix. Note that this code has been taken from scikit-learn.org. We execute the following code to create the plot_confusion_matrix() function:

# code from 
# http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html
def plot_confusion_matrix(cm, classes,
                          normalize=False,
                          title='Confusion matrix',
                          cmap=plt.cm.Blues):
    """
    This function prints and plots the confusion matrix.
    """
    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
    plt.colorbar()
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)

    thresh = cm.max() / 2.
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, cm[i, j],
                 horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")

    plt.tight_layout()
    plt.ylabel('Actuals')
    plt.xlabel('Predicted')

9. We use the preceding plot_confusion_matrix() function to plot our confusion matrix:

# This variable holds the class labels of our target variable
target_names = [ '1', '2', '3']

import itertools
from sklearn.metrics import confusion_matrix

# Constructing the Confusion Matrix
cm = confusion_matrix(Y_test, predictedvalues)

# Plotting the confusion matrix
plt.figure(figsize=(3,3))
plot_confusion_matrix(cm, classes=target_names, normalize=False)
plt.show()

The confusion matrix plot looks as follows: