Prediction probabilities for training data

We can now look at the probabilities of the three classes for all nine images in the training data that this model provides. The following is the code:

# Prediction probabilities
prob <- model %>% predict_proba(trainx)
cbind(prob, Predicted_class = pred, Actual = trainy)

OUTPUT
Predicted_class Actual [1,] 0.9431666135788 0.007227868307 0.049605518579 0 0 [2,] 0.8056846261024 0.005127847660 0.189187481999 0 0 [3,] 0.9556384682655 0.001881886506 0.042479615659 0 0 [4,] 0.0018005876336 0.988727569580 0.009471773170 1 1 [5,] 0.0002136278927 0.998095452785 0.001690962003 1 1 [6,] 0.0008950306219 0.994426369667 0.004678600468 1 1 [7,] 0.0367377623916 0.010597365908 0.952664911747 2 2 [8,] 0.0568452328444 0.011656147428 0.931498587132 2 2 [9,] 0.0295505002141 0.011442330666 0.959007143974 2 2

In the preceding output, the first three columns show the probability of an image being a bicycle, car, or airplane, and the total of these three probabilities is 1. We can make the following observations from the output:

  • The probabilities for the first image in the training data are 0.943, 0.007, and 0.049 for bicycle, car, and airplane respectively. Since the highest probability is for the first class, the predicted class based on the model is 0 (for bicycle), and this is also the actual class of the image.
  • Although all 9 images are correctly classified, the probability of correct classification varies from 0.806 (image 2) to 0.998 (image 5).
  • For the car images (rows 4 to 6), the probability of correct classification ranges from 0.989 to 0.998 and is consistently high for all three images. Therefore, this classification model gives its best performance when classifying car images.
  • For bicycle images (rows 1 to 3), the probability of correct classification ranges from 0.806 to 0.956, which indicates some difficulty in correctly classifying bicycle images.
  • For the second sample, which represents a bicycle image, the second-highest probability is 0.189 of being an airplane image. Clearly, this model is little bit confused when it comes to deciding whether this image is a bicycle or an airplane.
  • For the airplane images (rows 7 to 9), the probability of correct classification ranges from 0.931 to 0.959, which is  also consistently high for all three images.

Looking at prediction probabilities allows us to dig deeper into the classification performance of the model, which cannot be obtained only by looking at the accuracy value. However, while good performance with training data is necessary, it is not sufficient to arrive at a reliable image-classification model. When a classification model suffers from an overfitting problem, we have difficulty replicating good results based on training data on test data that the model has not seen. Therefore, a real test of a good classification model is when it performs well with the test data. Let's now review the image-classification performance of the model for test data.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset