In this first experiment, we will use the adam optimizer when compiling the model. At the time of training the model, we will also increase the number of epochs to 20.
The plot of the accuracy and loss values after training the model is as follows:
The preceding loss and accuracy plot for this model shows that the values related to the training data are flat after about six epochs. For the validation data, the loss values show a gradual increase, whereas the accuracy values are flat after the third epoch.
The code for obtaining the loss, accuracy, and confusion matrix for the test data is as follows:
# Loss and accuracy
model %>% evaluate(testx, testy)
$loss
[1] 4.005393
$acc
[1] 0.7715
# Confusion matrix
pred <- model %>% predict_classes(testx)
table(Predicted = pred, Actual = data$test$y[1:2000,])
Actual
Predicted 0 1 2 3 4 5 6 7 8 9
0 136 0 20 4 2 0 1 5 2 4
1 3 177 1 1 0 0 0 0 2 26
2 7 0 124 2 3 1 3 0 1 0
3 2 0 4 80 7 6 7 2 1 0
4 3 1 18 9 151 4 8 9 0 0
5 2 0 3 58 3 152 4 5 0 3
6 3 2 8 22 8 8 190 0 6 2
7 1 0 14 18 22 14 2 172 0 0
8 36 11 3 5 2 0 1 0 205 12
9 3 7 0 0 0 0 0 0 0 156
# Accuracy for each category
100*diag(tab)/colSums(tab)
0 1 2 3 4
69.38776 89.39394 63.58974 40.20101 76.26263
5 6 7 8 9
82.16216 87.96296 89.11917 94.47005 76.84729
From the preceding output, we can make the following observations:
- The loss and accuracy of the test data are 4.005 and 0.772, respectively.
- These results are marginally better than they are for model_two.
- The confusion matrix shows a somewhat different image classification pattern compared to the previous model.
- The best classification results are obtained for category 8 (ship), with 205 correct image classifications out of 217 (94.5% accuracy).
- The lowest classification performance is for category 3 (cat), with 80 correct predictions out of 199 (40.2% accuracy).
- The worst misclassification is of 58 images from category 3 (cat) when they are misclassified as category-5 (dog).
Next, we will carry out an experiment with hyperparameter tuning.