The code for obtaining the loss, accuracy, and confusion matrix based on the test data is as follows:

# Loss and accuracy
model %>% evaluate(testx, testy)
$loss
[1] 16.4562
$acc
[1] 0.2325

# Confusion matrix
pred <- model %>% predict_classes(testx)
table(Predicted = pred, Actual = data$test$y[1:2000,])
         Actual
Predicted  0  1  2  3  4  5  6  7  8  9
        0 82 24 29 17 16 10 17 19 67 19
        1 16 65 20 26 18 21 26 26 33 53
        2 10  0 26 20 20 18 14  5  1  2
        3  6  5  8 21 12 22  9 12  9  3
        4  4  8 22 11 22 16 25  9  6  4
        5  5  7 12 29 17 29  9 19  4  9
        6  6  6 20 17 23 15 51 25  6 13
        7  3 10 10 15 21 16 11 37  3  5
        8 34 22 20 12 22  2  7  7 61 24
        9 30 51 28 31 27 36 47 34 27 71

From the preceding output, the following observations can be made:

• The loss and accuracy values for the test data are 16.456 and 0.232, respectively.
• These results are not as impressive as what we observed for the training data due to the overfitting problem.

Although we can try and develop a deeper network in an effort to improve image classification results or to try and increase training data to provide more samples to learn from, here, we will make use of pretrained networks to obtain better results.