We start by looking at loss and accuracy values based on the test data:

# Model evaluation
model %>% evaluate(testx, testy)

$loss  0.240465
$acc   0.9226

We observe that loss is higher and accuracy is lower compared to the values obtained from the train data. This is as expected considering a similar situation with validation data that we observed earlier.

Confusion matrix for test data is provided as follows:

# Prediction and confusion matrix
pred <- model %>% predict_classes(testx)
table(Predicted=pred, Actual=mnist$test$y)

OUTPUT
         Actual
Predicted   0   1   2   3   4   5   6   7   8   9
        0 878   0  14  15   0   0  91   0   0   0
        1   1 977   0   2   1   0   1   0   2   0
        2  22   1 899   9  55   0  65   0   2   0
        3  12  14   6 921  14   0  20   0   3   0
        4   2   5  34  26 885   0  57   0   0   0
        5   1   0   0   0   0 988   0   8   1   6
        6  74   1  43  23  43   0 755   0   2   0
        7   0   0   0   0   0   6   0 969   3  26
        8  10   2   4   4   2   0  11   0 987   1
        9   0   0   0   0   0   6   0  23   0 967

From the preceding confusion matrix, we can make the following observations:

• This model is most confused regarding item 6 (shirt), with 91 instances where it classifies fashion items as item-0 (t-shirt/top).
• The best image recognition and classification performance is for item-5 (sandal), with 988 correct predictions out of 1,000.
• Overall, the confusion matrix exhibits a similar pattern to the one we observed with the training data.

Looking at prediction probabilities for the first five items in the test data, we observe that all five predictions are correct. Prediction probability for all five items is quite high:

# Prediction probabilities
prob <- model %>% predict_proba(testx) 
prob <- round(prob, 3)
cbind(prob, Predicted_class = pred, Actual = mnist$test$y)[1:5,]

OUTPUT
                                   Predicted_class Actual 
[1,] 0.000 0 0.000 0 0.000 0 0.000 0 0 1     9         9 
[2,] 0.000 0 1.000 0 0.000 0 0.000 0 0 0     2         2 
[3,] 0.000 1 0.000 0 0.000 0 0.000 0 0 0     1         1 
[4,] 0.000 1 0.000 0 0.000 0 0.000 0 0 0     1         1 
[5,] 0.003 0 0.001 0 0.004 0 0.992 0 0 0     6         6

Now, with sufficiently high classification performances with both the training and testing data in terms of accuracy, let's see if we can do the same with the 20 images of fashion items with which we started this chapter.