We read the 20 colored images from the desktop and change them to gray to maintain compatibility with the data and model that we have used so far. Take a look at the following code:

setwd("~/Desktop/image20")
temp = list.files(pattern = "*.jpg")
mypic <- list()
for (i in 1:length(temp))  {mypic[[i]] <- readImage(temp[[i]])}
for (i in 1:length(temp))  {mypic[[i]] <- channel(mypic[[i]], "gray")}
for (i in 1:length(temp)) {mypic[[i]] <- 1-mypic[[i]]}
for (i in 1:length(temp)) {mypic[[i]] <- resize(mypic[[i]], 28, 28)}
par(mfrow = c(5,4), mar = rep(0, 4))
for (i in 1:length(temp)) plot(mypic[[i]])

As seen previously, we also resize all 20 images to 28 x 28, and the resulting 20 images to be classified are as follows:

As we can observe from the preceding plot, there are two fashion items belonging to each of the 10 categories of the fashion-MNIST data:

# Reshape and row-bind
for (i in 1:length(temp)) {mypic[[i]] <- array_reshape(mypic[[i]], c(1,28,28,1))}
new <- NULL
for (i in 1:length(temp)) {new <- rbind(new, mypic[[i]])}
str(new)

OUTPUT

num [1:20, 1:784] 0.0458 0.0131 0 0 0 ...

We reshape images in the required dimensions and then row-bind them. Looking at the structure of new, we see a 20 x 784 matrix. However, to get an appropriate, structure we will reshape it further to 20 x 28 x 28 x 1, as shown in the following code:

# Reshape
newx <- array_reshape(new, c(nrow(new),28,28,1))
newy <- c(0,4,5,5,6,6,7,7,8,8,9,0,9,1,1,2,2,3,3,4)

We reshape new to get the appropriate format, and save the result in newx. We use newy to store the actual labels for the 20 fashion items.

Now, we are ready to use the prediction model, and create a confusion matrix as shown in the following code:

# Confusion matrix for 20 images
pred <- model %>%   predict_classes(newx)
table(Predicted=pred, Actual=newy)

OUTPUT
         Actual
Predicted 0 1 2 3 4 5 6 7 8 9
        0 1 0 0 0 0 0 0 0 0 0
        1 0 1 0 0 0 0 0 0 0 0
        2 0 0 1 0 0 0 0 0 0 0
        3 1 1 0 2 0 0 0 0 0 2
        4 0 0 1 0 1 0 0 0 0 0
        5 0 0 0 0 0 0 0 1 0 0
        6 0 0 0 0 0 0 2 0 0 0
        8 0 0 0 0 1 2 0 1 2 0

We observe from the numbers on the diagonal that only 10 items are correctly classified out of 20. This translates to a low accuracy of only 50%, compared to over 90% accuracy observed for the train and test data.

Next, we summarize these predictions in the form of a plot that includes the prediction probabilities, predicted class, and actual class, using the following code:

# Images with prediction probabilities, predicted class, and actual class 
setwd("~/Desktop/image20")
temp = list.files(pattern = "*.jpg")
mypic <- list()
for (i in 1:length(temp))  {mypic[[i]] <- readImage(temp[[i]])}
for (i in 1:length(temp))  {mypic[[i]] <- channel(mypic[[i]], "gray")}
for (i in 1:length(temp)) {mypic[[i]] <- 1-mypic[[i]]}
for (i in 1:length(temp)) {mypic[[i]] <- resize(mypic[[i]], 28, 28)}
predictions <-  predict_classes(model, newx)
probabilities <- predict_proba(model, newx)
probs <- round(probabilities, 2)
par(mfrow = c(5, 4), mar = rep(0, 4))
for(i in 1:length(temp)) {plot(mypic[[i]])
         legend("topleft", legend = max(probs[i,]),  
                bty = "n",text.col = "white",cex = 2)
         legend("top", legend = predictions[i],  
                bty = "n",text.col = "yellow", cex = 2) 
         legend("topright", legend = newy[i],  
                bty = "",text.col = "darkgreen", cex = 2) }

The preceding plot summarizes the performance of the classification model with the help of prediction probabilities, predicted class, and actual class (model-one):

In the preceding plot, the first number in the top-left position is the prediction probability, the second number in the top-middle position is the predicted class, and the third number in the top-right position is the actual class. Looking at some of these misclassifications, what stands out is the fact that surprisingly, all images of the sandal (item 5), sneakers (item 7), and ankle boots (item 9) are incorrectly classified. These categories of images were classified with high accuracy in the training as well as test data. These six misclassifications have contributed to the significantly low accuracy value.

Two key aspects of what we have done so far can now be summarized as follows:

• The first one is what we would generally expect—model performance with the test data is usually lower compared to what is observed with the training data. 
• The second one is a bit of an unexpected outcome. The 20 fashion item images that were downloaded from the internet had significantly reduced accuracy with the same model.

Let's see whether we can devise a strategy or make changes to the model to obtain better performance. We plan to have a closer look at the data and find a way to translate the performance that we saw with training and test data, if possible, to the 20 new images.