To further explore any changes in the image-classification performance of the model that may not be obvious from a graphical summary, let's look at some numerical summaries:

1. We will look at the results based on the training data first, and will make use of the following code:

# Loos and accuracy
model %>% evaluate(trainx, trainLabels)
OUTPUT
12/12 [==============================] - 0s 198us/step
$loss
[1] 0.03438224643

$acc
[1] 1

# Confusion matrix
pred <- model %>%   predict_classes(trainx)
table(Predicted=pred, Actual=trainy)

OUTPUT
         Actual
Predicted 0 1 2
        0 3 0 0
        1 0 3 0
        2 0 0 3

From the preceding output, we can see that the loss value has now reduced to 0.034 and the accuracy is maintained at 1.0. We obtain the same confusion matrix results for the training data as we did earlier as all nine images are correctly classified by the model, which gives an accuracy level of 100%.

2. To look more deeply at the classification performance of the model, we make use of the following code and output:

# Prediction probabilities
prob <- model %>%   predict_proba(trainx) 
cbind(prob, Predicted_class = pred, Actual = trainy)

OUTPUT        
                                                   Predicted_class Actual
 [1,] 0.97638195753098 0.0071088117547 0.01650915294886     0         0
 [2,] 0.89875286817551 0.0019298568368 0.09931717067957     0         0
 [3,] 0.98671281337738 0.0004396488657 0.01284754090011     0         0
 [4,] 0.00058794603683 0.9992876648903 0.00012432398216     1         1
 [5,] 0.00005639552546 0.9999316930771 0.00001191849515     1         1
 [6,] 0.00020669832884 0.9997472167015 0.00004611289114     1         1
 [7,] 0.03771930187941 0.0022936603054 0.95998704433441     2         2
 [8,] 0.08463590592146 0.0022607713472 0.91310334205627     2         2
 [9,] 0.03016609139740 0.0019471622072 0.96788680553436     2         2

From the preceding prediction probabilities that we obtain as an output of the training data, we can make the following observations:

• Correct classifications are now made with higher probability values than the earlier model.
• The lowest correct classification probability based on the second row is 0.899.
• Therefore, this model seems to be more sure when correctly classifying images compared to what was observed with the previous model.

3. Now let's see whether this improvement is also seen with the test data. We will use the following codes and output:

# Loss and accuracy
model %>% evaluate(testx, testLabels)

OUTPUT

6/6 [==============================] - 0s 345us/step
$loss
[1] 0.40148338683

$acc
[1] 0.8333333

# Confusion matrix
pred <- model %>%   predict_classes(testx)
table(Predicted=pred, Actual=testy)

OUTPUT
         Actual
Predicted 0 1 2
        0 2 0 0
        1 0 1 0
        2 0 1 2

As indicated in the preceding output, the test data loss and accuracy values are 0.401 and 0.833 respectively. We do see some improvement in loss values; however, the accuracy value is again the same as it was earlier. Looking at the confusion matrix, we can see that this time, an image of a car is misclassified as an airplane. Therefore, we do not see any major differences based on the confusion matrix.

4. Next, let's review the prediction probabilities using the following code and its output:

# Prediction probabilities
prob <- model %>%   predict_proba(testx) 
cbind(prob, Predicted_class = pred, Actual = testy)

OUTPUT
                                                   Predicted_class Actual
[1,] 0.7411330938339 0.015922509134 0.242944419384           0      0
[2,] 0.7733710408211 0.021422179416 0.205206796527           0      0
[3,] 0.3322730064392 0.237866103649 0.429860889912           2      1
[4,] 0.0005808877177 0.999227762222 0.000191345287           1      1
[5,] 0.2163420319557 0.009395645000 0.774262309074           2      2
[6,] 0.1447975188494 0.002772571286 0.852429926395           2      2

Using the prediction probabilities for the test data, we can make the following two observations:

• We see a consistently similar pattern to the one that we observed based on the results from the training data. This model correctly classifies images in the test data with higher probabilities (0.74 to 0.99) than the earlier model (0.53 to 0.98).

• For the fourth sample in the test data, the model seems to be confused between the image of a bicycle and an airplane, when in reality, this image is of a car.

Therefore, overall, we have observed that by developing a deeper neural network, we are able to improve the model's performance. The improvement in the performance was not obvious from the accuracy calculation; however, the calculation of prediction probabilities allowed us to develop better insights and compare model performance.