We will now use the test data to obtain loss and accuracy values for the model using the following code:
# Evaluate
model %>% evaluate(test_x, test_y)
$loss
[1] 0.3997277
$acc
[1] 0.81992
As seen from the preceding output, for the test data, we obtain a loss value of 0.399 and an accuracy of about 0.819. These values, as expected, are slightly inferior to those obtained for the train data. However, they are close enough to results based on the train data to consider this model behavior consistent.
The code to obtain a confusion matrix using test data is as follows:
# Confusion Matrix
pred1 <- model %>$ predict_classes(text_x)
table(Predicted=pred1, Actual=imdb$test$y)
Actual
Predicted 0 1
0 9159 1161
1 3341 11339
From the confusion matrix shown above, the following observations can be made:
- The confusion matrix based on predictions using the test data shows a similar pattern that we observed earlier for the training data.
- This model also seems to perform better when accurately classifying positive movie reviews (at a rate of about 90.7%), compared to correctly classifying negative reviews (at a rate of about 73.3%).
- Hence, the model continues to show bias in the performance when correctly classifying positive movie reviews.
In the next section, we will carry out some experimentation to explore possible improvements for the model's movie review sentiment classification performance.