First, we will evaluate the model's performance using training data. Then, we will use the model to predict the class representing each of the 50 authors. The code for evaluating the model is as follows:
# Loss and accuracy
model %>% evaluate(trainx, trainy)
$loss
[1] 1.45669
$acc
[1] 0.5346288
Here, we can see that, by using the training data, we obtain a loss value of about 1.457 and an accuracy of about 0.535. Next, we use the model to make a prediction about the classes for the articles in the training data. These predictions are then used to arrive at an accuracy reading for each of the 50 classes representing 50 authors. The code that's used to achieve this is as follows:
# Prediction and confusion matrix
pred <- model %>% predict_classes(trainx_org)
tab <- table(Predicted=pred, Actual=trainy_org)
(accuracy <- 100*diag(tab)/colSums(tab))
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
82 40 30 10 54 46 54 82 8 56 46 36 76 18 52 90 50 56 8 66 80 24 30 46 32
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
46 88 62 22 64 76 2 74 88 72 74 76 86 70 60 86 38 32 0 48 6 24 76 8 22
In the preceding code, to conserve space, we haven't printed the output of the confusion matrix since it will be a 50 x 50 matrix. However, we have used information in the confusion matrix to arrive at the model's accuracy by correctly predicting each author based on the articles they have written. The output that we've obtained is as follows:
The preceding bar plot provides further insight into the model's performance with respect to each author:
- The accuracy of correctly classifying an author has the highest value of 90% for author 15.
- The accuracy of correctly classifying an author has the lowest value of 0% for author 43.
- This model struggles to correctly classify articles from certain authors, such as those labeled 3, 8, 18, 31, 43, 45, and 48.
Having assessed the model using training data, we will repeat this process with the test data.