Loss and accuracy calculation

The code for obtaining loss and accuracy values using the test data along with the output is shown in the following:

# Model evaluation
model %>%
evaluate(test, testLabels)

OUTPUT
## $loss
## [1] 0.4439415
##
## $acc
## [1] 0.8424544

As you can see in the preceding code, using the evaluate function, we can obtain loss and accuracy values as 0.4439 and 0.8424 respectively. Using colSums(testLabels), we can find that there are 460, 94, and 49 cases of normal, suspect, and pathological patients respectively in the test data. Converting these numbers to percentages using a total of 603 samples in the test data, we obtain 76.3%, 15.6%, and 8.1% respectively. The highest number of samples belongs to the normal category of patients, and we can use 76.3% as a benchmark for the model performance. If we do not use any model and simply classify all cases in the test data as belonging to the normal category of patients, then we will still be correct about 76.3% of the time since we will be right about all normal patients and incorrect about the other two categories.

In other words, the accuracy of our prediction will be as high as 76.3%; therefore, the model that we develop here should perform at least better than this benchmark number. If it functions below this number, then it is not likely to be of much practical use. Since we get an accuracy of 84.2% for the test data, we are definitely doing better than the benchmark value, but clearly we must also try to improve our model in order to perform even better. To do that, let's dig even deeper and learn about model performance for each category of the response variable with the help of a confusion matrix.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset