In the application examples that we've provided so far in this book, after we developed a classification or prediction deep network model, we carried out visualizations to view the overall performance of the models. These assessments are done using training and test data. The main idea behind such an assessment is to obtain an overall or global understanding of the model's performance. However, there are situations where we want to obtain a deeper understanding and also interpretations for a specific prediction. For example, we may be interested in understanding the main features or variables that have influenced a specific prediction in the test data. Such "local" interpretations are the focus of a package called Local Interpretable Model-Agnostic Explanations, or LIME. LIME can help provide deeper insights into each prediction.

The code for carrying out visualization using LIME for the model we developed in Keras is as follows:

# LIME package
library(lime)

# Using LIME with keras
model_type.keras.engine.sequential.Sequential <- 
function(x, ...) {"classification"}
predict_model.keras.engine.sequential.Sequential <- 
  function(x,newdata,type, ...) {p <- predict_proba(object=x, x=as.matrix(newdata))
         data.frame(p)}

# Create explainer using lime
explainer <- lime(x = data.frame(training), 
             model = model, 
             bin_continuous = FALSE)

# Create explanation
explanation <- explain(data.frame(test)[1:5,],  
                  explainer    = explainer, 
                  n_labels     = 1,  
                  n_features   = 4,  
                  kernel_width = 0.5)
testtarget[1:5]
[1] 0 0 0 2 2

As shown in the preceding code, we use two functions to be able to use LIME with the Keras model. In the first function, we indicate that we will be working with a classification model. The second function obtains prediction probabilities. In this section, we will use model_one from Chapter 2, Deep Neural Networks for Multi-Class Classification. Then, we'll use the lime function with the training data, the model (that is, model_one), and specify the binning of continuous variables as FALSE. The resulting explainer is used with the explain function, where we will specify the number of labels to use as one and specify the number of most important features to use for each case as four. We specify the kernel width as 0.5. We can also see that the first three patients in the test data have the class labeled as 0, indicating that they belong to the normal patient category. Similarly, the 4th and 5th patients in the test data have been labeled as 2, indicating that they belong to the pathological patient category.

We obtained the following plot using plot_features(explanation):

The preceding plot provides individual plots for the first five patients in the test data. Here are some of the observations that can be made from this plot:

• All five patients have been correctly classified.
• The first three patients have been classified as belonging to a class labeled as 0, representing a normal patient.

• The remaining two patients are classified as belonging to a class labeled as 2, representing a pathological patient.
• The prediction probability for the first three cases is 0.97 or above and the prediction probability for the 4th and 5th patients is 0.72 and above.
• This plot depicts the four most important features that have contributed to the specific classification of each patient.
• Features with blue bars support the model's conclusion, whereas features with red bars contradict the model's conclusion for each patient.
• Higher values for the X8, X10, and X20 variables seem to have a higher influence on a patient being classified as pathological.
• Higher values for the X12 variable seems to influence a patient being classified as normal.

The following heatmap can be obtained using plot_explanations(explanation):

We can make the following observations from the preceding heatmap:

• The heatmap makes comparing the different variables for each patient easier and thus helps with interpretation.
• It summarizes the results of the case, feature, and label combination and doesn't provide as much detail as the previous plot.
• For class-X1, or patients labeled as normal (1, 2, and 3), all four features (X8, X10, X12, and X20) have very similar weights.
• For class-X3, or patients labeled as pathological (4 and 5), once again, all four features (X8, X10, X13, and X20) have an approximately similar weight.