index

A

activation-based methods 148

AdaBoost 66

AdaBoostClassifier class 71

adaptive boosting 66

adult income prediction example 240 – 246

exploratory data analysis 241 – 243

prediction model 244 – 246

AI (artificial intelligence) 3

anchors 119 – 123

ANN (artificial neural network) 95

attribution methods, saliency mapping 161 – 162

AUC (area under the curve) 253

autograd package 291

B

backpropagation methods 148

backward function 292

bagging 65

BCE (binary cross-entropy) loss function 97

BERT (bidirectional encoder representations from transformers) 215

bias

correcting label bias through reweighting 262 – 266

Diagnostics+ AI system 11 – 12

fairness through unawareness 261 – 262

mitigating 261 – 266

binary classification 91, 128

BinaryClassifier class 294

black-box models 52 – 53

boosting 65

BoW (bag of words) 208

Broden 173

business stakeholder 15

C

CamExtractor class 157

CART (classification and regression tree) algorithm 34

categorical features 59

CBOW (continuous bag of words) 208

classification 9, 58

CNNs (convolutional neural networks) 52, 95, 130 – 140, 200, 260

data preparation 135 – 138

interpreting 140 – 148

layers and units 169 – 171

LIME 141 – 147

probability landscape 140 – 141

training and evaluating 138 – 140

visual attribution methods 147 – 148

coalition vector 116

coarse-grained activation map 148

Computer Vision Annotation Tool (CVAT) 174

concept detectors, network dissection

by training task 189 – 193

overview 183 – 188

visualizing 195 – 198

concept drift 12

concept naming 174

Conda environment 281 – 282

context words 208

continuous bag of words (CBOW) 208

counterfactual fairness 255

coverage metric 119

CPU tensors 287

Cramer’s V statistic 85

criterion parameter 35

cross-validation 45

cubic spline 44

CustomDataset class 289

CVAT (Computer Vision Annotation Tool) 174

D

data leakage, Diagnostics+ AI system 11

data preparation

CNNs 135 – 138

DNNs 100 – 101

Data randomization test 161

data scientists 15

data types, PyTorch tensors 286 – 287

DATA_DIRECTORY setting 180

DataLoader class 137, 288, 290

DataLoader object 290

Dataset class 136, 288 – 290

datasets, fairness and 266 – 268

decision trees 33 – 40

interpreting 35 – 39

limitations of 39 – 40

tree ensembles 65 – 73

deep learning 95

degree 3 spline 44

degrees of freedom 44

demographic parity 248 – 251

densely labeled dataset 173

Diagnostics+ AI system

bias 11 – 12

breast cancer diagnosis 90 – 91

building 9 – 10, 12 – 14

concept drift 12

data leakage 11

diabetes progression prediction 24 – 26, 46 – 48

IDC detection 127 – 128

overview 4

regulatory noncompliance 12

DiCE (diverse counterfactual explanations) 277

discrimination

via input features 256 – 259

via representation 260 – 261

distributed representations 207

DNNs (deep neural networks) 52, 95 – 104, 130

data preparation 100 – 101

interpreting 104 – 105

training and evaluating 101 – 104

Docker 282 – 283

E

end users 15

engineers 15

equality of opportunity and odds 251 – 254

experts 15

explainability

counterfactual explanations 275 – 279

interpretability vs. 14 – 16

overview 272 – 275

explainable AI (XAI) 274

explaining phase 13

exploratory data analysis

adult income prediction example 241 – 243

high school student performance predictor 59 – 63

model-agnostic methods 91 – 95

saliency mapping 128 – 130

semantic similarity 203 – 206

exponential kernel function 107

F

F1 score 70

FAIR (Facebook’s AI Research) 284

fairness 269

adult income prediction example 240 – 246

exploratory data analysis 241 – 243

prediction model 244 – 246

counterfactual fairness 255

datasheets for datasets 266 – 268

demographic parity 248 – 251

equality of opportunity and odds 251 – 254

fairness notions 246 – 255

interpretability and 256 – 261

discrimination via input features 256 – 259

discrimination via representation 260 – 261

mitigating bias 261 – 266

correcting label bias through reweighting 262 – 266

fairness through unawareness 261 – 262

predictive quality parity 255

through unawareness 255

FCNNs (fully connected neural networks) 95

feature vector 116

feature_extraction function 181

FEATURE_NAMES setting 181

feature-learning layers 170

FeatureOperator class 181

FeatureOperator object 181

fine-grained activation map 148

freedom, degrees of 44

fully connected neural networks (FCNNs) 95

G

GAMs (generalized additive models) 16, 40 – 51, 271

for Diagnostics+ diabetes progression 46 – 48

interpreting 48 – 51

limitations of 51

regression splines 42 – 46

GDPR (General Data Protection Regulation) 12, 267

Git code repository 281

global interpretability 16, 74 – 87

feature interactions 80 – 87

partial dependence plots 74 – 80

GloVe embeddings 212 – 213

GPT (generative pretrained transformer) 215

GPU setting 180

GPU tensors 287

grad-CAM (gradient-weighted class activation mapping) 18, 157 – 160

GradCam class 158

gradient-based methods, saliency mapping 156 – 157

gradient-boosting algorithm 67

GradientBoostingClassifier class 71

group fairness 255

guided backpropagation 153 – 155

guided Grad-CAM 148

H

high school student performance predictor

exploratory data analysis 59 – 63

overview 58 – 59

hyperparameters 10

I

ICE (individual conditional expectation) plots 18

IDC (invasive ductal carcinoma) detection 127 – 128

individual fairness 255

integrated gradients method 148

interpretability

explainability vs. 14 – 16

fairness and 256 – 261

discrimination via input features 256 – 259

discrimination via representation 260 – 261

global interpretability 74 – 87

feature interactions 80 – 87

partial dependence plots 74 – 80

local interpretability 105 – 115

techniques 15 – 16

interpreting

CNNs 140 – 148

decision trees 35 – 39

DNNs 104 – 105

GAMs 48 – 51

layers and units 178 – 199

linear regression 30 – 32

random forest model 71 – 73

semantic similarity 215 – 231

measuring similarity 217 – 220

principal component analysis 220 – 225

t-SNE 225 – 230

validating visualizations 231

intrinsic interpretability techniques 15

IoU (Intersection over Union) score 177

J

Jupyter notebooks 282

K

kernel width 108, 142

knots 44

L

label bias, correcting through reweighting 262 – 266

LabelEncoder class 244

layers and units 199

CNNs 169 – 171

interpreting 178 – 198

concept detectors 183 – 198

limitations of network dissection 198

running network dissection 179 – 183

network dissection 171 – 178

concept definition 173 – 175

network probing 175 – 177

quantifying alignment 177 – 178

visual understanding 168 – 169

LimeImageExplainer class 109

LIMEs (local interpretable model-agnostic explanations) 18, 272

CNNs 141 – 147

overview 105 – 115

LimeTabularExplainer class 109

Linear module 294

linear regression 27 – 33

interpreting 30 – 32

limitations of 33

Linear units 102

local interpretability 16, 105, 147

See also model-agnostic methods

M

machine learning systems 4 – 9

for Diagnostics+ AI 9

reinforcement learning 8 – 9

representation of data 5

supervised learning 6 – 7

unsupervised learning 7

MAE (mean absolute error) 29, 295

manifold learning 225

MAPE (mean absolute percentage error) 29

mean squared error (MSE) 34, 296

MLPs (multilayer perceptrons) 95

model parameter 144

Model parameter randomization test 161

model-agnostic methods 16 – 125, 147

anchors 119 – 123

DNNs 95 – 104

data preparation 100 – 101

interpreting 104 – 105

training and evaluating 101 – 104

exploratory data analysis 91 – 95

global interpretability 74 – 87

feature interactions 80 – 87

partial dependence plots 74 – 80

high school student performance predictor

exploratory data analysis 59 – 63

overview 58 – 59

LIME 105 – 115

SHapley Additive exPlanations 115 – 119

tree ensembles 65 – 73

model-specific interpretability techniques 15

modeling, PyTorch 290 – 297

automatic differentiation 291 – 293

model definition 293 – 295

training 295 – 297

monitoring phase 14

MSE (mean squared error) 34, 296

multicollinearity 32

multilayer perceptrons (MLPs) 95

N

natural language processing (NLP) 201

negative sampling 210

network dissection 171 – 178

concept definition 173 – 175

concept detectors

by training task 189 – 193

overview 183 – 188

visualizing 195 – 198

limitations of 198

network probing 175 – 177

quantifying alignment 177 – 178

running 179 – 183

network probing 175 – 177

neural word embeddings 206 – 214

GloVe embeddings 212 – 213

one-hot encoding 207 – 208

sentiment analysis 213 – 214

Word2Vec 208 – 212

NLP (natural language processing) 201

O

one-hot encoding 207 – 208

operations, PyTorch tensors 288

overfitting 39

P

PCA (principal component analysis) 18, 220 – 225, 272

PDPs (partial dependence plots) 18, 74 – 80, 239, 272

perplexity 229

perturbation-based methods 147

perturbed dataset 106

PIL (Python Imaging Library) 143

polynomial regression 40

post-hoc interpretability techniques 15, 147

precision metric 70, 119

precision threshold 119

predicates 119

prediction model

adult income prediction example 244 – 246

diabetes progression prediction 24 – 26, 46 – 48

high school student performance predictor 59 – 63

predictive quality parity 255

pretrained parameter 138, 170

principal component analysis (PCA) 18, 220 – 225, 272

probability landscape, CNNs 140 – 141

Python 281

PyTorch 284 – 297

DataLoader class 288 – 290

Dataset class 288 – 290

defined 284

installing 284 – 285

modeling 290 – 297

automatic differentiation 291 – 293

model definition 293 – 295

training 295 – 297

tensors 285 – 288

CPU 287

data types 286 – 287

GPU 287

operations 288

Q

quantifying alignment, network dissection 177 – 178

R

random forest algorithm 65

random forest model

interpreting 71 – 73

training 67 – 71

RandomForestClassifier class 244

recall metric 70

regression splines, GAMs 42 – 46

regularization 45

regulators 15

regulatory noncompliance, Diagnostics+ AI system 12

reinforcement learning 8 – 9

ReLU (rectified linear unit) 98, 153, 293

ReLU activation function 294

representation of data 5

ResNet (residual network) 134

reweighting, correcting label bias through 262 – 266

RMSE (root mean squared error) 29

RNNs (recurrent neural networks) 52, 95, 213

ROC (receiver operator characteristic) 253

S

saliency mapping 164

attribution methods 161 – 162

CNNs 130 – 140

data preparation 135 – 138

interpreting 140 – 148

LIME 141 – 147

probability landscape 140 – 141

training and evaluating 138 – 140

visual attribution methods 147 – 148

exploratory data analysis 128 – 130

Grad-CAM technique 157 – 160

gradient-based methods 156 – 157

guided backpropagation 153 – 155

guided Grad-CAM technique 157 – 160

IDC detection 127 – 128

vanilla backpropagation 148 – 153

saliency maps 148

score parameter 112

segmentation quality 174

segmentation quantity 174

semantic similarity 233

exploratory data analysis 203 – 206

interpreting 215 – 231

measuring similarity 217 – 220

principal component analysis 220 – 225

t-SNE 225 – 230

validating visualizations 231

neural word embeddings 206 – 214

GloVe embeddings 212 – 213

one-hot encoding 207 – 208

sentiment analysis 213 – 214

Word2Vec 208 – 212

sentiment analysis 201 – 203

sentiment analysis 201 – 203, 213 – 214

Sequential class 102

Sequential container 294

setting up

Conda environment 281 – 282

Docker 282 – 283

Git code repository 281

Jupyter notebooks 282

Python 281

SHAP (SHapley Additive exPlanations) 18, 115 – 119, 272

SHAP kernel 116

shap_values variable 258

Shapley value 115

Sigmoid activation function 294

SMEs (subject matter experts) 46

SmoothGrad (smooth gradients) 18, 148

SoTA (state-of-the-art) machine learning techniques 134

spaCy library 203

superpixels 142

supervised learning 6 – 7

surrounding words 208

synsets 168

T

t-SNE (t-distributed stochastic neighbor embedding) 225 – 230, 261, 272

tensors, PyTorch 285 – 288

CPU 287

data types 286 – 287

GPU 287

operations 288

TF-IDF (term frequency inverse document frequency) 208

torch.nn.Module base class 293

torch.nn.Sequential container 293

torch.tensor function 290

torchtext package 203

torchvision package 137, 170, 284

trace_to_input parameter 259

training and evaluating

CNNs 138 – 140

DNNs 101 – 104

random forest model 67 – 71

treatment equality 255

tree ensembles 65 – 71

overview 65 – 67

random forest model

interpreting 71 – 73

training 67 – 71

U

UAT (user acceptance testing) 10

underfitting 33

understanding phase 12

unsupervised learning 7

V

vanilla backpropagation 148 – 153

visual attribution methods, CNNs 147 – 148

visual understanding, layers and units 168 – 169

VoTT (Visual Object Tagging Tool) 174

W

weakly model-dependent techniques 148

white-box models 15

decision trees 33 – 40

interpreting 35 – 39

limitations of 39 – 40

diabetes progression prediction 24 – 26

GAMs 40 – 51

diabetes progression prediction 46 – 48

interpreting 48 – 51

limitations of 51

regression splines 42 – 46

linear regression 27 – 33

interpreting 30 – 32

limitations of 33

overview 24

word embeddings 207

word vectors 207

Word2Vec (Word to Vector) 208 – 212

X

XAI (explainable AI) 274