contents
Part 1: Core PyTorch
1 Introducing deep learning and the PyTorch Library
1.1 The deep learning revolution
1.3 Why PyTorch?
The deep learning competitive landscape
1.4 An overview of how PyTorch supports deep learning projects
1.5 Hardware and software requirements
1.6 Exercises
1.7 Summary
2.1 A pretrained network that recognizes the subject of an image
Obtaining a pretrained network for image recognition 19 AlexNet
Ready, set, almost run 22 Run! 25
2.2 A pretrained model that fakes it until it makes it
A network that turns horses into zebras
2.3 A pretrained network that describes scenes
2.4 Torch Hub
2.5 Conclusion
2.6 Exercises
2.7 Summary
3.1 The world as floating-point numbers
3.2 Tensors: Multidimensional arrays
From Python lists to PyTorch tensors
Constructing our first tensors
3.3 Indexing tensors
3.4 Named tensors
Specifying the numeric type with dtype
Managing a tensor’s dtype attribute
3.6 The tensor API
3.7 Tensors: Scenic views of storage
Modifying stored values: In-place operations
3.8 Tensor metadata: Size, offset, and stride
Views of another tensor’s storage
Transposing in higher dimensions 60 Contiguous tensors 60
Managing a tensor’s device attribute
3.11 Generalized tensors are tensors, too
3.12 Serializing tensors
3.13 Conclusion
3.14 Exercises
3.15 Summary
4 Real-world data representation using tensors
Loading an image file 72 Changing the layout
4.2 3D images: Volumetric data
Loading a wine data tensor 78 Representing scores
Shaping the data by time period
One-hot-encoding characters 94 One-hot encoding whole words
Text embeddings 98 Text embeddings as a blueprint 100
4.6 Conclusion
4.7 Exercises
4.8 Summary
5.1 A timeless lesson in modeling
5.2 Learning is just parameter estimation
Choosing a linear model as a first try
5.5 PyTorch’s autograd: Backpropagating all things
Computing the gradient automatically
Training, validation, and overfitting 131 Autograd nits and switching it off 137
5.6 Conclusion
5.7 Exercise
5.8 Summary
6 Using a neural network to fit the data
Composing a multilayer network
Understanding the error function
Choosing the best activation function 148 What learning means for a neural network 149
Using __call__ rather than forward
Inspecting the parameters 159 Comparing to the linear model 161
6.4 Conclusion
6.5 Exercises
6.6 Summary
7 Telling birds from airplanes: Learning from images
The Dataset class 166 Dataset transforms
7.2 Distinguishing birds from airplanes
A fully connected model 174 Output of a classifier
Representing the output as probabilities
The limits of going fully connected 189
7.3 Conclusion
7.4 Exercises
7.5 Summary
8 Using convolutions to generalize
Detecting features with convolutions
Looking further with depth and pooling 202 Putting it all together for our network 205
How PyTorch keeps track of parameters and submodules
Saving and loading our model 214 Training on the GPU 215
8.5 Model design
Helping our model to converge and generalize: Regularization
Going deeper to learn more complex structures: Depth
Comparing the designs from this section
8.6 Conclusion
8.7 Exercises
8.8 Summary
Part 2: Learning from images in the real world: Early detection of lung cancer
9 Using PyTorch to fight cancer
9.1 Introduction to the use case
9.2 Preparing for a large-scale project
9.3 What is a CT scan, exactly?
9.4 The project: An end-to-end detector for lung cancer
Why can’t we just throw data at a neural network until it works?
Our data source: The LUNA Grand Challenge
9.5 Conclusion
9.6 Summary
10 Combining data sources into a unified dataset
10.1 Raw CT data files
10.2 Parsing LUNA’s annotation data
Unifying our annotation and candidate data
10.3 Loading individual CT scans
10.4 Locating a nodule using the patient coordinate system
Converting between millimeters and voxel addresses
Extracting a nodule from a CT scan
10.5 A straightforward dataset implementation
Caching candidate arrays with the getCtRawCandidate function
Constructing our dataset in LunaDataset .__init__
10.6 Conclusion
10.7 Exercises
10.8 Summary
11 Training a classification model to detect suspected tumors
11.1 A foundational model and training loop
11.2 The main entry point for our application
11.3 Pretraining setup and initialization
Initializing the model and optimizer
Care and feeding of data loaders
11.4 Our first-pass neural network design
11.5 Training and validating the model
The validation loop is similar
11.6 Outputting performance metrics
11.7 Running the training script
Interlude: The enumerateWithEstimate function
11.8 Evaluating the model: Getting 99.7% correct means we’re done, right?
11.9 Graphing training metrics with TensorBoard
Adding TensorBoard support to the metrics logging function
11.10 Why isn’t the model learning to detect nodules?
11.11 Conclusion
11.12 Exercises
11.13 Summary
12 Improving training with metrics and augmentation
12.1 High-level plan for improvement
12.2 Good dogs vs. bad guys: False positives and false negatives
12.3 Graphing the positives and negatives
Precision is Preston’s forte 326 Implementing precision and recall in logMetrics
Our ultimate performance metric: The F1 score
How does our model perform with our new metrics? 332
12.4 What does an ideal dataset look like?
Making the data look less like the actual and more like the “ideal” 336 Contrasting training with a balanced LunaDataset to previous runs
Recognizing the symptoms of overfitting 343
12.5 Revisiting the problem of overfitting
An overfit face-to-age prediction model
12.6 Preventing overfitting with data augmentation
Specific data augmentation techniques
Seeing the improvement from data augmentation
12.7 Conclusion
12.8 Exercises
12.9 Summary
13 Using segmentation to find suspected nodules
13.1 Adding a second model to our project
13.2 Various types of segmentation
13.3 Semantic segmentation: Per-pixel classification
13.4 Updating the model for segmentation
Adapting an off-the-shelf model to our project
13.5 Updating the dataset for segmentation
U-Net has very specific input size requirements
U-Net trade-offs for 3D vs. 2D data
Building the ground truth data
Implementing Luna2dSegmentationDataset
Designing our training and validation data
Implementing TrainingLuna2dSegmentationDataset
13.6 Updating the training script for segmentation
Initializing our segmentation and augmentation models 387 Using the Adam optimizer
Getting images into TensorBoard
Updating our metrics logging 396 Saving our model 397
13.7 Results
13.8 Conclusion
13.9 Exercises
13.10 Summary
14 End-to-end nodule analysis, and where to go next
14.2 Independence of the validation set
14.3 Bridging CT segmentation and nodule candidate classification
Getting malignancy information
An area under the curve baseline: Classifying by diameter
Reusing preexisting weights: Fine-tuning
14.6 What we see when we diagnose
Training, validation, and test sets
14.7 What next? Additional sources of inspiration (and data)
Preventing overfitting: Better regularization
Competition results and research papers
14.8 Conclusion
14.9 Exercises
14.10 Summary
Part 3: Deployment
Our model behind a Flask server
15.2 Exporting models
Interoperability beyond PyTorch with ONNX
Our server with a traced model
15.3 Interacting with the PyTorch JIT
What to expect from moving beyond classic Python/PyTorch 458 The dual nature of PyTorch as interface and backend 460 TorchScript
Scripting the gaps of traceability 464
C++ from the start: The C++ API
15.5 Going mobile
Improving efficiency: Model design and quantization
15.6 Emerging technology: Enterprise serving of PyTorch models
15.7 Conclusion
15.8 Exercises
15.9 Summary