Table of Contents
Chapter 1. Toward deep learning: a machine-learning introduction
1.1. What is machine learning?
1.2. Machine learning by example
Chapter 2. Go as a machine-learning problem
2.2. A lightning introduction to the game of Go
2.2.1. Understanding the board
2.2.2. Placing and capturing stones
2.5. What can we teach a machine?
2.5.1. Selecting moves in the opening
2.6. How to measure your Go AI’s strength
Chapter 3. Implementing your first Go bot
3.1. Representing a game of Go in Python
3.1.1. Implementing the Go board
3.2. Capturing game state and checking for illegal moves
3.4. Creating your first bot: the weakest Go AI imaginable
2. Machine learning and game AI
Chapter 4. Playing games with tree search
4.2. Anticipating your opponent with minimax search
4.3. Solving tic-tac-toe: a minimax example
4.4. Reducing search space with pruning
4.5. Evaluating game states with Monte Carlo tree search
4.5.1. Implementing Monte Carlo tree search in Python
Chapter 5. Getting started with neural networks
5.1. A simple use case: classifying handwritten digits
5.2. The basics of neural networks
5.2.1. Logistic regression as simple artificial neural network
5.4. How good are our predictions? Loss functions and optimization
5.4.1. What is a loss function?
5.4.3. Finding minima in loss functions
5.4.4. Gradient descent to find minima
5.5. Training a neural network step-by-step in Python
5.5.1. Neural network layers in Python
5.5.2. Activation layers in neural networks
5.5.3. Dense layers in Python as building blocks for feed-forward networks
5.5.4. Sequential neural networks with Python
5.5.5. Applying your network handwritten digit classification
Chapter 6. Designing a neural network for Go data
6.1. Encoding a Go game position for neural networks
6.2. Generating tree-search games as network training data
6.3. Using the Keras deep-learning library
6.3.1. Understanding Keras design principles
6.3.2. Installing the Keras deep-learning library
6.3.3. Running a familiar first example with Keras
6.3.4. Go move prediction with feed-forward neural networks in Keras
6.4. Analyzing space with convolutional networks
6.4.1. What convolutions do intuitively
6.5. Predicting Go move probabilities
6.5.1. Using the softmax activation function in the last layer
6.6. Building deeper networks with dropout and rectified linear units
6.7. Putting it all together for a stronger Go move-prediction network
Chapter 7. Learning from data: a deep-learning bot
7.1. Importing Go game records
7.2. Preparing Go data for deep learning
7.2.1. Replaying a Go game from an SGF record
7.2.2. Building a Go data processor
7.2.3. Building a Go data generator to load data efficiently
7.3. Training a deep-learning model on human game-play data
7.4. Building more-realistic Go data encoders
7.5. Training efficiently with adaptive gradients
7.5.1. Decay and momentum in SGD
7.6. Running your own experiments and evaluating performance
7.6.1. A guideline to testing architectures and hyperparameters
7.6.2. Evaluating performance metrics for training and test data
Chapter 8. Deploying bots in the wild
8.1. Creating a move-prediction agent from a deep neural network
8.2. Serving your Go bot to a web frontend
8.3. Training and deploying a Go bot in the cloud
8.4. Talking to other bots: the Go Text Protocol
8.5. Competing against other bots locally
8.6. Deploying a Go bot to an online Go server
Chapter 9. Learning by practice: reinforcement learning
9.1. The reinforcement-learning cycle
9.2. What goes into experience?
9.3. Building an agent that can learn
9.3.1. Sampling from a probability distribution
9.3.2. Clipping a probability distribution
9.4. Self-play: how a computer program practices
Chapter 10. Reinforcement learning with policy gradients
10.1. How random games can identify good decisions
10.2. Modifying neural network policies with gradient descent
10.3. Tips for training with self-play
10.3.1. Evaluating your progress
10.3.2. Measuring small differences in strength
10.3.3. Tuning a stochastic gradient descent (SGD) optimizer
Chapter 11. Reinforcement learning with value methods
11.1. Playing games with Q-learning
11.2.1. Building two-input networks in Keras
Chapter 12. Reinforcement learning with actor-critic methods
12.1. Advantage tells you which decisions are important
12.2. Designing a neural network for actor-critic learning
12.3. Playing games with an actor-critic agent
3. Greater than the sum of its parts
Chapter 13. AlphaGo: Bringing it all together
13.1. Training deep neural networks for AlphaGo
13.1.1. Network architectures in AlphaGo
13.2. Bootstrapping self-play from policy networks
13.3. Deriving a value network from self-play data
13.4. Better search with policy and value networks
13.4.1. Using neural networks to improve Monte Carlo rollouts
13.5. Practical considerations for training your own AlphaGo
Chapter 14. AlphaGo Zero: Integrating tree search with reinforcement learning
14.1. Building a neural network for tree search
14.2. Guiding tree search with a neural network
14.4. Improving exploration with Dirichlet noise
14.5. Modern techniques for deeper neural networks
Vectors, matrices, and beyond: a linear algebra primer
B. The backpropagation algorithm
The backpropagation algorithm for feed-forward networks
Backpropagation for sequential neural networks
D. Training and deploying bots by using Amazon Web Services
E. Submitting a bot to the Online Go Server