Cover image for Deep Learning and the Game of Go

Deep Learning and the Game of Go

Author Kevin Ferguson , Max Pumperla
Release Date: 2019/01/01
ISBN: 9781617295324
Topic:
0%
38
Chapters
0-1
Hours read
0k
Total Words
    Start Reading Now    
       Add to Wishlist       
View table of contents

Book Description

Deep Learning and the Game of Go introduces deep learning by teaching you to build a Go-winning bot. As you progress, you’ll apply increasingly complex training techniques and strategies using the Python deep learning library Keras. You’ll enjoy watching your bot master the game of Go, and along the way, you’ll discover how to apply your new deep learning skills to a wide range of other scenarios!

Table of Contents

  1. Copyright
  2. Brief Table of Contents
  3. Table of Contents
  4. Foreword
  5. Preface
  6. Acknowledgments
  7. About this book
  8. About the authors
  9. About the cover illustration
  10. Part 1. Foundations
    1. Chapter 1. Toward deep learning: a machine-learning introduction
      1. 1.1. What is machine learning?
      2. 1.2. Machine learning by example
      3. 1.3. Deep learning
      4. 1.4. What you’ll learn in this book
      5. 1.5. Summary
    2. Chapter 2. Go as a machine-learning problem
      1. 2.1. Why games?
      2. 2.2. A lightning introduction to the game of Go
      3. 2.3. Handicaps
      4. 2.4. Where to learn more
      5. 2.5. What can we teach a machine?
      6. 2.6. How to measure your Go AI’s strength
      7. 2.7. Summary
    3. Chapter 3. Implementing your first Go bot
      1. 3.1. Representing a game of Go in Python
      2. 3.2. Capturing game state and checking for illegal moves
      3. 3.3. Ending a game
      4. 3.4. Creating your first bot: the weakest Go AI imaginable
      5. 3.5. Speeding up game play with Zobrist hashing
      6. 3.6. Playing against your bot
      7. 3.7. Summary
  11. Part 2. Machine learning and game AI
    1. Chapter 4. Playing games with tree search
      1. 4.1. Classifying games
      2. 4.2. Anticipating your opponent with minimax search
      3. 4.3. Solving tic-tac-toe: a minimax example
      4. 4.4. Reducing search space with pruning
      5. 4.5. Evaluating game states with Monte Carlo tree search
      6. 4.6. Summary
    2. Chapter 5. Getting started with neural networks
      1. 5.1. A simple use case: classifying handwritten digits
      2. 5.2. The basics of neural networks
      3. 5.3. Feed-forward networks
      4. 5.4. How good are our predictions? Loss functions and optimization
      5. 5.5. Training a neural network step-by-step in Python
      6. 5.6. Summary
    3. Chapter 6. Designing a neural network for Go data
      1. 6.1. Encoding a Go game position for neural networks
      2. 6.2. Generating tree-search games as network training data
      3. 6.3. Using the Keras deep-learning library
      4. 6.4. Analyzing space with convolutional networks
      5. 6.5. Predicting Go move probabilities
      6. 6.6. Building deeper networks with dropout and rectified linear units
      7. 6.7. Putting it all together for a stronger Go move-prediction network
      8. 6.8. Summary
    4. Chapter 7. Learning from data: a deep-learning bot
      1. 7.1. Importing Go game records
      2. 7.2. Preparing Go data for deep learning
      3. 7.3. Training a deep-learning model on human game-play data
      4. 7.4. Building more-realistic Go data encoders
      5. 7.5. Training efficiently with adaptive gradients
      6. 7.6. Running your own experiments and evaluating performance
      7. 7.7. Summary
    5. Chapter 8. Deploying bots in the wild
      1. 8.1. Creating a move-prediction agent from a deep neural network
      2. 8.2. Serving your Go bot to a web frontend
      3. 8.3. Training and deploying a Go bot in the cloud
      4. 8.4. Talking to other bots: the Go Text Protocol
      5. 8.5. Competing against other bots locally
      6. 8.6. Deploying a Go bot to an online Go server
      7. 8.7. Summary
    6. Chapter 9. Learning by practice: reinforcement learning
      1. 9.1. The reinforcement-learning cycle
      2. 9.2. What goes into experience?
      3. 9.3. Building an agent that can learn
      4. 9.4. Self-play: how a computer program practices
      5. 9.5. Summary
    7. Chapter 10. Reinforcement learning with policy gradients
      1. 10.1. How random games can identify good decisions
      2. 10.2. Modifying neural network policies with gradient descent
      3. 10.3. Tips for training with self-play
      4. 10.4. Summary
    8. Chapter 11. Reinforcement learning with value methods
      1. 11.1. Playing games with Q-learning
      2. 11.2. Q-learning with Keras
      3. 11.3. Summary
    9. Chapter 12. Reinforcement learning with actor-critic methods
      1. 12.1. Advantage tells you which decisions are important
      2. 12.2. Designing a neural network for actor-critic learning
      3. 12.3. Playing games with an actor-critic agent
      4. 12.4. Training an actor-critic agent from experience data
      5. 12.5. Summary
  12. Part 3. Greater than the sum of its parts
    1. Chapter 13. AlphaGo: Bringing it all together
      1. 13.1. Training deep neural networks for AlphaGo
      2. 13.2. Bootstrapping self-play from policy networks
      3. 13.3. Deriving a value network from self-play data
      4. 13.4. Better search with policy and value networks
      5. 13.5. Practical considerations for training your own AlphaGo
      6. 13.6. Summary
    2. Chapter 14. AlphaGo Zero: Integrating tree search with reinforcement learning
      1. 14.1. Building a neural network for tree search
      2. 14.2. Guiding tree search with a neural network
      3. 14.3. Training
      4. 14.4. Improving exploration with Dirichlet noise
      5. 14.5. Modern techniques for deeper neural networks
      6. 14.6. Exploring additional resources
      7. 14.7. Wrapping up
      8. 14.8. Summary
      9. Vectors, matrices, and beyond: a linear algebra primer
      10. Rank 3 tensors
      11. Calculus in five minutes: derivatives and finding maxima
      12. A bit of notation
      13. The backpropagation algorithm for feed-forward networks
      14. Backpropagation for sequential neural networks
      15. Backpropagation for neural networks in general
      16. Computational challenges with backpropagation
      17. Go programs
      18. Go servers
      19. Model training on AWS
      20. Hosting a bot on AWS over HTTP
      21. Registering and activating your bot at OGS
      22. Testing your OGS bot locally
      23. Deploying your OGS bot on AWS
  13. Appendix A. Mathematical foundations
  14. Appendix B. The backpropagation algorithm
  15. Appendix C. Go programs and servers
  16. Appendix D. Training and deploying bots by using Amazon Web Services
  17. Appendix E. Submitting a bot to the Online Go Server
  18. Index
  19. List of Figures
  20. List of Tables
  21. List of Listings