Index

[SYMBOL][A][B][C][D][E][F][G][H][I][J][K][L][M][N][O][P][Q][R][S][T][U][V][W][X][Z]

SYMBOL

* operator

A

accuracy
action-value functions, 2^nd
activating bots at OGS
activation argument, 2^nd
activation layers
actor-critic algorithm

actor-critic methods
  designing neural networks for actor-critic learning
  playing games with agents
  reinforcement learning with
  training agents from experience data
Adadelta optimizer, 2^nd
Adagrad optimizer

adaptive gradients
  refining with Adadelta
  training with
    decay and momentum in SGD
    optimizing neural networks with Adagrad
add_liberty function

advantage
  calculating during self-play
  overview of
  to determine important decisions
affine linear
AGA (American Go Association)
Agent class

agents
  loading from disk
  saving from disk
AGZ (AlphaGo Zero)
  additional resources for
  building neural network for tree search
  guiding tree search with neural networks
    expanding trees
    selecting moves
    walking down trees
  improving exploration with Dirichlet noise
  modern techniques for deeper neural networks
    batch normalization
    residual networks
  training
AI (artificial intelligence)
algorithms
alpha-beta pruning, 2^nd

AlphaGo program
  board encoder
  bootstrapping self-play from policy networks
  deriving value networks from self-play data
  network architectures in
  policy networks for improved searches
    implementing AlphaGo search algorithm
    neural networks to improve Monte Carlo rollouts
    tree search with combined value function
  practical considerations for training
  training deep neural networks for
  training policy networks
  value networks for improved searches
    implementing AlphaGo search algorithm
    neural networks to improve Monte Carlo rollouts
    tree search with combined value function
AlphaGo Zero.
    See AGZ.
alphago_model function
alphago_sl_agent
AlphaGoEncoder
AlphaGoMCTS class, 2^nd
AlphaGoNode
Amazon Machine Image (AMI)
Amazon Web Services (AWS).
    See AWS.
American Go Association (AGA)
AMI (Amazon Machine Image)
ANNs (artificial neural networks), 2^nd
apply_move function
architectures, testing
area counting
area scoring
argsort function
array type
artificial intelligence (AI)
artificial neural networks (ANNs), 2^nd
average pooling
average_digit function
AveragePooling2D
avg_eight function

AWS (Amazon Web Service)
  deploying bots with
  hosting bots on
  model training on
  overview of, 2^nd, 3^rd
  training bots with
axis argument

B

backends
backpropagation
  computational challenges with
  for feed-forward networks
  for sequential neural networks
  notation in
  overview of
backward passes, 2^nd
batch normalization
batch_size parameter
begin_episode method
benchmarking Go AI (artificial intelligence)
BensonDarr
best_black value
best_result function
best_white value
BetagoBot account
bias term, 2^nd
binary classification
binary features
binomial test
black_player function

board
  AlphaGo encoders
  implementing in Python
  overview of
Board class, 2^nd, 3^rd
board-game AI, overall structure of
  evaluating game states
  reducing number of moves to consider
  searching game states
  selecting opening moves
boardsize command
bootstrapping self-play from policy networks
bot account
bot_v_bot function

bots
  activating at OGS
  competing against other bots locally
  deploying
    GTP (Go Text Protocol)
    in cloud
    serving to web frontend
    to online servers
    with AWS
  hosting on AWS over HTTP
  losing gracefully
  playing against
  playing against other Go programs
  registering at OGS
    overview of
    registering bots at OGS
  submitting to OGS
  testing locally
  training in cloud
  training with AWS
  when bots should pass or resign.
    See also deep-learning bots; ; Go bot.

C

calculus
  derivatives
  finding maxima
callbacks
can_add_child property
categorical cross-entropy loss function
chain rule, 2^nd
chained layers
channel
channels first convention
channels_first format
channels_last format
child node
children property
Chinese counting
chunking
class imbalance

classifying
  cross-entropy loss for problems with
  games
  handwritten digits
    MNIST data preprocessing
    MNIST data set of handwritten digits
clear_deltas method
CLI (command-line interface)
clipnorm parameter

clipping
  overview of
  probability distributions

cloud
  deploying bots in
  training bots in
clustering
combined value functions
command_and_response method
command-line interface (CLI)
competing against other bots locally
  bots playing against other Go programs
  when bots should pass or resign
complete_episode method, 2^nd
completing captures
computing the gradient
Concatenate layer
concentration parameter
consolidate_games function
Conv2D layer, 2^nd
convolutional kernel
convolutional layer

convolutional networks
  analyzing space with
  building with Keras
  overview of convolutions
convolutions
counting
covariate shift
credit assignment problem, 2^nd
cross-entropy loss
curvature of function
cutoff

D

data
  building data generators
  building data processors
  for training networks
  human game-play data
  one-dimensional data
  parallel data processing and generators
  preparing for deep learning
    parallel data processing and generators
    replaying Go game from an SGF record
  training and test data
  two-dimensional data.
    See also experience data; Go data, designing neural networks for.
data generator
dead ReLUs problem
decay in SGD
decision threshold
decode_move_index function
decode_point_index function
decoding operation

deep learning
  overview of
  preparing data for
    building data generators
    building data processors
    parallel data processing and generators
    replaying Go game from an SGF record

deep neural networks
  creating move-prediction agents from
  modern techniques for
    batch normalization
    residual networks
deep-learning bots
  building data encoders
  importing Go game records
    downloading and replaying Go game records from KGS
    SGF file format
  preparing data for deep learning
    building data generators
    building data processors
    parallel data processing and generators
    replaying Go game from an SGF record, 6^th
    building data generators
    building data processors
    parallel data processing and generators
    replaying Go game from an SGF record
  running experiments
    guidelines for testing architectures
    guidelines for testing hyperparameters
  training deep-learning models on human game-play data
  training with adaptive gradients
    decay and momentum in SGD
    optimizing neural networks with Adagrad
    refining adaptive gradients with Adadelta
deep-learning library, in Keras
  design principles
  installing
  move prediction with feed-forward neural networks
  running first example
DeepLearningAgent, 2^nd, 3^rd, 4^th
dense layers, 2^nd, 3^rd

deploying
  bots
    competing against other bots locally
    creating move-prediction agents from deep neural network
    GTP (Go Text Protocol)
    in cloud
    serving to web frontend
    to online servers
    with AWS
  OGS on AWS
depth pruning, 2^nd
derivatives
descriptive model
deterministic games
diagonal matrix
differentiable function
Dirichlet noise
discounting technique

disks
  loading agents from
  saving agents from
dlgo module, 2^nd, 3^rd, 4^th
dlgo.data.parallel_processor
dlgo.encoders module
dlgo.networks module
does_move_violate_ko
dot product, 2^nd
download_files method
downloading Go game records from KGS
dropout technique, dropping neurons for regularization

E

ϵ-greedy policy, 2^nd
EC2 (Elastic Compute Cloud)
Elastic Compute Cloud (EC2)
empty points
encode_move function
encode_point function
Encoder class, 2^nd
encoders
  AlphaGo board encoder
  building data encoders
encoding Go game positions
episodes
epochs
error terms, propagating
evaluate_generator function
evaluation functions

evaluations
  position evaluation
  rollout policies for
expanding trees

experience data
  overview of
  representing
  training actor-critic agents from
experience_simulation function
ExperienceBuffer class
ExperienceCollector class, 2^nd, 3^rd
expert systems
exploitation, 2^nd
exploration, 2^nd

F

fast policy network
feature maps
feature planes
feature vectors
features
feed-forward networks
  backpropagation for
  dense layers in Python as building blocks for
  predicting moves in Keras
filter
fit function, 2^nd, 3^rd
fit_generator function
fitting models
Flatten layer
forgetting
forward passes, 2^nd, 3^rd
FreeGoban
frontend.
    See web frontend, serving bots to.
frozenset
Fuego
fully connected layer
fuzzy logic

G

game states
  capturing
  evaluating
    overview of
    with MCTS
  searching
game tree
game_state property

games
  as machine-learning problems
  classifying
  playing with actor-critic agents
  playing with Q-learning
  simulating
GameState class, 2^nd, 3^rd, 4^th
generate method
generators, data generators
GenericLinearFunction
genmove command
get_backward_input function
get_encoder_by_name function
get_forward_input function
get_go_string function
get_web_app method
global minimum
GNU Go program, 2^nd

Go AI
  benchmarking
  measuring strength of
Go bot
  capturing game states
  checking for illegal moves
    ko
    self-capture
  creating first bot
  ending games
  playing against bots
  speeding up game play with Zobrist hashing

Go game
  additional resources for
  as machine-learning problem
    measuring Go AI strength
    overall structure of board-game AI
  downloading records from KGS
  encoding positions for neural networks
  handicap stones
  importing records
  overview of
    capturing stones
    counting
    ending game
    placing stones
    understanding board
    understanding ko
  replaying from SGF records
  replaying records from KGS
  representing in Python
    capturing stones on board
    implementing board
    placing stones on board
    tracking connected groups of stones
  traditional Go ranks
Go programs
  GNU Go
  Pachi
Go servers
  IGS (Internet Go Server)
  OGS (Online Go Server)
  Tygem
Go Text Protocol (GTP), 2^nd, 3^rd
GoDataProcessor, 2^nd, 3^rd
GOFAI (good old-fashioned AI)
gosgf submodule
GoString class
gradient ascent algorithm

gradient descent algorithm
  modifying neural network policies with
  overview of
  to find minima
gradients, propagate back through network.
    See also policy gradients.
graphical interfaces
greedy policy, 2^nd
groups
GTP (Go Text Protocol), 2^nd, 3^rd
gtp2ogs library
GTPFrontend, 2^nd, 3^rd

H

Hadamard product, 2^nd
handicap stones
handwritten digits, classifying, 2^nd
  MNIST data preprocessing
  MNIST data set of handwritten digits
hash values
heavy rollouts
hidden information games
hidden layers
hidden units
HTTP (Hypertext Transfer Protocol), 2^nd
httpfrontend module
human game-play data
hyperbolic tangent function

hyperparameters
  overview of, 2^nd
  testing
Hypertext Transfer Protocol (HTTP), 2^nd

I

IGS (Internet Go Server)
illegal moves, checking for
  ko
  self-capture
importing Go game records
init_ac_agent.py script
inner product
input image
input neurons
installing Keras deep-learning library
Internet Go Server
is_over function
is_point_an_eye function
is_terminal property

J

JagoClient
Japanese rules
jgoboard library

K

Keras API
  building convolutional neural networks with
  building two-input networks in
  deep-learning library in
    installing
    move prediction with feed-forward neural networks
    running first example with Keras
  design principles
  functional API
  implementing E-greedy policy with
  with Q-learning
Keras library
keras.datasets module
keras.layers module
kernel

KGS Go Server
  downloading Go game records from
  overview of
  replaying Go game records from
KGSIndex class

ko
  checking for
  overview of
komi

L

labels
ladders
Layer class

layers
  activation layers in neural networks
  dense layers in Python
  neural network layers in Python
  overview of, 2^nd, 3^rd
  pooling layers
  softmax activation function in
LeakyReLU function
learning rate
learning.
    See RL (reinforcement learning); ; supervised learning; ; unsupervised learning.
Leela Zero
legal_moves function
liberties
light rollouts
load_experience function
load_go_data method
load_policy_agent function
loading agents from disk
local maximum
local minimum, 2^nd
LocalGtpBot
logic production systems
logistic regression, 2^nd
long-running process
loss functions
  finding minima in
  gradient descent to find minima
  MSE for
  overview of, 2^nd, 3^rd
  propagate gradients back through network
  SGD for
loss weights
loss_derivative function

M

machine learning
  AI and
  applications of
  deep learning
  games as
  Go game as
    overall structure of board-game AI
  in software applications
  limits of
  overview of
  reinforcement learning
  supervised learning
  unsupervised learning
matrices
matrix transposition
max pooling
max_depth parameter
maxima
MaxPooling2D

MCTS (Monte Carlo tree search)
  applying to Go
    bot losing gracefully
    improved rollout policies for better evaluations
  evaluating game states with
    selecting branches to explore
  implementing in Python
  overview of
mcts module
MCTSNode class
mean squared error (MSE)

measuring
  Go AI strength
    benchmarking Go AI
    traditional Go ranks
  small differences in strength
merged_with method, 2^nd
meta-information
mini-batch
Minigo

minima
  finding in loss functions
  gradient descent to find

minimax search algorithm
  anticipating opponent with
  example of
  overview of
MLP (multilayer perceptron)

MNIST (Modified National Institute of Standards and Technology)
  data preprocessing
  data set of handwritten digits
  overview of
ModelCheckpoint tool

models
  calibrating
  deep-learning
  training on AWS
Modified National Institute of Standards and Technology.
    See MNIST.
momentum in SGD
Monte Carlo rollouts
Monte Carlo tree search (MCTS).
    See MCTS.
Move class
move property
move-prediction agents
move-prediction networks, 2^nd

moves
  implementing move selection
  predicting probabilities of
    cross-entropy loss for classification problems
    softmax activation function in last layer
  predicting with feed-forward neural networks in Keras
  reducing number of moves to consider
  selecting
    opening moves
    overview of
MSE (mean squared error)
multilayer neural network
multilayer perceptron (MLP)

N

neural networks
  activation layers in
  backpropagation for
  building for tree search
  classifying handwritten digits
    MNIST data preprocessing
    MNIST data set of handwritten digits
  designing for actor-critic learning
  encoding Go game positions for
  feed-forward networks
  guiding tree search with
    expanding trees
    selecting moves
    walking down trees
  loss functions
    finding minima in
    gradient descent to find minima
    MSE
    overview of
    propagate gradients back through network
    SGD for
  modifying policies with gradient descent
  optimizing with Adagrad
  overview of, 2^nd
    logistic regression as simple artificial neural network
    networks with more than one output dimension
  to improve Monte Carlo rollouts
  training in Python
    applying network handwritten digit classification
    dense layers in Python as building blocks for feed-forward networks
    neural network layers in Python
    sequential neural networks with Python.
    See deep neural networks.
neurons, 2^nd
next() function
node probabilities
nodes
nondeterministic games
nonsequential neural networks
normalizing
notation in backpropagation
np.array function
np.clip function
np.random.choice function
np.random.dirichlet function
num_cols function
num_liberties function, 2^nd
num_rollouts property
num_rows function
num_simulations
num_total_examples function
numerical input
numpy arrays
NumPy library
NVIDIA

O

objective functions
OCR (optical character recognition)
OGS (Online Go Server)
  activating bots at
  deploying on AWS
  overview of, 2^nd, 3^rd
  registering bots at, 2^nd
  submitting bots to
  testing bots locally
one-dimensional data
oneplane encoder
OnePlaneEncoder, 2^nd, 3^rd
online servers, deploying bots to
OpenAI Gym
opening moves, selecting
optical character recognition (OCR)
optimization, starting point for

optimizers
  overview of
  tuning
optimizing neural networks with Adagrad
other method
outlier detection
output dimensions
output neurons
output probabilities
overfitting, 2^nd

P

Pachi program, 2^nd, 3^rd
parallel data processing
parallelized search
parent property
parse function
passing
patches
pattern recognition
perfect information games
performance, evaluating
PhoenixGo
picture labels
place_stone method
planes
play command
play_our_move method
play_their_move method
Player class
Point class
Polanyi’s paradox
policy function

policy gradients
  overview of
  reinforcement learning with
    identifying good decisions with random games
    modifying neural network policies with gradient descent
    training with self-play
policy learning

policy networks
  AlphaGo-style, training
  bootstrapping self-play from
  for improved searches
    implementing AlphaGo search algorithm
    neural networks to improve Monte Carlo rollouts
    tree search with combined value function
  overview of
policy_probabilities
policy_rollout
PolicyAgent class, 2^nd, 3^rd
pool size
pooling layers
position evaluation
predict labels

predicting
  move probabilities
    cross-entropy loss for classification problems
    softmax activation function in last layer
  moves with feed-forward neural networks in Keras.
    See also move-prediction agents.

predictions
  calculating
  computing
  cutoff value in
  evaluating
prepare_experience_data function
previous_states variable
print_board function
print_move function
prior probabilities

probabilities
  clipping distributions
  of moves, predicting
    cross-entropy loss for classification problems
    softmax activation function in last layer
  sampling from distributions
process method
process_zip function
propagating error terms
propagating gradients
pruning, reducing search space with
  reducing search depth with position evaluation
  reducing search width with alpha-beta pruning

Python programming language
  and machine learning
  implementing MCTS in
  neural network layers in
  representing Go game in
    capturing stones on board
    implementing board
    placing stones on board
    tracking connected groups of stones
  sequential neural networks with
  training neural networks in

Q

Q-learning technique
  playing games with
  with Keras
    building two-input networks
    implementing E-greedy policy
    training action-value functions
QAgent class, 2^nd

R

random games
RandomAgent
RandomBot
rank 3 tensors
rank 4 tensors
record_decision method, 2^nd
rectified linear units (ReLU) activation function, 2^nd
refining adaptive gradients with Adadelta
registering bots at OGS, 2^nd
regression problems
regularization
REINFORCE method
reinforcement-learning techniques
ReLU (rectified linear units) activation function, 2^nd
remove_liberty function

replaying
  Go game from an SGF record
  Go game records from KGS
residual networks
resigning
ResignLargeMargin strategy
rewards, calculating
RL (reinforcement learning)
  building agents that can learn
    clipping probability distributions
    implementing move selection
    loading agents from disk
    sampling from probability distributions
    saving agents from disk
  cycle of
  experience data
  integrating tree search with
    building neural network for tree search
    guiding tree search with neural networks
    improving exploration with Dirichlet noise
    modern techniques for deeper neural networks
    training
  self-play
    representing experience data
    simulating games
  with actor-critic methods
    advantage to determine important decisions
    designing neural networks for actor-critic learning
    playing games with actor-critic agents
    training actor-critic agents from experience data
  with policy gradients
    identifying good decisions with random games
    modifying neural network policies with gradient descent
    training with self-play
  with value methods
    playing games with Q-learning
    Q-learning with Keras
rollout policies, 2^nd

S

sampling from probability distributions
saving agents from disk
scalar
search algorithm
search depth
search space, reducing with pruning
  reducing search depth with position evaluation
  reducing search width with alpha-beta pruning
search width
searching game states
select_move function, 2^nd, 3^rd, 4^th, 5^th, 6^th
self-capture, 2^nd
self-play
  bootstrapping from policy networks
  calculating advantage during
  deriving value networks from data
  overview of
  representing experience data
  simulating games
  training with
    evaluating progress
    measuring small differences in strength
    tuning SGD optimizers
self.previous_states
Sensei’s Library, 2^nd

sequential neural networks
  backpropagation for
  overview of
  with Python
SequentialNetwork class, 2^nd
serialize method
servers.
    See Go servers; ; online servers, deploying bots to.
serving bots to web frontend
set_collector method, 2^nd
set_handicap method
set_temperature method
SevenPlaneEncoder

SGD (stochastic gradient descent)
  decay and momentum in
  for loss functions
  tuning optimizers
SGF (Smart Game Format) file format, 2^nd
Sgf_game class
SGFWriter
shape attribute
shoulder hit
sigmoid function, 2^nd, 3^rd
sigmoid_double function
simple heuristics
simple models
simulate_game function
simulate_random_game function
simulating games
situational superko rule
skip connection
Smart Game Format (SGF) file format, 2^nd
Smart Go Format
snapback
Sobel kernel
softmax activation function, 2^nd
splitting data
ssh command
standardized protocols
star points
steps_per_epoch function
stochastic gradient descent (SGD).
    See SGD.
stochastic policy, 2^nd
stones
  capturing
    on board in Python
    overview of
  placing
    on board in Python
    overview of
  tracking connected groups of in Python
strings
strong policy network
suggest_tags function
supervised learning

T

tanh activation
temperature, 2^nd
Tencent
TensorFlow library
tensorflow-gpu library

tensors
  overview of
  rank 3 tensors
  rank 4 tensors
termination strategies
TerminationAgent class
territory scoring
test data, splitting

testing
  architectures
  hyperparameters
  OGS bots locally
text-based protocols
Theano library
to_buffer method
train function, 2^nd
train_batch method
train_data method
trainable parameters

training
  action-value functions
  actor-critic agents from experience data
  AlphaGo
  AlphaGo-style policy networks
  and test data
  bots
    in cloud
    with AWS
  deep neural networks for AlphaGo
    AlphaGo board encoder
    network architectures in AlphaGo
    training AlphaGo-style policy networks
  deep-learning models on human game-play data
  models on AWS
  networks
  with adaptive gradients
    decay and momentum in SGD
    optimizing neural networks with Adagrad
    refining adaptive gradients with Adadelta
  with self-play
    evaluating progress
    measuring small differences in strength
    tuning SGD optimizers
training models
tree search
  building neural networks for
  classifying games
  evaluating game states with MCTS
    applying MCTS to Go
    implementing MCTS in Python
    selecting branches to explore
  generating games as network training data
  guiding with neural networks
    expanding trees
    selecting moves
    walking down trees
  integrating with reinforcement learning
    improving exploration with Dirichlet noise
    training
  minimax algorithm
  minimax search algorithm
  reducing search space with pruning
    reducing search depth with position evaluation
    reducing search width with alpha-beta pruning
  with combined value function

trees
  expanding
  walking down
tuning SGD optimizers
two-dimensional data
two-input networks
Tygem Go server, 2^nd

U

UCT (upper confidence bound for trees) formula, 2^nd
uniform random policy
unsupervised learning
unvisited_moves property
update rules, for parameters
upper confidence bound for trees (UCT) formula, 2^nd
use_generator flag

V

validation data
validation set

value methods
  overview of
  reinforcement learning with
    playing games with Q-learning
    Q-learning with Keras

value networks
  deriving from self-play data
  for improved searches
    implementing AlphaGo search algorithm
    neural networks to improve Monte Carlo rollouts
    tree search with combined value function
  overview of
value-estimating function
ValueAgent
vectors
  one-dimensional data
  overview of
vertical edges
visible units

W

web frontend, serving bots to
weight initializers
weights, 2^nd
white_player function
win_counts property
winner function
winning_frac property

X

XOR operation

Z

ZeroAgent
ZeroPadding2D layer, 2^nd
ZeroTreeNode class
Zobrist hashing