Training an agent with using keras-rl is very easy.
- Step 1- define the policy you want the training to follow. We will be using the Epsilon greedy policy. The equivalent of this in the DQN section would be the agent action function. To know more about other policies visit https://github.com/keras-rl/keras-rl/blob/master/rl/policy.py
- step2 - Load the agent you would like to use. In this case, SARSA agent which has a lot of parameters of which the important ones that need to be defined are model, nb_actions, and policy. model is the deep learning agent you have defined above, nb_actions is the number of possible actions in the system, and policy is your preferred choice of policy to train the SARSA agent.
- step3 - We compile the SARSA agent with loss and optimizer of choice.
- step4 - Finally we fit the SARSA agent by feeding the .fit function the arguments environment and number of steps to train.
To get complete details on the usage of agents from keras-rl library and their parameter definitions, visit this documentation by keras http://keras-rl.readthedocs.io/en/latest/agents/sarsa/#sarsaagent
# Define the policy
policy = EpsGreedyQPolicy()
# Loading SARSA agent by feeding it the policy and the model
sarsa = SARSAAgent(model=model, nb_actions=actions, policy=policy)
# compile sarsa with mean squared error loss
sarsa.compile('adam', metrics=['mse'])
# train the agent for 50000 steps
sarsa.fit(env, nb_steps=50000, visualize=False, verbose=1)
To view the Cart-Pole game on your screen when training, set visualize argument to true inside the .fit function. But Visualizing the game will slow down the training.
Figure 15.15: Scores output when training SARSA agent