In this recipe, we will use the ReinforcementLearning package, which performs model-free RL.

Let's import the ReinforcementLearning package:

library(ReinforcementLearning)

In this recipe, we will work on the same navigation example that we used in the previous section, Model-based RL using MDPtoolbox. In this case, we won't have any predetermined input data, and we will solve the problem using a model-free approach. The agent will interact dynamically with an environment representing the problem and generate state-action transition tuples. The structure of the environment is specific to the problem at hand. An environment is typically a stochastic finite state machine that represents the rules of operating in any specific problem. It provides feedback to the agent about its actions in terms of rewards and penalties.

The following is some generic pseudocode for an environment:

environment <- function(state, action) {
 ...
 return(list("NextState" = newState,"Reward" = reward))
 }

In the next section, we will create an environment of the navigation grid and train an agent to navigate through the grid using model-free RL.