In this section, let us define an agent travelling across a 4 x 4 grid, as shown in following figure:

This grid has 16 states (S1, S2....S16). In each state, the agent can perform four actions (up, right, down, left). However, the agent will be restricted to some actions based on the following constraints:

• The states across the edges shall be restricted to actions which point only toward states in the grid. For example, an agent in S1 is restricted to the right or down action.
• Some state transitions have barriers, marked in red. For example, the agent cannot go down from S2 to S3.

Each state is also assigned to a reward. The objective of the agent is to reach the destination with minimum moves, thereby achieving the maximum reward. Except state S15 with a reward value of 100, all the remaining states have a reward value of -1.

Here, we will use the MDPtoolbox package in R.