The QLAction class specifies the transition from one state to another state. It takes two parameters—that is, from and to. Both of them have their own integer identifiers that need to be greater than 0:

• from: A source of the action
• to: Target of the action

The signature is given as follows:

case class QLAction(from: Int, to: Int) {
    require(from >= 0, s"QLAction found from: 
    $from required: >=0")require(to >= 0, s"QLAction found to: 
    $to required: >=0")

override def toString: String = s"n
    Action: state 
    $from => state $to"
}

The QLState class defines the state in the Q-learning. It takes three parameters:

• id: An identifier that uniquely identifies a state
• actions: A list of actions for the transition from this state to other states,
• instance: A state may have properties of type T, independent from the state transition

Here is the signature of the class:

case class QLState[T](id: Int, actions: Seq[QLAction] = List.empty, instance: T) {
    import QLState._check(id)
    final def isGoal: Boolean = actions.nonEmpty
    override def toString: String =s"state: $id ${actions.mkString(" ")
        }
    nInstance: ${instance.toString}"
}

In the preceding code, the toString() method is used for textual representation of a state in Q-learning. The state is defined by its ID and the list of actions it may potentially trigger.

Now we know the state and action to perform. However, the QLearning agent needs to know the search space of the form (States x Actions). The next step consists of creating the graph or search space.