Let's define an agent/function approximator.

The agent is nothing but a simple deep neural network which takes in the state (4 variables) of the Cart-Pole system and returns the maximum possible reward for each of the two actions.

The 1st, 2nd, and 3rd layers are simple Dense layers with 16 neurons and with activation as 'relu'.

The final layer is a dense layer with 2 neurons equal to the number of possible actions.

def agent(states, actions):
    """Simple Deep Neural Network."""
    model = Sequential()
    model.add(Dense(16, input_dim=states))
    model.add(Activation('relu'))
    model.add(Dense(16))
    model.add(Activation('relu'))
    model.add(Dense(16))
    model.add(Activation('relu'))
    model.add(Dense(actions))
    model.add(Activation('linear'))
    return model

# print summary of the agent
print(agent(states, actions).summary())