The code for incorporating this change and then compiling/fitting the model is as follows:

# Model architecture
model <- keras_model_sequential() 
model %>% 
         layer_embedding(input_dim = 500, output_dim = 32) %>%
         layer_simple_rnn(units = 32) %>% 
         layer_dense(units = 1, activation = "sigmoid")

# Compile model
model %>% compile(optimizer = "rmsprop",
         loss = "binary_crossentropy",
         metrics = c("acc"))

# Fit model
model_two <- model %>% fit(train_x, train_y,
         epochs = 10,
         batch_size = 128,
         validation_split = 0.2)

Here, we change the architecture by increasing the number of units in the simple RNN layer from 8 to 32. Everything else is kept the same. Then, we compile and fit the model, as shown in the preceding code.

The accuracy and loss values after 10 epochs can be seen in the following graph:

The preceding plot indicates the following:

• A significantly bigger gap between training and validation data on epoch 3 onward.
• This clearly suggests an increased level of overfitting compared to the preceding plot, where the number of units in the simple RNN was 8.
• This is also reflected in the higher loss value of 0.585 and the lower accuracy value of 0.757 that we obtained for the test data based on this new model.

Now, let's experiment with a different activation function in the simple RNN layer and see whether this overfitting issue can be resolved.