Deeper network architecture

The code used for this experiment is as follows:

# Model Architecture
model <- keras_model_sequential()
model %>%
layer_dense(units = 100, activation = 'relu', input_shape = c(13)) %>%
layer_dropout(rate = 0.4) %>%
layer_dense(units = 50, activation = 'relu') %>%
layer_dropout(rate = 0.3) %>%
layer_dense(units = 20, activation = 'relu') %>%
layer_dropout(rate = 0.2) %>%
layer_dense(units = 1)
summary(model)

OUTPUT
## ___________________________________________________________________________
## Layer (type) Output Shape Param #
## ===========================================================================
## dense_4 (Dense) (None, 100) 1400
## ___________________________________________________________________________
## dropout_1 (Dropout) (None, 100) 0
## ___________________________________________________________________________
## dense_5 (Dense) (None, 50) 5050
## ___________________________________________________________________________
## dropout_2 (Dropout) (None, 50) 0
## ___________________________________________________________________________
## dense_6 (Dense) (None, 20) 1020
## ___________________________________________________________________________
## dropout_3 (Dropout) (None, 20) 0
## ___________________________________________________________________________
## dense_7 (Dense) (None, 1) 21
## ===========================================================================
## Total params: 7,491
## Trainable params: 7,491
## Non-trainable params: 0
## _________________________________________________________________________

# Compile model
model %>% compile(loss = 'mse',
optimizer = 'rmsprop',
metrics = 'mae')

# Fit model
model_two <- model %>%
fit(training,
trainingtarget,
epochs = 100,
batch_size = 32,
validation_split = 0.2)
plot(model_two)

From the preceding code, we can observe that we now have three hidden layers with 100, 50, and 20 units respectively. We have also added a dropout layer after each hidden layer with rates of 0.4, 0.3, and 0.2 respectively. As an example of what a dropout layer's rate means, a rate of 0.4 means that 40% of the units in the first hidden layer are dropped to zero at the time of training, which helps to avoid overfitting. The total number of parameters in this model has now increased to 7,491. Note that, in the previous model, the total number of parameters was 201, and clearly we are going for a significantly bigger neural network. Next, we compile the model with the same settings that we used earlier, and subsequently, we will fit the model and store the results in model_two

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset