In this section, we will use the concepts we learned about in this chapter to build a deeper neural network to classify handwritten digits:

1. We will start with a new notebook and then load the required dependencies:

import numpy as np
np.random.seed(42)
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
# new!
from
keras.layers.normalization
# new!
import
BatchNormalization
# new!
from keras import regularizers
# new! 
from keras.optimizers
import SGD

2. We will now load and pre-process the data:

(X_train,y_train),(X_test,y_test)= mnist.load_data()
X_train= X_train.reshape(60000,784).
astype('float32')
X_test= X_test.reshape(10000,784).astype('float32')
X_train/=255
X_test/=255
n_classes=10
y_train=keras.utils.to_categorical(y_train,n_classes)
y_test =keras.utils.to_categorical(y_test,n_classes)
  

3. Now, we will design a deeper neural architecture with measures to take care of overfitting and to provide better generalization:

model=Sequential()
model.add(Dense(64,activation='relu',input_shape=(784,)))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Dense(64,activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Dense(10,activation='softmax'))
model.summary()
_______________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 64)                50240     
_______________________________________________________________
batch_normalization_1 (Batch (None, 64)                256       
_______________________________________________________________
dropout_1 (Dropout)          (None, 64)                0         
_______________________________________________________________
dense_2 (Dense)              (None, 64)                4160      
_______________________________________________________________
batch_normalization_2 (Batch (None, 64)                256       
_______________________________________________________________
dropout_2 (Dropout)          (None, 64)                0         
_______________________________________________________________
dense_3 (Dense)              (None, 10)                650       
=================================================================
Total params: 55,562
Trainable params: 55,306
Non-trainable params: 256_______________________________________________________________

4. This time, we will configure the model using an adam optimizer:

model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])

5. Now, we will post that we will train the model for 200 epochs at a batch size of 128:

model.fit(X_train, y_train, batch_size= 128, epochs= 200, verbose= 1, validation_data= (X_test,y_test))

6. Train on 60,000 samples and validate on 10,000 samples:

Epoch 1/200
60000/60000 [==============================] - 3s - loss: 0.8586 - acc: 0.7308 - val_loss: 0.2594 - val_acc: 0.9230
Epoch 2/200
60000/60000 [==============================] - 2s - loss: 0.4370 - acc: 0.8721 - val_loss: 0.2086 - val_acc: 0.9363
.
.
.
Epoch 200/200
60000/60000 [==============================] - 2s - loss: 0.1323 - acc: 0.9589 - val_loss: 0.1136 - val_acc: 0.9690
<keras.callbacks.History at 0x7f321175a748>