Now that our datasets are ready, let's get started with the modeling process. We already know how to build a deep convolutional network from scratch. We also understand the amount of fine-tuning required to achieve good performance. For this task, we will be utilizing the concepts of transfer learning.

A pretrained model is the basic ingredient required to begin with the task of transfer learning. As discussed in earlier chapters, transfer learning can be leveraged by either fine-tuning weights of a pretrained network on a current task, or utilizing the pretrained model as a feature extractor.

In this use case, we will concentrate on utilizing a pretrained model as a feature extractor. As we know, a deep learning model is basically a stacking of interconnected layers of neurons, with the final one acting as a classifier. This architecture enables deep neural networks to capture different features at different levels in the network. Thus, we can utilize this property to use them as feature extractors. This is made possible by removing the final layer or using the output from the penultimate layer. This output from the penultimate layer is then fed into an additional set of layers, followed by a classification layer. The following code snippet showcases feature extraction based on the InceptionV3 pretrained model and stacking additional layers to prepare the classifier:

# Get the InceptionV3 model so we can do transfer learning 
base_inception = InceptionV3(weights='imagenet',  
                             include_top = False,  
                             input_shape=(299, 299, 3)) 
 
 
# Add a global spatial average pooling layer 
out = base_inception.output 
out = GlobalAveragePooling2D()(out) 
out = Dense(512, activation='relu')(out) 
out = Dense(512, activation='relu')(out) 
total_classes = y_train_ohe.shape[1] 
 
predictions = Dense(total_classes,  
                    activation='softmax')(out) 

As shown in the preceding code snippet, Keras provides simple utilities to work with many pretrained models, and using them as feature extractors is as simple as setting the flag include_top to False. In the following code snippet, we prepare the final model by stacking the two sets of layers one on top of the other, and then freezing the layers from InceptionV3:

model = Model(inputs=base_inception.input,  
              outputs=predictions) 
 
# only if we want to freeze layers 
for layer in base_inception.layers: 
    layer.trainable = False 

We now have the model in place, all set to be trained on our Dog Breed Identification dataset. We train the model using the fit_generator() method to leverage the data augmentation prepared in the previous step. We set the batch size to 32, and train the model for 13 epochs. The following snippet sets the ball rolling:

batch_size = BATCH_SIZE 
train_steps_per_epoch = x_train.shape[0] // batch_size 
val_steps_per_epoch = x_val.shape[0] // batch_size 
 
history = model.fit_generator(train_generator, 
                              steps_per_epoch=train_steps_per_epoch, 
                              validation_data=val_generator, 
                              validation_steps=val_steps_per_epoch, 
                              epochs=15, 
                              verbose=1) 

Since we are saving the output of model parameters and performance after every epoch (the history object), we will now utilize it to understand the model performance. The following figure plots the model's train and test accuracies along with its loss performance:

The model achieves a commendable performance of more than 80% accuracy on both train and validation sets within just 15 epochs. The plot on the right-hand side shows how quickly the loss drops and converges to around 0.5. This is a clear example of how powerful, yet simple, transfer learning can be.

Training and validation performance is pretty good, but how about performance on unseen data? Since we already divided our original dataset into three separate portions. The important thing to remember here is that the test dataset has to undergo similar preprocessing as the training dataset. To account for this, we scale the test dataset as well, before feeding it into the function.

The model achieves an amazing 85% accuracy as well as a 0.85 F1 score on the test dataset. Given that we just trained for 15 epochs with minimal inputs from our side, transfer learning helped us achieve a pretty decent classifier:

The preceding image presents a visual proof of the model's performance. As we can see, in most of the cases the model is not only predicting the correct dog breed, it also does so with very high confidence.