Run the following steps to implement fine-tuning (with transfer learning) using keras.

1. Define the train and test directories along with the size that the input images will get resized to before training:

train_dir = 'images/flower_photos/train'
test_dir = 'images/flower_photos/test'
image_size = 224

2. Load the pretrained VGG16 network, but without the top FC layers:

vgg_conv = VGG16(weights='imagenet', include_top=False, input_shape=(image_size, image_size, 3))

3. Freeze all the convolutional layers except the last two convolutional layers, along with the FC layers, to reuse the pretrained weights for those layers (without retraining them):

for layer in vgg_conv.layers[:-2]:  
   layer.trainable = False 

4. Now let's verify the status (trainable or not) of each the layers using the following code:

for layer in vgg_conv.layers:
  print(layer, layer.trainable)

5. Now create a sequential model with Keras and add the VGG16 base model (with convolutional layers) to it. Then add a couple of new FC layers. Print the model summary to see the model structure, along with the number of trainable parameters:

model = models.Sequential()
model.add(vgg_conv)
model.add(layers.Flatten())
model.add(layers.Dense(1024, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(3, activation='softmax'))
model.summary()
# Total params: 40,408,899
# Trainable params: 25,694,211
# Non-trainable params: 14,714,688

6. Load the training and test input images. Set the validation split as 0.2—that is, 20% of the training images will be held out (as a validation dataset) to evaluate the classifier model that is trained on the remaining 80% of the training images (validation is a popular machine learning technique that helps improve the generalizability of the model by reducing model variance; the model then becomes more likely to achieve higher accuracy on unseen test images). Define the train batch size (that is, the number of training images to be passed in one forward/backward pass):

train_datagen = ImageDataGenerator(rescale=1./255, validation_split=0.2) # set validation split
test_datagen = ImageDataGenerator(rescale=1./255)
train_batchsize = 100

7. Define the training, validation, and test data generators to read/generate batches of images and labels from the appropriate directories. Configure the model created previously for training:
   

train_generator = train_datagen.flow_from_directory(
        train_dir,
        target_size=(image_size, image_size),
        batch_size=train_batchsize,
        class_mode='categorical',
        subset='training')

validation_generator = train_datagen.flow_from_directory(
        train_dir,
        target_size=(image_size, image_size),
        batch_size=train_batchsize,
        class_mode='categorical',
        classes = ['roses', 'sunflowers', 'tulips'],
        subset='validation') # set as validation data

test_generator = test_datagen.flow_from_directory(
        test_dir,
        target_size=(image_size, image_size),
        batch_size=1,
        class_mode='categorical',
        classes = ['roses', 'sunflowers', 'tulips'],
        shuffle=False)

model.compile(loss='categorical_crossentropy',
              optimizer=optimizers.RMSprop(lr=1e-5),
              metrics=['acc'])

8. Plot the model architecture using the following line of code:

plot_model(model, to_file='images/model.png')

If you run the preceding line of code, the model structure will be saved to the .png file provided. The model architecture saved will look like the one shown in the following diagram:

9. Train the model with the following code snippet. Save the weights obtained:

history = model.fit_generator(
      train_generator,
      steps_per_epoch=train_generator.
            samples/train_generator.batch_size,
      epochs=20,
      validation_data=validation_generator,
      validation_steps=validation_generator.
            samples/validation_generator.batch_size,
      verbose=1)
model.save('all_freezed.h5')

10. Extract the accuracy and loss values from the training history:

acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

11. Get the test filenames and the corresponding ground truths from the generator and map the class indices to labels:

test_generator.reset()
fnames = test_generator.filenames
ground_truth = test_generator.classes
label2index = test_generator.class_indices
index2label = dict((v,k) for k,v in label2index.items())

12. Finally, predict the labels of the test images using the test-data generator. Get the predictions from the model using the generator using the following code snippets, compare them with the corresponding ground truths, and compute the number of mistakes (errors) made by the model:

predictions = model.predict_generator(test_generator, steps=len(fnames))
predicted_classes = np.argmax(predictions,axis=-1)
predicted_classes = np.array([index2label[k] for k in predicted_classes])
ground_truth = np.array([index2label[k] for k in ground_truth])
errors = np.where(predicted_classes != ground_truth)[0]
print("No of errors = {}/{}".format(len(errors),test_generator.samples))
# No of errors = 45/300