As mentioned earlier, the convolution step got its name from the convolution operation. The main purpose of having these convolution steps is to extract features from the input images and then feed them to a linear classifier.

In natural images, features could be anywhere in the image. For example, edges could be in the middle or at the corner of the images, so the whole idea of stacking a bunch of convolution steps is to be able to detect these features anywhere in the image.

It's very easy to define a convolution step in TensorFlow. For example, if we wanted to apply 20 filters each of size 5 by 5 to the input layer with a ReLU activation function, then we can use the following line of code to do that:

conv_layer1 = tf.layers.conv2d(
 inputs=input_layer,
 filters=20,
 kernel_size=[5, 5],
 padding="same",
 activation=tf.nn.relu)

The first parameter for this conv2d function is the input layer that we have defined in the preceding code, which has the appropriate shape, and the second argument is the filters argument which specifies the number of filters to be applied to the image where the higher the number of filters, the more features are extracted from the input image. The third parameter is the kernel_size, which represents the size of the filter or the feature detector. The padding parameters specifies where we use "same" here to introduce zero-padding to the corner pixels of the input image. The last argument specifies the activation function that should be used for the output of the convolution operation.

So, in our MNIST example, the input tensor will have the following shape:

[batch_size, 28, 28, 1]

And the output tensor for this convolution step will have the following shape:

[batch_size, 28, 28, 20]

The output tensor has the same dimensions as the input images, but now we have 20 channels that represent applying the 20 filters to the input image.