In this recipe, we will learn about two key configuration hyperparameters of CNN, which are strides and padding. Strides are used mainly to reduce the size of the output volume. Padding is another technique that lets us preserve the dimensions of the input volume in the output volume, thus enabling us to extract the low-level features efficiently.
Strides: Stride, in very simple terms, means the step of the convolution operation. Stride specifies the amount by which filters convolve around the input. For example, if we specify the value of stride argument as 1, that means the filter will shift one unit at a time over the input matrix.
Strides can be used for multiple purposes, primarily the following:
- To avoid feature overlapping
- To achieve smaller spatial dimensionality of the output volume
In the following diagram, you can see an example of a convolution operation on a 7 × 7 input data with a filter size of a 3 × 3 and a stride of 1:
Padding: For better modeling performance, we need to preserve the information about the low-level features of the input volume in the early layers of the network. As we keep applying convolutional layers, the size of the output volume decreases faster. Also, the pixels at corners of the input matrix are traversed a lesser number of times, compared to the pixels in the middle of the input matrix, which leads to throwing away a lot of the information near the edge of the image. To avoid this, we use zero padding. Zero padding symmetrically pads the input volume with zeros around the border.
There are two types of padding:
-
Valid: If we specify valid padding, our convolutional layer is not going to pad anything around the input matrix and the size of the output volume will keep decreasing on adding layers.
-
Same: This pads the original input with zeros around the edges of the input matrix before we convolve it so that the output size is the same size as the input size.
In the following screenshot, we can see a pictorial representation of zero padding:
Now that we are aware of the concepts of strides and padding, let's move further to the implementation part.