The CNN generally ends with a fully connected multilayered perceptron using softmax activation in the output layer. Here, each neuron in the previous convoluted-flattened layer is connected to every neuron in the next (fully connected) layer.
The key purpose of the fully convoluted layer is to use the features generated in the convolution and pooling stage to classify the given input image into various outcome classes (here, 10L). It also helps in learning the non-linear combinations of these features to define the outcome classes.
In this chapter, we use two fully connected layers for optimization. This function is primarily used to define the mathematical formulas in the TensorFlow graph, which is later used in actual computation during optimization.