Our first task is to define the model which can perform regression on provided MNIST dataset. So we will create a TensorFlow model with 2 hidden layers that are in a Fully Connected Neural Network. You may also hear it referred to as MLP.
The model will perform the operation that will fit the Equation 2.1 where y is the label, x is the image, W is the weight which model will learn and b is the bias which will also be learned by the model:
Each iteration will try to generalize the values of weight and bias and reduce the error rate. Also keep in mind, that we need to ensure that the model is not overfitting which may lead to wrong predictions for the unseen dataset. We'll show you how to code this and visualize the progress to aid in your intuition of model performance.