In step 1, while configuring generic neural network parameters, we set the default learning rate as shown here:

configBuilder.updater(new AdaGrad(learningRate));

The Adagrad optimizer is based on how frequently a parameter gets updated during training. Adagrad is based on a vectorized learning rate. The learning rate will be small when there are many updates received. This is crucial for high-dimensional problems. Hence, this optimizer can be a good fit for our autoencoder use case.

We are performing dimensionality reduction at the input layers in an autoencoder architecture. This is also known as encoding the data. We want to ensure that the same set of features are decoded from the encoded data. We calculate reconstruction errors to measure how close we are compared to the real feature set before encoding. In step 2, we are trying to encode the data from a higher dimension (784) to a lower dimension (10).