The auto encoder neural network is an unsupervised learning algorithm that applies backpropagation setting the target values to be equal to the inputs y⁽ⁱ⁾ = x⁽ⁱ⁾. Auto encoder tries to learn a function h_w,b(x) ≈ x, means it tries to learn an approximation to the identity function, so that output    that is similar to x.

Though trying to replicate the identity function seems trivial function to learn, by placing the constraints on the network, such as by limiting number of hidden units, we can discover interesting structures about the data. Let's say input picture of size 10 x 10 pixels has intensity values which have, altogether, 100 input values, the number of neurons in the second layer (hidden layer) is 50 units, and the output layer, finally, has 100 units of neurons as we need to pass the image to map it to itself and while achieving this representation in the process we would force the network to learn a compressed representation of the input, which is hidden unit activations a⁽²⁾ ε  R¹⁰⁰, with which we must try to reconstruct the 100 pixel input x. If the input data is completely random without any correlations, and so on. it would be very difficult to compress, whereas if the underlying data have some correlations or detectable structures, then this algorithm will be able to discover the correlations and represent them compactly. In fact, auto encoder often ends up learning a low-dimensional representation very similar to PCAs.