A lot of popular models that have been proposed for neural machine translation across various domains belong to the encoder-decoder architecture family. However, this architecture restricts the encoder to encoding the input sequence to a fixed-length representation, which results in deteriorated performance for lengthy input sequences. One of the ways to overcome this bottleneck in performance is to use the attention mechanism within the sequences, which makes the network learn to pay selective attention to the inputs that are relevant for predicting a target output. Most importantly, attention allows the network to encode the input sequence into a sequence of vectors and choose among these vectors while decoding, thus freeing the network to encode all the information in one fixed-length vector. Although the idea of the attention mechanism originated from the context of neural machine translation, it can be extended to a wide range of problems, such as image captioning and descriptions, speech recognition, and text summarization.