The LSTM has a similar structure to RNN, however, the basic cell is very different as traditional RNN uses single multi-layer perceptron (MLP), whereas a single cell of LSTM includes four input layers interacting with each other. These three layers are:

• forget gate
• input gate
• output gate

The forget gate in LSTM decides which information to throw away and it depends on the last hidden state output h_t-1, X_t, which represents input at time t.

In the earlier figure, C_t represents cell state at time t. The input data is represented by X_t and the hidden state is represented as h_t-1. The earlier layer can be formulated as:

The input gate decides update values and decides the candidate values of the memory cell and updates the cell state, as shown in the following figure:

• The input i_t at time t is updated as:

• The expected value of current state and the output from input gate is used to update the current state at time t as:

The output gates, as shown in the following figure, compute the output from the LSTM cell based on input X_t , previous layer output h_t-1, and current state C_t:

The output based on output gate can be computed as follows: