We will have a lot of information in the cell state (memory). The output gate is responsible for deciding what information should be taken from the cell state to give as an output. Consider the following sentence:

Zayn's debut album was a huge success. Congrats ____.

The output gate will look up all the information in the cell state and select the correct information to fill the blank. Here, congrats is an adjective that is used to describe a noun. So, the output gate will predict Zayn (noun) to fill the blank. Similar to other gates, it is also controlled by a sigmoid function. The output gate at time step is expressed as follows:

Here, the following applies:

• is the input-to-hidden weights of the output gate
• is the hidden-to-hidden weights of the output gate
• is the bias of the output gate

The output gate is shown in the following diagram: