Now we will perform forward propagation and predict the output, , and initialize a list called y_hat for storing the output:

y_hat = []

For each iteration, we compute the output and store it in the y_hat list:

for i in range(batch_size):

We initialize the hidden state and cell state:

    hidden_state = np.zeros([1, hidden_layer], dtype=np.float32) 
    cell_state = np.zeros([1, hidden_layer], dtype=np.float32)

We perform the forward propagation and compute the hidden state and cell state of the LSTM cell for each time step:

    for t in range(window_size):
        cell_state, hidden_state = LSTM_cell(tf.reshape(input[i][t], (-1, 1)), hidden_state, cell_state)

We know that output can be computed as follows:

Compute y_hat, and append it to the y_hat list:

    y_hat.append(tf.matmul(hidden_state, V) + b_v)