We will now build the encoder network with a slight modification to the original architecture that was described in the overview section. We will use a bidirectional RNN in place of a unidirectional one, thereby capturing both forward and backward dependencies in the input:

def get_rnn_cell(rnn_cell_size,dropout_prob):
    rnn_c = GRUCell(rnn_cell_size)
    rnn_c = DropoutWrapper(rnn_c, input_keep_prob = dropout_prob)
    return rnn_c

def encoding_layer(rnn_cell_size, sequence_len, n_layers, rnn_inputs, dropout_prob):
    for l in range(n_layers):
        with tf.variable_scope('encoding_l_{}'.format(l)):
            rnn_fw = get_rnn_cell(rnn_cell_size,dropout_prob)
            rnn_bw = get_rnn_cell(rnn_cell_size,dropout_prob)
    encoding_output, encoding_state = tf.nn.bidirectional_dynamic_rnn(rnn_fw, rnn_bw, rnn_inputs,
                                        sequence_len,dtype=tf.float32)
    encoding_output = tf.concat(encoding_output,2)
    return encoding_output, encoding_state

We used a GRU as the recurrent cell of the RNN. The input to the rnn_inputs encoder will be the embeddings of the input sentence. The other arguments to the encoding_layer function are the cell size, sequence length, and the dropout probability. We will later set the sequence length to be equal to the maximum length of the French text.