We will now combine all of the individual components created earlier to build the complete graph:

train_graph = tf.Graph()
with train_graph.as_default(): 
    input_data, targets, learning_rate, dropout_probs, 
                            en_len, max_en_len, fr_len =model_inputs()
logits_tr, logits_inf = seq2seq_model(tf.reverse(input_data, [-1]), targets, dropout_probs,
                        fr_len,en_len,max_en_len,
                        len(en_word2int)+1,rnn_len, n_layers,
                        en_word2int,batch_size)
logits_tr = tf.identity(logits_tr.rnn_output, 'logits_tr')
logits_inf = tf.identity(logits_inf.sample_id, name='predictions')
seq_masks = tf.sequence_mask(en_len, max_en_len, dtype=tf.float32, name='masks')
with tf.name_scope("optimizer"):
    tr_cost = sequence_loss(logits_tr,targets,seq_masks)
    optimizer = tf.train.AdamOptimizer(learning_rate)
    gradients = optimizer.compute_gradients(tr_cost)
    capped_gradients = [(tf.clip_by_value(gradient, -5., 5.), var) for gradient, var in gradients 
                        if gradient is not None]
train_op = optimizer.apply_gradients(capped_gradients)
tf.summary.scalar("cost", tr_cost)
print("Graph created.")

We first use the seq2seq_model function to create the encoder-decoder network with attention. We compute the loss using the sequence_loss function of the seq2seq TensorFlow library with the output logits values. We also mask out the padding in the loss calculation. Finally, we use AdamOptimizer as our optimizer on the loss.