The attention context is obtained by first concatenating the output vector generated by the bidirectional GRU layer of the CBHG-based encoder with the output of the attention RNN. Then, the resulting vector feeds a tanh activated dense layer, followed by another dense layer. A softmax layer allows for getting the activation weights that give the attention context, through a dot product with the encoder output vector:

def get_attention_context(encoder_output,attention_rnn_output):
    attention_input=Concatenate(axis=-1)([encoder_output, 
                                          attention_rnn_output])
    e=Dense(10, activation = "tanh")(attention_input)
    energies=Dense(1, activation = "relu")(e)
    attention_weights=Activation('softmax')(energies)
    context=Dot(axes = 1)([attention_weights,
                           encoder_output])
    
    return context