Defining the weights

First, let's define all the weights. The weights of the update gate are defined as follows:

 Uz = tf.get_variable("Uz", [vocab_size, hidden_size], initializer=init)
Wz = tf.get_variable("Wz", [hidden_size, hidden_size], initializer=init)
bz = tf.get_variable("bz", [hidden_size], initializer=init)

The weights of the reset gate are defined as shown:

Ur = tf.get_variable("Ur", [vocab_size, hidden_size], initializer=init)
Wr = tf.get_variable("Wr", [hidden_size, hidden_size], initializer=init)
br = tf.get_variable("br", [hidden_size], initializer=init)

The weights of the content state are defined as follows:

Uc = tf.get_variable("Uc", [vocab_size, hidden_size], initializer=init)
Wc = tf.get_variable("Wc", [hidden_size, hidden_size], initializer=init)
bc = tf.get_variable("bc", [hidden_size], initializer=init)

Weights of the output layer are defined as follows:

V = tf.get_variable("V", [hidden_size, vocab_size], initializer=init)
by = tf.get_variable("by", [vocab_size], initializer=init)
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset