Variables are containers used to store values. Variables are used as input to several other operations in a computational graph. A variable can be created using the tf.Variable() function, as shown in the following code:

x = tf.Variable(13)

Let's create a variable called W, using tf.Variable(), as follows:

W = tf.Variable(tf.random_normal([500, 111], stddev=0.35), name="weights")

As you can see in the preceding code, we create a variable, W, by randomly drawing values from a normal distribution with a standard deviation of 0.35.

What is that parameter called name in tf.Variable()?

It is used to set the name of the variable in the computational graph. So, in the preceding code, Python saves the variable as W but in the TensorFlow graph, it will be saved as weights.

We can also initialize a new variable with a value from another variable using initialized_value(). For instance, if we want to create a new variable called weights_2, using a value from the previously defined weights variable, it can be done as follows:

W2 = tf.Variable(weights.initialized_value(), name="weights_2")

However, after defining a variable, we need to initialize all of the variables in the computational graph. That can be done using tf.global_variables_initializer().

Once we create a session, first, we run the initialization operation, which will initialize all of the defined variables, and only then can we run the other operations, as shown in the following code:

x = tf.Variable(1212)
init = tf.global_variables_initializer()

with tf.Session() as sess:
  sess.run(init) 
  print sess.run(x)

We can also create a TensorFlow variable using tf.get_variable(). It takes the three important parameters, which are name, shape, and initializer.

Unlike tf.Variable(), we cannot pass the value directly to tf.get_variable(); instead, we use initializer. There are several initializers available for initializing values. For example, tf.constant_initializer(value) initializes the variable with a constant value, and tf.random_normal_initializer(mean, stddev) initializes the variable by drawing values from random normal distribution with a specified mean and standard deviation.

Variables created using tf.Variable() cannot be shared, and every time we call tf.Variable(), it will create a new variable. But tf.get_variable() checks the computational graph for an existing variable with the specified parameter. If the variable already exists, then it will be reused; otherwise, a new variable will be created:

W3 = tf.get_variable(name = 'weights', shape = [500, 111], initializer = random_normal_initializer()))

So, the preceding code checks whether there is any variable already existing with the given parameters. If yes, then it will reuse it; otherwise, it will create a new variable.

Since we are reusing variables using tf.get_variable(), in order to avoid name conflicts, we use tf.variable_scope, as shown in the following code. A variable scope is basically a name-scoping technique that just adds a prefix to the variable within the scope to avoid the naming clash:

with tf.variable_scope("scope"):
 a = tf.get_variable('x', [2])

with tf.variable_scope("scope", reuse = True):
 b = tf.get_variable('x', [2])

If you print a.name and b.name, then it will return the same name, which is scope/x:0. As you can see, we specified the reuse=True parameter in the variable scope named scope, which implies that the variables can be shared. If we don't set reuse = True, then it will give an error saying that the variable already exists.