As we have learned, every computation in TensorFlow is represented by a computational graph. They consist of several nodes and edges, where nodes are mathematical operations, such as addition and multiplication, and edges are tensors. Computational graphs are very efficient at optimizing resources and promote distributed computing.

A computational graph consists of several TensorFlow operations, arranged in a graph of nodes.

Let's consider a basic addition operation:

import tensorflow as tf

x = 2
y = 3
z = tf.add(x, y, name='Add')

The computational graph for the preceding code would look like the following:

A computational graph helps us to understand the network architecture when we work on building a really complex neural network. For instance, let's consider a simple layer, . Its computational graph would be represented as follows:

There are two types of dependency in the computational graph, called direct and indirect dependency. Say we have the b node, the input of which is dependent on the output of the a node; this type of dependency is called direct dependency, as shown in the following code:

a = tf.multiply(8,5)
b = tf.multiply(a,1)

When the b node doesn't depend on the a node for its input, it is called indirect dependency, as shown in the following code:

a = tf.multiply(8,5)
b = tf.multiply(4,3)

So, if we can understand these dependencies, we can distribute the independent computations in the available resources and reduce the computation time. Whenever we import TensorFlow, a default graph is created automatically and all of the nodes we create are associated with the default graph. We can also create our own graphs instead of using the default graph, and this is useful when building multiple models that do not depend on one another in one file. A TensorFlow graph can be created using tf.Graph(), as follows:

graph = tf.Graph()

with graph.as_default():
     z = tf.add(x, y, name='Add')

If we want to clear the default graph (that is, if we want to clear the previously defined variables and operations), then we can do that using tf.reset_default_graph().