Every computation in TensorFlow is represented by a computational graph. It consists of several nodes and edges, where nodes are the mathematical operations, such as addition, multiplication, and so on, and edges are the tensors. A computational graph is very efficient in optimizing resources and it also promotes distributed computing.
A computational graph with the operations on the node and tensors to its edges will only be created, and in order to execute the graph, we use a TensorFlow session.
A TensorFlow session can be created using tf.Session(), and it will allocate the memory for storing the current value of the variable.
Variables are the containers used to store values. Variables will be used as input to several other operations in the computational graph. We can think of placeholders as variables, where we only define the type and dimension, but will not assign the value. Values for the placeholders will be fed at runtime. We feed the data to the computational graphs using placeholders. Placeholders are defined with no values.
TensorBoard is TensorFlow's visualization tool that can be used to visualize the computational graph. It can also be used to plot various quantitative metrics and the results of several intermediate calculations. When we are training a really deep neural network, it would become confusing when we have to debug the model. As we can visualize the computational graph in TensorBoard, we can easily understand, debug and optimize such complex models. It also supports sharing.
Scoping is used to reduce complexity and helps us to better understand the model by grouping the related nodes together. Having a name scope helps us in grouping similar operations in a graph. It comes in handy when we are building a complex architecture. Scoping can be created usingtf.name_scope().
Eager execution in TensorFlow is more Pythonic and allows for rapid prototyping. Unlike the graph mode, where we need to construct a graph every time to perform any operations, eager execution follows the imperative programming paradigm, where any operations can be performed immediately without having to create a graph, just like we do in Python.
