A great feature of TensorFlow is TensorBoard, which is a module for visualizing the graph as well as visualizing the learning of a model. Visualizing the graph allows us to see the connection between nodes, explore their dependencies, and debug the model if needed.

So let's visualize a network that we've already built, one which consists of a generator and a classifier part. We'll repeat some code that we previously used for defining the helper functions. So, revisit the Reusing variables section earlier in this chapter, for the function definitions of build_generator and build_classifier. Using these two helper functions, we will build the graph as follows:

>>> batch_size=64
>>> g = tf.Graph()
>>>
>>> with g.as_default():
...     tf_X = tf.placeholder(shape=(batch_size, 100),
...                           dtype=tf.float32,
...                           name='tf_X')
...
...     ## build the generator
...     with tf.variable_scope('generator'):
...         gen_out1 = build_generator(data=tf_X,
...                                    n_hidden=50)
...     
...     ## build the classifier
...     with tf.variable_scope('classifier') as scope:
...         ## classifier for the original data:
...         cls_out1 = build_classifier(data=tf_X,
...                                     labels=tf.ones(
...                                        shape=batch_size))
...         
...         ## reuse the classifier for generated data
...         scope.reuse_variables()
...         cls_out2 = build_classifier(data=gen_out1[1],
...                                     labels=tf.zeros(
...                                         shape=batch_size))

Note that no changes were needed so far for building the graph. So after building the graph, its visualization is straightforward. The following lines of code export the graph for visualization purposes:

>>> with tf.Session(graph=g) as sess:
...     sess.run(tf.global_variables_initializer())
...     
...     file_writer = tf.summary.FileWriter(
...                              logdir='./logs/', graph=g)

This will create a new directory: logs/. Now, we just need to run the following command in a Linux or macOS Terminal:

tensorboard --logdir logs/

This command will print a message, which is a URL address. You can try launching TensorBoard by copying the link, for example, http://localhost:6006/#graphs, and pasting it into your browser's address bar. You should see the graph that corresponds to this model, as shown in the following figure:

The large rectangular boxes indicate the two subnetworks that we built: generator and classifier. Since we used the tf.variable_scope function when we built this graph, all the components of each of these subnetworks are grouped into those rectangular boxes, as shown in the previous figure.

We can expand these boxes to explore their details: using your mouse, click on the plus sign on the top-right corner of these boxes to expand them. Doing this, we can see the details of the generator subnetwork, as shown in the following figure:

By exploring this graph, we can easily see that the generator has two weight tensors, named w1 and w2. Next, let's expand the classifier subnetwork, as shown in the following figure:

As you can see in this figure, the classifier has two sources of input, where one input comes from the tf_X placeholder and the other one is in fact the output of the generator subnetwork.