As you now know, placeholders are defined using the tf.placeholder function. When we define placeholders, we need to decide what their shape and type should be, according to the shape and type of the data that will be fed through them upon execution.

Let's start with a simple example. In the following code, we will define the same graph that was shown in the previous section for evaluating . This time, however, we use placeholders for the scalars a, b, and c. Also, we store the intermediate tensors associated with and , as follows:

>>> import tensorflow as tf
>>>
>>> g = tf.Graph()
>>> with g.as_default():
...     tf_a = tf.placeholder(tf.int32, shape=[],
...                           name='tf_a')
...     tf_b = tf.placeholder(tf.int32, shape=[],
...                           name='tf_b')
...     tf_c = tf.placeholder(tf.int32, shape=[],
...                           name='tf_c')
...
...     r1 = tf_a-tf_b
...     r2 = 2*r1
...     z  = r2 + tf_c

In this code, we defined three placeholders, named tf_a, tf_b, and tf_c, using type tf.int32 (32-bit integers) and set their shape via shape=[] since they are scalars (tensors of rank 0). In the current book, we always precede the placeholder objects with tf_ for clarity and to be able to distinguish them from other tensors.

Note that in the previous code example, we were dealing with scalars, and therefore, their shapes were specified as shape=[]. However, it is very straightforward to define placeholders of higher dimensions. For example, a rank 3 placeholder of type float and shape 3 x 4 x 5 can be defined as tf.placeholder(dtype=tf.float32, shape=[3, 4, 5]).

When we execute a node in the graph, we need to create a python dictionary to feed the values of placeholders with data arrays. We do this according to the type and shape of the placeholders. This dictionary is passed as the input argument feed_dict to a session's run method.

In the previous graph, we added three placeholders of the type tf.int32 to feed scalars for computing z. Now, in order to evaluate the result tensor z, we can feed arbitrary integer values (here, 1, 2, and 3) to the placeholders, as follows:

>>> with tf.Session(graph=g) as sess:
...     feed = {tf_a: 1,
...             tf_b: 2,
...             tf_c: 3}
...     print('z:',
...           sess.run(z, feed_dict=feed))
z: 1

This means that having extra arrays for placeholders does not cause any error; it is just redundant to do so. However, if a placeholder is needed for the execution of a particular node, and is not provided via the feed_dict argument, it will cause a runtime error.

Sometimes, when we are developing a neural network model, we may deal with mini-batches of data that have different sizes. For example, we may train a neural network with a specific mini-batch size, but we want to use the network to make predictions on one or more data input.

A useful feature of placeholders is that we can specify None for the dimension that is varying in size. For example, we can create a placeholder of rank 2, where the first dimension is unknown (or may vary), as shown here:

>>> import tensorflow as tf
>>>
>>> g = tf.Graph()
>>>
>>> with g.as_default():
...     tf_x = tf.placeholder(tf.float32,
...                           shape=[None, 2],
...                           name='tf_x')
...     
...     x_mean = tf.reduce_mean(tf_x,
...                             axis=0,
...                             name='mean')

Then, we can evaluate x_mean with two different input, x1 and x2, which are NumPy arrays of shape (5, 2) and (10, 2), as follows:

>>> import numpy as np
>>> np.random.seed(123)
>>> np.set_printoptions(precision=2)
>>> with tf.Session(graph=g) as sess:
...     x1 = np.random.uniform(low=0, high=1,
...                            size=(5, 2))
...     print('Feeding data with shape ', x1.shape)
...     print('Result:', sess.run(x_mean,
...                               feed_dict={tf_x: x1}))
...     x2 = np.random.uniform(low=0, high=1,
...                            size=(10,2))
...     print('Feeding data with shape', x2.shape)
...     print('Result:', sess.run(x_mean,
...                               feed_dict={tf_x: x2}))

This prints the following output:

Feeding data with shape (5, 2)
Result: [ 0.62  0.47]
Feeding data with shape (10, 2)
Result: [ 0.46  0.49]

Lastly, if we try printing the object tf_x, we will get Tensor("tf_x:0", shape=(?, 2), dtype=float32), which shows that the shape of this tensor is (?, 2).