Once we have the frozen graph, we can further optimize the file for inference-only purposes by removing the parts of the graph that are only needed during training. According to the documentation, this includes:

• Removing training-only operations, such as checkpoint saving
• Stripping out parts of the graph that are never reached
• Removing debug operations, such as CheckNumerics
• Folding batch normalization ops into the pre-calculated weights
• Fusing common operations into unified versions

TensorFlow provides optimize_for_inference_lib in tensorflow.python.tools for this purpose:

# Optimize for inference 
input_graph_def = tf.GraphDef() with tf.gfile.Open(output_frozen_graph_name, "r") as f: data = f.read() input_graph_def.ParseFromString(data) 
output_graph_def = optimize_for_inference_lib.optimize_for_inference( input_graph_def, ["a", "b"], 
# an array of the input node(s) ["c"], 
# an array of output nodes tf.int32.as_datatype_enum)

# Save the optimized graph f = tf.gfile.FastGFile(output_optimized_graph_name, "w") f.write(output_graph_def.SerializeToString()) tf.train.write_graph(output_graph_def, './', output_optimized_graph_name)

Take note of the input and output nodes in the preceding code. Our graph only has one input node, named I, and one output node, named O. These names correspond to the names you use when you define your tensors. You should adjust these based on your graph in case you are using a different one.

Now we have a binary file, called optimized_tfdroid.pb, which means we are ready to build our Android app. If you got an exception when creating optimized_tfdroid.pb, you can use tfdroid.somewhat, which is an unoptimized version of the model – it is fairly large.