What’s the purpose of tf.app.flags in TensorFlow?

TensorFlow is the most popular Python Machine Learning framework developed by Google. TensorFlow makes it easy to implement Neural Networks and their supporting architectures such as Convolution Neural Networks and Recurrent Neural Networks. TensorFlow is an Open Source and is built using lots of functionalities. Sometimes it’s tricky to understand what each method performs. 

In this article, we shall see the purpose of the tf.app.flags use a case in TensorFlow.

What’s the purpose of tf.app.flags in TensorFlow?

Before we proceed with the code sample, let’s understand why we need flags in the program. Flag terminology was first brought up during C programming, but it is not just restricted to C programming. In programming, a Flag usually indicates a program that a certain condition should be satisfied. Whereas speaking of Flags in TensorFlow, they are defined to meet some conditions but in addition, they are used to config command line arguments. In Python, we mostly achieve this task using argsparse. TensorFlow Flags are mainly used when you need to config the Hyperparameters through the command line. 

Let’s look at an example of tf.app.flags. 

Note: tf.app.flags is not directly supported in TensorFlow < 2.0. Thus we call it using tf.compat.v1.app.flags. 

In the below example, we implement or define flags for the Hyperparameters used to train Deep Neural Networks. 

Python3

import tensorflow as tf

flag = tf.compat.v1.app.flags
FLAGS = flag.FLAGS

flag.DEFINE_integer('n_epochs', 100, 'Number of steps')
flag.DEFINE_float('lr', 0.005, 'Learning rate')
flag.DEFINE_integer('batch_size', 32, 'Batch Size')

print("Default Learning Rate:", FLAGS.lr)
print("Default Epochs:", FLAGS.n_epochs)
print("Default Batch Size:", FLAGS.batch_size)

Output:

Default Learning Rate: 0.005
Default Epochs: 100
Default Batch Size: 32

As we mentioned earlier that tf.app.flags are mainly used for Command Line Parser. We shall now see how you can carry out tf.app.flags using the command line. 

Command Line Parser in Tensorflow using tf.app.flags

Considering the above code example, save the file name with the .py file. In our case it is flag_check.py. In most of the Machine Learning statements, the hyperparameters have some default value, which may or may not be updated. Say you need to update the default hyperparameters. 

Update the float Hyperparameter. Even if you pass an integer value, the result will be in float itself. 

Python3

python3 flag_check.py --lr 1

Output:

Default Learning Rate: 1.0
Default Epochs: 100
Default Batch Size: 32

In this example, we update the integer flag value i.e., batch_size. 

Python3

python3 flag_check.py --batch_size 128

Output:

Default Learning Rate: 0.005
Default Epochs: 100
Default Batch Size: 128

Selecting Multiple Flags together in the command line is also possible. 

Python3

python3 flag_check.py --batch_size 128 --lr 1.0 --n_epochs 50

Output:

Default Learning Rate: 1.0
Default Epochs: 50
Default Batch Size: 128