CNN – Image data pre-processing with generators
The article aims to learn how to pre-processing the input image data to convert it into meaningful floating-point tensors for feeding into Convolutional Neural Networks. Just for the knowledge tensors are used to store data, they can be assumed as multidimensional arrays. A tensor representing a 64 X 64 image having 3 channels will have its dimensions (64, 64, 3). Currently, the data is stored on a drive as JPEG files, So let’s see the steps taken to achieve it.
Attention reader! Don’t stop learning now. Get hold of all the important Machine Learning Concepts with the Machine Learning Foundation Course at a student-friendly price and become industry ready.
- Read the picture files (stored in data folder).
- Decode the JPEG content to RGB grids of pixels with channels.
- Convert these into floating-point tensors for input to neural nets.
- Rescale the pixel values (between 0 and 255) to the [0, 1] interval (as training neural networks with this range gets efficient).
It may seem a bit fussy, but Keras has utilities to take over this whole algorithm and do the heavy lifting for you. Keras has a module with image-processing helping tools, located at
keras.preprocessing.image. It contains the class ImageDataGenerator, which lets you quickly set up Python generators that can automatically turn image files on disk into batches of preprocessed tensors.
It yields batches of 150 × 150 RGB images of shape (20, 150, 150, 3) and binary labels of shape (20, ).
Fitting the model:
Let’s fit the model to the data using the generator, it is done using the fit_generator method, the equivalent of fit for data generators like given below. Its first argument is a Python generator that will yield batches of inputs and targets indefinitely because the data is being generated endlessly, the Keras model needs to know how many samples to draw from the generator before declaring an epoch over. This is the role of the steps_per_epoch argument.
Now deciding the steps_per_epoch parameter, as we have total of 2000 training images and each batch is of size 20, hence, the steps_per_epoch will be 2000 / 20 = 100.