Multiple Labels Using Convolutional Neural Networks

Tensorflow and Keras are modules in Python which are used for Deep Learning Neural Networks. We train the neural network to recognize or identify the labels correctly with good confidence.

In this post, we will build, train and test the Deep Learning classifier which is used to recognize Hindu Deities. It will be solved in 3 parts.

  • We build a Deep Learning Dataset using Images.
  • We train a Deity detector using deep learning.
  • We test with example images to check whether the classifier recognizes the labels properly or not.

Part 1: Build a Deep Learning Dataset using Images
Our Dataset contains images of Lord Shiva and Lord Ganpati
Lord Ganpati Dataset:


Lord Shiva Dataset:

  • Lord Ganpati Dataset(461 images)
  • Lord Shiva Dataset(461 images)

Before training the classifier, we need to define the network architecture.
Code: Network architecture

filter_none

edit
close

play_arrow

link
brightness_4
code

# We import the necessary packages
from keras.models import Sequential
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.layers.core import Activation
from keras.layers.core import Flatten
from keras.layers.core import Dense
from keras import backend as K
  
class CNN:
    @staticmethod
    def build(width, height, depth, classes):
        # initialize the model
        model = Sequential()
        inputShape = (height, width, depth)
  
        # if we are using "channels first", update the input shape
        if K.image_data_format() == "channels_first":
            inputShape = (depth, height, width)
  
        # first set of CONV => RELU => POOL layers
        model.add(Conv2D(20, (5, 5), padding ="same",
            input_shape = inputShape))
        model.add(Activation("relu"))
        model.add(MaxPooling2D(pool_size =(2, 2), strides =(2, 2)))
  
        # second set of CONV => RELU => POOL layers
        model.add(Conv2D(50, (5, 5), padding ="same"))
        model.add(Activation("relu"))
        model.add(MaxPooling2D(pool_size =(2, 2), strides =(2, 2)))
  
        # first (and only) set of FC => RELU layers
        model.add(Flatten())
        model.add(Dense(500))
        model.add(Activation("relu"))
  
        # softmax classifier
        model.add(Dense(classes))
        model.add(Activation("softmax"))
  
        # return the constructed network architecture
        return model

chevron_right


Part 2: Train the Classifier using the Image Dataset
We will train our classifier with gs.model over the datasets to recognize the Images
Code: Training the classifier



filter_none

edit
close

play_arrow

link
brightness_4
code

# We set the matplotlib backend so figures can be saved in the background
import matplotlib
matplotlib.use("Agg")
  
# We import the necessary packages
from keras.preprocessing.image import ImageDataGenerator
from keras.optimizers import Adam
from sklearn.model_selection import train_test_split
from keras.preprocessing.image import img_to_array
from keras.utils import to_categorical
from deep_learning.cnn import CNN
from imutils import paths
import matplotlib.pyplot as plt
import numpy as np
import argparse
import random
import cv2
import os
  
# We construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-d", "--dataset", required = True,
    help ="path to input dataset")
ap.add_argument("-m", "--model", required = True,
    help ="path to output model")
ap.add_argument("-p", "--plot", type = str, default ="plot.png",
    help ="path to output loss / accuracy plot")
args = vars(ap.parse_args())
  
# We initialize the number of epochs to train for,
# initia learning rate, and batch size
EPOCHS = 25
INIT_LR = 1e-3
BS = 32
  
# We initialize the data and labels
print("[INFO] loading images...")
data = []
labels = []
  
# We grab the image paths and randomly shuffle them
imagePaths = sorted(list(paths.list_images(args["dataset"])))
random.seed(42)
random.shuffle(imagePaths)
  
# We loop over the input images
for imagePath in imagePaths:
    # We load the image, pre-process it, and store it in the data list
    image = cv2.imread(imagePath)
    image = cv2.resize(image, (28, 28))
    image = img_to_array(image)
    data.append(image)
  
    # We extract the class label from the image
    # path and update the labels list
    label = imagePath.split(os.path.sep)[-2]
    label = 1 if label == "ganpati" else 0
    labels.append(label)
  
# We scale the raw pixel intensities to the range [0, 1]
data = np.array(data, dtype ="float") / 255.0
labels = np.array(labels)
  
# We partition the data into training and testing splits using 
# 75 % of the data for training and the remaining 25 % for testing
(trainX, testX, trainY, testY) = train_test_split(data,
    labels, test_size = 0.25, random_state = 42)
  
# We convert the labels from integers to vectors
trainY = to_categorical(trainY, num_classes = 2)
testY = to_categorical(testY, num_classes = 2)
  
# We construct the image generator for data augmentation
aug = ImageDataGenerator(rotation_range = 30, width_shift_range = 0.1,
    height_shift_range = 0.1, shear_range = 0.2, zoom_range = 0.2,
    horizontal_flip = True, fill_mode ="nearest")
  
# We initialize the model
print("[INFO] compiling model...")
model = CNN.build(width = 28, height = 28, depth = 3, classes = 2)
opt = Adam(lr = INIT_LR, decay = INIT_LR / EPOCHS)
model.compile(loss ="binary_crossentropy", optimizer = opt,
    metrics =["accuracy"])
  
# We train the network
print("[INFO] training network...")
H = model.fit_generator(aug.flow(trainX, trainY, batch_size = BS),
    validation_data =(testX, testY), steps_per_epoch = len(trainX) // BS,
    epochs = EPOCHS, verbose = 1)
  
# We save the model to disk
print("[INFO] serializing network...")
model.save(args["model"])
  
# We plot the training loss and accuracy
plt.style.use("ggplot")
plt.figure()
N = EPOCHS
plt.plot(np.arange(0, N), H.history["loss"], label ="train_loss")
plt.plot(np.arange(0, N), H.history["val_loss"], label ="val_loss")
plt.plot(np.arange(0, N), H.history["accuracy"], label ="train_acc")
plt.plot(np.arange(0, N), H.history["val_accuracy"], label ="val_acc")
plt.title("Training Loss and Accuracy on Santa / Not Santa")
plt.xlabel("Epoch #")
plt.ylabel("Loss / Accuracy")
plt.legend(loc ="lower left")
plt.savefig(args["plot"])

chevron_right


Training the CLassifier:
Open Command Prompt and Go to the path where your code is saved. Execute the below Command

python train.py --dataset images --model gs.model
Using TensorFlow backend.
[INFO] loading images...
[INFO] compiling model...
[INFO] training network...
Train for 21 steps, validate on 231 samples
Epoch 1/25
 1/21 [>.............................] - ETA: 11s - loss: 0.6757 - accuracy: 0.7368
21/21 [==============================] - 1s 43ms/step - loss: 0.7833 - accuracy: 0.4947 - val_loss: 0.5988 - val_accuracy: 0.5022
Epoch 2/25
21/21 [==============================] - 0s 21ms/step - loss: 0.5619 - accuracy: 0.6783 - val_loss: 0.4819 - val_accuracy: 0.7143
Epoch 3/25
21/21 [==============================] - 0s 21ms/step - loss: 0.4472 - accuracy: 0.8194 - val_loss: 0.4558 - val_accuracy: 0.7879
...
Epoch 20/25
21/21 [==============================] - 5s 259ms/step - loss: 0.2882 - accuracy: 0.8595 - val_loss: 0.3664 - val_accuracy: 0.8217
Epoch 21/25
21/21 [==============================] - 6s 272ms/step - loss: 0.2727 - accuracy: 0.8580 - val_loss: 0.3565 - val_accuracy: 0.8435
Epoch 22/25
21/21 [==============================] - 4s 176ms/step - loss: 0.2945 - accuracy: 0.8687 - val_loss: 0.3197 - val_accuracy: 0.8609
Epoch 23/25
21/21 [==============================] - 3s 152ms/step - loss: 0.3214 - accuracy: 0.8427 - val_loss: 0.3670 - val_accuracy: 0.8478
Epoch 24/25
21/21 [==============================] - 3s 160ms/step - loss: 0.2870 - accuracy: 0.8550 - val_loss: 0.3446 - val_accuracy: 0.8261
Epoch 25/25
21/21 [==============================] - 3s 165ms/step - loss: 0.2866 - accuracy: 0.8733 - val_loss: 0.3783 - val_accuracy: 0.8954
[INFO] serializing network...


We see that we got 89.54 accuracy for the Network trained at 25 epochs and low loss that follows the training loss, as is apparent from the image shown above

Part 3: Testing the Classifier using the Example Image Dataset
Here, we will test with some example images to check whether the network recognizes them correctly or not.

Code: Testing the classifier

filter_none

edit
close

play_arrow

link
brightness_4
code

# We import the necessary packages
from keras.preprocessing.image import img_to_array
from keras.models import load_model
import numpy as np
import argparse
import imutils
import cv2
  
# We construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-m", "--model", required = True,
    help ="path to trained model model")
ap.add_argument("-i", "--image", required = True,
    help ="path to input image")
args = vars(ap.parse_args())
  
# We load the image
image = cv2.imread(args["image"])
orig = image.copy()
  
# We pre-process the image for classification
image = cv2.resize(image, (28, 28))
image = image.astype("float") / 255.0
image = img_to_array(image)
image = np.expand_dims(image, axis = 0)
  
# We load the trained convolutional neural network
print("[INFO] loading network...")
model = load_model(args["model"])
  
# We classify the input image
(shiva, ganpati) = model.predict(image)[0]
  
# We build the label
labell ="Deity: "
label =  "Lord Ganpati" if ganpati > shiva else "Lord Shiva"
proba = ganpati if ganpati > shiva else shiva
new = labell + label
labels = "{} Confidence: {:.2f}%".format(new, proba * 100)
# We draw the label on the image
output = imutils.resize(orig, width = 700)
cv2.putText(output, labels, (10, 25),  cv2.FONT_HERSHEY_TRIPLEX,
    0.9, (0, 255, 255), 2)
  
# We show the output image
cv2.imshow("Output", output)
cv2.waitKey(0)

chevron_right


Testing the classifier:
Open Command Prompt and Go to the path where your code is saved. Give the folder path where example images are downloaded.
Example 1.
Execute the below Command

python test.py --model gs.model --image examples/ganpati_01.jpg
Using TensorFlow backend.
[INFO] loading network...


Example 2.
Execute the below Command

python test.py --model gs.model --image examples/ganpati_02.jpg
Using TensorFlow backend.
[INFO] loading network...


Example 3.
Execute the below Command

python test.py --model gs.model --image examples/shiva_01.jpg
Using TensorFlow backend.
[INFO] loading network...


Example 4.
Execute the below Command

python test.py --model gs.model --image examples/shiva_02.jpg
Using TensorFlow backend.
[INFO] loading network...

Refrences: