Sign Language Recognition System using TensorFlow in Python
The first step of any machine learning problem is finding the appropriate dataset. For Sign language recognition let’s use the Sign Language MNIST dataset. It has images of signs corresponding to each alphabet in the English language. Since the sign language of J and Z requires motion, those two classes are not available in the dataset.
Python libraries make it very easy for us to handle the data and perform typical and complex tasks with a single line of code.
- Pandas – This library helps to load the data frame in a 2D array format and has multiple functions to perform analysis tasks in one go.
- Numpy – Numpy arrays are very fast and can perform large computations in a very short time.
- Matplotlib – This library is used to draw visualizations.
- Tensorflow – This is an open-source library that is used for Machine Learning and Artificial intelligence and provides a range of functions to achieve complex functionalities with single lines of code.
The dataset is available as two CSV files, sign_mnist_train.csv and sign_mnist_test.csv. Each row in the CSV file is a training sample with the 0th index having the labels from 0-25 and the rest of the row containing the 784-pixel values of a 28 x 28 image. Each pixel value will be in the range [0, 255].
Data Loading and Preprocessing
The dataset has been provided in two files one is for training and the other one is for testing. We will load this data and then one hot encode the labels considering the fact we are not building the classifier for ‘J’ and ‘Z’ alphabet.
Now let’s check the shape of the training and the testing data.
(27455, 28, 28, 1) (27455, 24) (7172, 28, 28, 1) (7172, 24)
In this section, we will try to visualize images for signs of some of the alphabets which have been provided to us to build the classifier for each class.
From this step onward we will use the TensorFlow library to build our CNN model. Keras framework of the tensor flow library contains all the functionalities that one may need to define the architecture of a Convolutional Neural Network and train it on the data.
We will implement a Sequential model which will contain the following parts:
- Three Convolutional Layers followed by MaxPooling Layers.
- The Flatten layer to flatten the output of the convolutional layer.
- Then we will have two fully connected layers followed by the output of the flattened layer.
- We have included some BatchNormalization layers to enable stable and fast training and a Dropout layer before the final layer to avoid any possibility of overfitting.
- The final layer is the output layer which outputs soft probabilities for the 24 classes.
Let’s print the summary of the model’s architecture:
While compiling a model we provide these three essential parameters:
- optimizer – This is the method that helps to optimize the cost function by using gradient descent.
- loss – The loss function by which we monitor whether the model is improving with training or not.
- metrics – This helps to evaluate the model by predicting the training and the validation data.
Now we will train our model:
Let’s visualize the training and validation accuracy with each epoch.
By using just a simple CNN model we are able to achieve an accuracy of 82% which is really great. This shows that this technology is certainly going to help us build some amazing applications which can be proved a really great tool for people with some special needs.
Please Login to comment...