Train a Support Vector Machine to recognize facial features in C++

Let’s see how to train a model of support vector machine, save the trained model and test the model to check the percentage of its prediction accuracy using the OpenCV.

Data Organization:

Using imagenetscraper and autocrop , we collect data from the web, crop faces and resize them to smaller sizes in bulk. The collected data needs to be organized meaningfully so we can access it programmatically and manually. Use the below folder structure-

FFR_dataset/
|-- Age
|   |-- adult
|   |-- child
|   |-- old
|   |-- teen
|-- Emotion
|   |-- anger
|   |-- contempt
|   |-- happy
|   |-- neutral
|   |-- sad
|   |-- surprise
|-- Gender
    |-- female
    |-- male

We use the same directory names in the code to access them to train, save and predict the recognition results. A minimum of 50 images in each folder is required to train the models to get good prediction results. Training more images can improve the results but not recommended as it takes a lot of time to execute that and does not give significant improvements.

Implementation

By making use of the sample provided in the official opencv repo to train the SVM with HOG, train_HOG.cpp, we implement the c++ code to train, save and predict the facial features on an image with multiple faces.

There are three feature types- Age, Emotion and Gender. Four age groups, six emotions and two gender types. Hence an n-class classifier is implemented to recognize each feature on a face data.

Step #1: For each feature type i.e. (Age, Emotion or Gender) loop through ‘n’ run times.

filter_none

edit
close

play_arrow

link
brightness_4
code

// CTrainTestHOG::Run(int run_times)
for (auto ft : m_FeatureList) {
    DEBUGLW("\tFeature type=[%s]\n", ft.first.c_str());
  
    std::vector<float> predictionAccuracyList;
    predictionAccuracyList.reserve(run_times);
  
    for (int run = 0; run < run_times; ++run) {
        DEBUGLW("\t\tRun=[%d]\n", run);
        vector<Mat> trainData, predData;
        vector<int> trainLabels, predLabels;
  
        this->get_ft_dataset(ft.first, trainData, predData,
                                  trainLabels, predLabels);
        // ... train, predict and measure the SVM model.
    }
}

chevron_right


Step #2: In each run, iterate through the feature values in the feature type and get the images into a vector or array, i.e. get all the images from folders Gender->Male and Gender->Female.

filter_none

edit
close

play_arrow

link
brightness_4
code

// CTrainTestHOG::get_ft_dataset()
std::set<cv::String>& featureValueList = m_FeatureList.find(ft)->second;
  
for (auto fv : featureValueList) {
    DEBUGLW("\t\t\tFeature value=[%s]\n", fv.c_str());
    std::vector<cv::Mat> _trainData;
    std::vector<cv::Mat> _predData;
    std::vector<int> _trainLabels;
    std::vector<int> _predLabels;
  
    errCode = this->get_ftfv_dataset(ft, fv, _trainData, _predData,
                                       _trainLabels, _predLabels);
    if (errCode != EXIT_SUCCESS)
        break;
    trainData.insert(trainData.end(), _trainData.begin(), _trainData.end());
    predData.insert(predData.end(), _predData.begin(), _predData.end());
    trainLabels.insert(trainLabels.end(), _trainLabels.begin(), _trainLabels.end());
    predLabels.insert(predLabels.end(), _predLabels.begin(), _predLabels.end());
}

chevron_right


Step #3 to #6:

  • Crop the images in the vector to the face rectangles and update the images vector with the new faces list.
  • Perform any pre-processing tasks such as resizing to smaller size (64, 64) on each image in faces list.
  • Shuffle the pre-processed face images in vector randomly to introduce random input data.
  • Split the dataset into training(80%) and prediction(20%) data.
filter_none

edit
close

play_arrow

link
brightness_4
code

//  CTrainTestHOG::get_ftfv_dataset()
std::vector<cv::Mat> imgList;
this->get_images(folderName, imgList);
this->get_cropped_faces(imgList);
this->get_preprocessed_faces(imgList);
  
//-- return on empty img list to prevent seg fault
if (imgList.empty()) {
    errCode = EXIT_FAILURE;
    DEBUGLE("Error img list is empty!\n");
    break;
}
DEBUGLD("\t\t\timgList.size()=[%ld]\n", imgList.size());
  
std::random_shuffle(imgList.begin(), imgList.end());
  
// 80% for training
int trainPart = imgList.size() * 0.8;
  
// 20% for predicting
int predPart = imgList.size() - trainPart;
DEBUGLD("\t\t\ttrainPart=[%d], predPart=[%d]\n", trainPart, predPart);
  
trainData.reserve(trainPart);
predData.reserve(predPart);
  
ft_t::iterator ft_iter = m_FeatureList.find(ft);
fv_t::iterator fv_iter = ft_iter->second.find(fv);
int label = std::distance(ft_iter->second.begin(),  fv_iter);
DEBUGLD("\t\t\tlabel=[%d]\n", label);
  
int i = 0;
for (; i < trainPart; ++i) {
    trainData.push_back(imgList.at(i));
    trainLabels.push_back(label);
}
DEBUGLD("\t\t\ti=[%d], trainData.size()=[%ld], 
        trainLabels.size()
        = [% ld]\n ", i, trainData.size(), 
                       trainLabels.size());
  
for (; i < imgList.size(); ++i) {
    predData.push_back(imgList.at(i));
    predLabels.push_back(label);
}
DEBUGLD("\t\t\ti=[%d], predData.size()=[%ld], 
        predLabels.size()
        = [% ld]\n ", i, predData.size(), predLabels.size());

chevron_right


Step #7 : Compute HOG for each image in training data.

filter_none

edit
close

play_arrow

link
brightness_4
code

// CTrainTestHOG::computeHOGs()
HOGDescriptor hog;
vector<Mat> hogMats;
vector<float> descriptors;
for (auto img : imgHogList) {
    hog.winSize = img.size() / 8 * 8;
    hog.compute(img, descriptors);
    cv::Mat descriptors_mat(Mat(descriptors).clone());
    hogMats.push_back(descriptors_mat);
}
imgHogList.swap(hogMats);

chevron_right


Step #8 : Convert the training data vector to opencv Mat object to train SVM.

filter_none

edit
close

play_arrow

link
brightness_4
code

//  CTrainTestHOG::convert_to_ml()
for (size_t i = 0; i < train_samples.size(); ++i) {
    CV_Assert(train_samples[i].cols == 1 || train_samples[i].rows == 1);
    if (train_samples[i].cols == 1) {
        cv::transpose(train_samples[i], tmp);
        tmp.copyTo(trainData.row((int)i));
    }
    else if (train_samples[i].rows == 1) {
        train_samples[i].copyTo(trainData.row((int)i));
    }
}

chevron_right


Step #9 : Pass the training data Mat object to svm train function along with a vector of labels for the training data.

filter_none

edit
close

play_arrow

link
brightness_4
code

//  CTrainTestHOG::Run()
trainLabels.resize(ml_train_data.rows);
// train svm
DEBUGLW("\t\tTraining SVM - begin\n");
m_pSVM->train(ml_train_data, ROW_SAMPLE, trainLabels);
DEBUGLW("\t\tTraining SVM - end\n");

chevron_right


Step #10: Save the trained model.

filter_none

edit
close

play_arrow

link
brightness_4
code

//-- step 10, CTrainTestHOG::Run()
cv::String svmModelFileName = cv::format("%s/cv4_svm_%s_model.xml",
                                         getenv(FFR_DATASET_PATH),
                                         ft.first.c_str());
  
m_pSVM->save(svmModelFileName.c_str());
DEBUGLW("\t\tSaved SVM model=[%s]\n",
        svmModelFileName.c_str());

chevron_right


Step #11: Predict the model by computing the HOG for each prediction image, convert the prediction dataset to opencv mat object and call svm predict with a vector of labels to store the result.

filter_none

edit
close

play_arrow

link
brightness_4
code

//-- step 11, CTrainTestHOG::Run()
// test the model
// compute HOG for each pre-processed face
errCode = this->computeHOGs(predData);
if (errCode != EXIT_SUCCESS) {
    DEBUGLE("Error in computing HOGs for the feature "
            "type=[%s]\n",
            ft.first.c_str());
    break;
}
  
// convert HOG feature vectors to SVM data
Mat ml_pred_data;
vector<int> resultLabels;
errCode = this->convert_to_ml(predData, ml_pred_data);
if (errCode != EXIT_SUCCESS) {
    DEBUGLE("Error in converting to ml for the "
            "feature type=[%s]\n",
            ft.first.c_str());
    break;
}
predLabels.resize(ml_pred_data.rows);
// resultLabels.resize(ml_pred_data.rows);
// test svm
DEBUGLW("\t\tTesting SVM - begin\n");
Mat responses_mat;
m_pSVM->predict(ml_pred_data, responses_mat);
for (size_t i = 0; i < ml_pred_data.rows; ++i) {
    resultLabels.push_back(responses_mat.at<int>(i));
}
DEBUGLW("\t\tTesting SVM - end\n");

chevron_right


Step #12 and #13: Calculate the percentage of its accuracy by comparing the expected prediction labels with the predicted labels.

filter_none

edit
close

play_arrow

link
brightness_4
code

//  CTrainTestHOG::Run()
// check the accuracy
float accuracy = 0.0f;
this->get_prediction_accuracy(predLabels, resultLabels, accuracy);
DEBUGLW("\t\tPrediction accuracy=[%lf]\n", accuracy);
predictionAccuracyList.push_back(accuracy);
  
//-- step 13, CTrainTestHOG::Run()
// check the mean accuracy of 'n' runs
float sum_of_accuracies = std::accumulate(
    predictionAccuracyList.begin(),
    predictionAccuracyList.end(), 0.0);
float mean_accuracy = sum_of_accuracies / predictionAccuracyList.size();
DEBUGLW("\t\tMean prediction accuracy=[%lf]\n",
        mean_accuracy);

chevron_right


Run the executable with below command line arguments.

./train_hog --test --in= --out= --show

Input:
Harry Potter sample input image

Output:
Harry Potter result output image

Results log for HOG SVM using OpenCV 2.4
Results log for HOG SVM using OpenCV 4.0

Note: Due to long hair for all three people in the image it is detecting the gender as ‘female’ which is a false positive. In machine learning algorithms the false positives are always common given the input sample image has ambiguous features.



My Personal Notes arrow_drop_up

A versatile coder, pragmatic thinker and a humorous writer

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.