OpenCV – Facial Landmarks and Face Detection using dlib and OpenCV

The world will be taken over by Artificial Intelligence very soon. Face Applications include identification of faces from videos or digital images. It is a good example of Computer Vision. For Identification with better accuracy and confidential value, the faces need to be detected properly. Face Detection Technology is used in applications to detect faces from digital images and videos. Also, just detecting the face will not help. We need more information about the face, i.e. whether a person smiles, laughs, or dimples seen while smiling etc. In short, facial expressions too give us information. In order to get more information about the face, we take the help of Facial Landmarks.

What are Facial Landmarks?
Facial landmarks are used for localizing and representing salient regions or facial parts of the person’s face, such as:

Facial landmarks is a technique which can be applied to applications like face alignment, head pose estimation, face swapping, blink detection, drowsiness detection, etc.

In this context of facial landmarks, our vital aim is to detect facial structures on the person’s face using a method called shape prediction.
Facial Landmarks Detection has 2 steps:

  1. To detect the key facial structures on the person’s face.
  2. It involves localizing the face in the image.

We can do Face detection in a number of ways. We can use OpenCV’s built-in Haar Cascade XML files or even TensorFlow or using Keras. Over here especially, We need to apply a HOG (Histogram of Gradients) and Linear SVM (Support Vector Machines) object detector specifically for the task of face detection. We can also do it using Deep Learning-based algorithms which are built for face localization. Also, The algorithm will be used for the detection of the faces in the image. We can obtain face bounding box through some method for which we use the (x, y) coordinates of the face in the image respectively.
We do have a variety of facial landmark detectors, but every method will essentially be trying to localize and also labelling the following facial regions will be done.



The facial landmark detector which is pre-trained inside the dlib library of python for detecting landmarks, is used to estimate the location of 68 points or (x, y) coordinates which map to the facial structures. These indexes of 68 coordinates or points can be easily visualized on the image below:


The Locations of the Facial Parts are as follows:

Following are the steps for Implementation of Face Landmarks Detection:

Code: Implementation of Facial Detection with Facial Landmarks using Python

filter_none

edit
close

play_arrow

link
brightness_4
code

# We import the necessary packages
from imutils import face_utils
import numpy as np
import argparse
import imutils
import dlib
import cv2
  
# We construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-p", "--shape-predictor", required = True,
    help ="path to facial landmark predictor")
ap.add_argument("-i", "--image", required = True,
    help ="path to input image")
args = vars(ap.parse_args())
  
# We are initializing the  dlib's face detector (HOG-based) and then 
# creation of the facial landmark predictor
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(args["shape_predictor"])
  
# We then load the input image, resize it, and convert it to grayscale
images = cv2.imread(args["image"])
images = imutils.resize(images, width = 500)
gray = cv2.cvtColor(images, cv2.COLOR_BGR2GRAY)
  
# We then detect faces in the grayscale image
rects = detector(gray, 1)
  
# Now, job is to loop over the face detections
for (i, rect) in enumerate(rects):
    # We will determine the facial landmarks for the face region, then
    # can convert the facial landmark (x, y)-coordinates to a NumPy array
    shape = predictor(gray, rect)
    shape = face_utils.shape_to_np(shape)
  
    # We then convert dlib's rectangle to a OpenCV-style bounding box
    # [i.e., (x, y, w, h)], then can draw the face bounding box
    (x, y, w, h) = face_utils.rect_to_bb(rect)
    cv2.rectangle(images, (x, y), (x + w, y + h), (255, 255, 0), 2)
  
    # We then show the face number 
    cv2.putText(images, 'Face % {}'.format(i + 1), (x - 10, y - 10),
        cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 255), 2)
  
    # We then loop over the (x, y)-coordinates for the facial landmarks 
    # and draw them on the image
    for (x, y) in shape:
        cv2.circle(images, (x, y), 1, (0, 0, 255), -1)
  
# Now show the output image with the face detections as well as 
# facial landmarks
cv2.imshow("Output", images)
cv2.waitKey(0)
chevron_right

Steps for the Output Generation:


As seen in the Output, the Landmarks are shown in red color dots and the Face Detection is in Cyan color box drawn around the face.
(Note:- The above steps for execution works for Windows and Linux.)

Code: Implementation of Facial Landmarks with Real Time using Python

filter_none

edit
close

play_arrow

link
brightness_4
code

# We Import the necessary packages needed
import cv2
import numpy as np
import dlib
  
cap = cv2.VideoCapture(0)
# We initialise detector of dlib
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
  
# Start the main program
while True:
    _, frame = cap.read()
# We actually Convert to grayscale conversion
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = detector(gray)
    for face in faces:
   # The face landmarks code begins from here
        x1 = face.left()
        y1 = face.top()
        x2 = face.right()
        y2 = face.bottom()
        # Then we can also do cv2.rectangle function (frame, (x1, y1), (x2, y2), (0, 255, 0), 3)
        landmarks = predictor(gray, face)
       # We are then accesing the landmark points 
        for n in range(0, 68):
            x = landmarks.part(n).x
            y = landmarks.part(n).y
            cv2.circle(frame, (x, y), 2, (255, 255, 0), -1)
    cv2.imshow("Frame", frame)
    key = cv2.waitKey(1)
    if key == 27:
        break # press esc the frame is destroyed
chevron_right

Steps for Output Generation:

Article Tags :