Python OpenCV – Drowsiness Detection

The number of fatalities on the road due to drowsiness is very high. Thus python allows the model of deep learning algorithm via including the use of OpenCV. Drowsiness Detection is the detection of a person to check whether the person is feeling sleepy while performing a significant task. This detection has many applications in medical and safety fields. This detection can be quickly done using “shape predictor face landmarks” that mark the essential landmarks on the face.

Thus python also provides the flexibility for detecting such serious and significant detection via OpenCV modules. One can easily make their own phone camera for detecting drowsiness with the below-provided code in python.

Steps Required:

1. Installing and importing all the modules needed for the code.

1. pip install numpy    
2. pip install opencv-python
3. pip install time
4. pip install pyttsx3
5. pip install dlib
6. pip install scipy 
7. Click and Download shape_predictor_68_face_landmarks1

2. Access the camera and mark the landmarks from the (.dat) file to predict the location of the ear and eyes.

3. Using the Euclidean distance function to calculate the distance between landmarks on the opposite sides of the eyes. This function requires two points in a plane to calculate the distance.

Euclidean distance formula:

4. Mark the eye points in a face so that it will be really easy for the user to get the detection.

5. Calculate the aspect ratio for the left and right eyes and set the criteria for the closing of eyes (drowsiness detection).

Import Modules and libraries:

import cv2
import dlib
import pyttsx3
from scipy.spatial import distance

Face landmark mapping points:

Here, the image below shows all the masking and landmarks numerically-wise of all the critical points in a normal face with the reference of the right and left eye accordingly.

Needed points in a face mask:

With reference to the above image, we are focusing on the right and the left eye and getting all the critical points connecting the eye.

Fig 1.2 All the critical points in an eye

Use of Important functions in a program:

pyttsx3.init(): This function is used to initialize the text to the audio conversion of modules and libraries used inside for the alerting purpose in the below code.
VideoCapture(): The function is used to declare the parts of the camera used for the OpenCV and to turn on the camera.
cap.read(): Store all the camera parameters inside the function that can be used for different purposes, such as deciding the width height Channel and even used for setting the color of the camera output.
cv2.imshow(): To display the camera output that is prior set by the user. It is the visual display by the camera on the screen.
dlib.shape_predictor(“location of the .dat file”): The face landmark features will be stored inside the “.dat file”.
destroyAllWindow(): Kill all the opened cv2 windows used inside the program.
cap.release(): It will release all the stored parameters inside the cap function.
cv2.waitkey(“time limit set by the user”): This will keep open the cv2 interface for the given declared time by the user.
Detect_Eye(): Function calculating the aspect ratio for the eye by using the Euclidean distance function.

Stepwise Implementation

Step 1: Import the required libraries and their functions. And accessing the camera of a device for detecting the face. Initializing the pyttsx3 for audio message alert.

Python3

import cv2

import dlib

import pyttsx3

from scipy.spatial import distance
 
# INITIALIZING THE pyttsx3 SO THAT 
# ALERT AUDIO MESSAGE CAN BE DELIVERED

engine = pyttsx3.init()
 
# SETTING UP OF CAMERA TO 1 YOU
# CAN EVEN CHOOSE 0 IN PLACE OF 1

cap = cv2.VideoCapture(1)
 
# MAIN LOOP IT WILL RUN ALL THE UNLESS 
# AND UNTIL THE PROGRAM IS BEING KILLED 
# BY THE USER

while True:

    null, frame = cap.read()

    gray_scale = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
 
    cv2.imshow("Drowsiness DETECTOR IN OPENCV2", frame)

    key = cv2.waitKey(9)

    if key == 20:

        break
cap.release()
cv2.destroyAllWindows()

Output:

Access only the camera in the OpenCV.

Fig 1.3 Opening camera in an OpenCV

Step 2: Face detection and calculating the aspect ratio as declared above in the Euclidean distance formula.

Python3

import cv2

import dlib

import pyttsx3

from scipy.spatial import distance
 
# INITIALIZING THE pyttsx3 SO THAT
# ALERT AUDIO MESSAGE CAN BE DELIVERED

engine = pyttsx3.init()
 
# SETTING UP OF CAMERA TO 1 YOU CAN 
# EVEN CHOOSE 0 IN PLACE OF 1

cap = cv2.VideoCapture(1)
 
# FACE DETECTION OR MAPPING THE FACE 
# TO GET THE Eye AND EYES DETECTED

face_detector = dlib.get_frontal_face_detector()
 
# PUT THE LOCATION OF .DAT FILE (FILE 
# FOR PREDECTING THE LANDMARKS ON FACE )

dlib_facelandmark = dlib.shape_predictor(

    "C:\\Users\Acer\\Desktop\\geeks\\article 9\\drowsinessDetector-master\\shape_predictor_68_face_landmarks1.dat")
 
# FUNCTION CALCULATING THE ASPECT RATIO 
# FOR THE Eye BY USING EUCLIDEAN DISTANCE 
# FUNCTION

def Detect_Eye(eye):

    poi_A = distance.euclidean(eye[1], eye[5])

    poi_B = distance.euclidean(eye[2], eye[4])

    poi_C = distance.euclidean(eye[0], eye[3])

    aspect_ratio_Eye = (poi_A+poi_B)/(2*poi_C)

    return aspect_ratio_Eye
 
# MAIN LOOP IT WILL RUN ALL THE UNLESS AND
# UNTIL THE PROGRAM IS BEING KILLED BY THE 
# USER

while True:

    null, frame = cap.read()

    gray_scale = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
 
    faces = face_detector(gray_scale)
 
    for face in faces:

        face_landmarks = dlib_facelandmark(gray_scale, face)

        leftEye = [] 

        rightEye = [] 
 
        # THESE ARE THE POINTS ALLOCATION FOR THE

        # LEFT EYES IN .DAT FILE THAT ARE FROM 42 TO 47

        for n in range(42, 48):

            x = face_landmarks.part(n).x

            y = face_landmarks.part(n).y

            rightEye.append((x, y))

            next_point = n+1

            if n == 47:

                next_point = 42

            x2 = face_landmarks.part(next_point).x

            y2 = face_landmarks.part(next_point).y

            cv2.line(frame, (x, y), (x2, y2), (0, 255, 0), 1)
 
        # THESE ARE THE POINTS ALLOCATION FOR THE 

        # RIGHT EYES IN .DAT FILE THAT ARE FROM 36 TO 41

        for n in range(36, 42):

            x = face_landmarks.part(n).x

            y = face_landmarks.part(n).y

            leftEye.append((x, y))

            next_point = n+1

            if n == 41:

                next_point = 36

            x2 = face_landmarks.part(next_point).x

            y2 = face_landmarks.part(next_point).y

            cv2.line(frame, (x, y), (x2, y2), (255, 255, 0), 1)
 
    cv2.imshow("Drowsiness DETECTOR IN OPENCV2", frame)

    key = cv2.waitKey(9)

    if key == 20:

        break
cap.release()
cv2.destroyAllWindows()

Output:

Fig 1.4 Eyes detection through OpenCV .dat file

Note: Download the above “shape_predictor_68_face_landmarks1.dat” file in the same folder where your whole code is present. And copy the location of the file and paste the location as described in the code.

Step 3: Main loop for reading and storing the camera parameters and allocating the mask on the right and left eye. Describe the audio and text message for the user to alert them of drowsiness.

Python3

# MAIN LOOP IT WILL RUN ALL THE UNLESS AND
# UNTIL THE PROGRAM IS BEING KILLED BY 
# THE USER

while True:

    null, frame = cap.read()

    gray_scale = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
 
    faces = face_detector(gray_scale)
 
    for face in faces:

        face_landmarks = dlib_facelandmark(gray_scale, face)

        leftEye = [] 

        rightEye = [] 
 
        # THESE ARE THE POINTS ALLOCATION FOR

        # THE LEFT EYE IN .DAT FILE THAT ARE 

        # FROM 42 TO 47

        for n in range(42, 48):

            x = face_landmarks.part(n).x

            y = face_landmarks.part(n).y

            rightEye.append((x, y))

            next_point = n+1

            if n == 47:

                next_point = 42

            x2 = face_landmarks.part(next_point).x

            y2 = face_landmarks.part(next_point).y

            # LINE DRAW IN LEFT EYE

            cv2.line(frame, (x, y), (x2, y2), (0, 255, 0), 1)
 
        # THESE ARE THE POINTS ALLOCATION FOR THE 

        # RIGHT EYE IN .DAT FILE THAT ARE FROM 36 TO 41

        for n in range(36, 42):

            x = face_landmarks.part(n).x

            y = face_landmarks.part(n).y

            leftEye.append((x, y))

            next_point = n+1

            if n == 41:

                next_point = 36

            x2 = face_landmarks.part(next_point).x

            y2 = face_landmarks.part(next_point).y

            # LINE DRAW IN RIGHT EYE

            cv2.line(frame, (x, y), (x2, y2), (255, 255, 0), 1)
 
        # CALCULATING THE ASPECT RATIO FOR LEFT

        # AND RIGHT EYE

        right_Eye = Detect_Eye(rightEye)

        left_Eye = Detect_Eye(leftEye)

        Eye_Rat = (left_Eye+right_Eye)/2
 
        # NOW ROUND OF THE VALUE OF AVERAGE MEAN 

        # OF RIGHT AND LEFT EYES

        Eye_Rat = round(Eye_Rat, 2)
 
        # THIS VALUE OF 0.25 (YOU CAN EVEN CHANGE IT)

        # WILL DECIDE WHETHER THE PERSONS'S EYES ARE

        # CLOSE OR NOT

        if Eye_Rat < 0.25:

            cv2.putText(frame, "DROWSINESS DETECTED", (50, 100),

                        cv2.FONT_HERSHEY_PLAIN, 2, (21, 56, 210), 3)

            cv2.putText(frame, "Alert!!!! WAKE UP DUDE", (50, 450),

                        cv2.FONT_HERSHEY_PLAIN, 2, (21, 56, 212), 3)
 
            # CALLING THE AUDIO FUNCTION OF TEXT TO AUDIO

            # FOR ALERTING THE PERSON

            engine.say("Alert!!!! WAKE UP DUDE")

            engine.runAndWait()

    cv2.imshow("Drowsiness DETECTOR IN OPENCV2", frame)

    key = cv2.waitKey(9)

    if key == 20:

        break
cap.release() 
cv2.destroyAllWindows()

Full Code:

Python3

import cv2

import dlib

import pyttsx3

from scipy.spatial import distance
 
# INITIALIZING THE pyttsx3 SO THAT 
# ALERT AUDIO MESSAGE CAN BE DELIVERED

engine = pyttsx3.init()
 
# SETTING UP OF CAMERA TO 1 YOU CAN
# EVEN CHOOSE 0 IN PLACE OF 1

cap = cv2.VideoCapture(1)
 
# FACE DETECTION OR MAPPING THE FACE TO
# GET THE Eye AND EYES DETECTED

face_detector = dlib.get_frontal_face_detector()
 
# PUT THE LOCATION OF .DAT FILE (FILE FOR
# PREDECTING THE LANDMARKS ON FACE )

dlib_facelandmark = dlib.shape_predictor(

    "C:\\Users\Acer\\Desktop\\geeks\\article 9\\drowsine\

    ssDetector-master\\shape_predictor_68_face_landmarks1.dat")
 
# FUNCTION CALCULATING THE ASPECT RATIO FOR
# THE Eye BY USING EUCLIDEAN DISTANCE FUNCTION

def Detect_Eye(eye):

    poi_A = distance.euclidean(eye[1], eye[5])

    poi_B = distance.euclidean(eye[2], eye[4])

    poi_C = distance.euclidean(eye[0], eye[3])

    aspect_ratio_Eye = (poi_A+poi_B)/(2*poi_C)

    return aspect_ratio_Eye
 
# MAIN LOOP IT WILL RUN ALL THE UNLESS AND 
# UNTIL THE PROGRAM IS BEING KILLED BY THE USER

while True:

    null, frame = cap.read()

    gray_scale = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
 
    faces = face_detector(gray_scale)
 
    for face in faces:

        face_landmarks = dlib_facelandmark(gray_scale, face)

        leftEye = [] 

        rightEye = [] 
 
        # THESE ARE THE POINTS ALLOCATION FOR THE 

        # LEFT EYES IN .DAT FILE THAT ARE FROM 42 TO 47

        for n in range(42, 48):

            x = face_landmarks.part(n).x

            y = face_landmarks.part(n).y

            rightEye.append((x, y))

            next_point = n+1

            if n == 47:

                next_point = 42

            x2 = face_landmarks.part(next_point).x

            y2 = face_landmarks.part(next_point).y

            cv2.line(frame, (x, y), (x2, y2), (0, 255, 0), 1)
 
        # THESE ARE THE POINTS ALLOCATION FOR THE 

        # RIGHT EYES IN .DAT FILE THAT ARE FROM 36 TO 41

        for n in range(36, 42):

            x = face_landmarks.part(n).x

            y = face_landmarks.part(n).y

            leftEye.append((x, y))

            next_point = n+1

            if n == 41:

                next_point = 36

            x2 = face_landmarks.part(next_point).x

            y2 = face_landmarks.part(next_point).y

            cv2.line(frame, (x, y), (x2, y2), (255, 255, 0), 1)
 
        # CALCULATING THE ASPECT RATIO FOR LEFT 

        # AND RIGHT EYE

        right_Eye = Detect_Eye(rightEye)

        left_Eye = Detect_Eye(leftEye)

        Eye_Rat = (left_Eye+right_Eye)/2
 
        # NOW ROUND OF THE VALUE OF AVERAGE MEAN 

        # OF RIGHT AND LEFT EYES

        Eye_Rat = round(Eye_Rat, 2)
 
        # THIS VALUE OF 0.25 (YOU CAN EVEN CHANGE IT) 

        # WILL DECIDE WHETHER THE PERSONS'S EYES ARE CLOSE OR NOT

        if Eye_Rat < 0.25:

            cv2.putText(frame, "DROWSINESS DETECTED", (50, 100),

                        cv2.FONT_HERSHEY_PLAIN, 2, (21, 56, 210), 3)

            cv2.putText(frame, "Alert!!!! WAKE UP DUDE", (50, 450),

                        cv2.FONT_HERSHEY_PLAIN, 2, (21, 56, 212), 3)
 
            # CALLING THE AUDIO FUNCTION OF TEXT TO 

            # AUDIO FOR ALERTING THE PERSON

            engine.say("Alert!!!! WAKE UP DUDE")

            engine.runAndWait()
 
    cv2.imshow("Drowsiness DETECTOR IN OPENCV2", frame)

    key = cv2.waitKey(9)

    if key == 20:

        break
cap.release()
cv2.destroyAllWindows()

Final Output:

Fig 1.5 Drowsiness Detection

Article Tags :

Python

OpenCV

Python-OpenCV