How to make a voice assistant for E-mail in Python?

As we know, emails are very important for communication as each professional communication can be done by emails and the best service for sending and receiving mails is as we all know GMAIL. Gmail is a free email service developed by Google. Users can access Gmail on the web and using third-party programs that synchronize email content through POP or IMAP protocols. 

To connect Gmail with Voice Assistant we have to use Gmail API. In this article, we will see how to send and check emails by giving voice commands. The Gmail API lets you view and manage Gmail mailbox data like threads, messages, and labels. More information about Gmail API can be found here- Gmail API.

Requirements:

• Python(3 or greater)
• A Gmail Account
• Google API Client
• pyttsx3 modulespeech_recognition module

Installation:

Install the libraries by running the following commands:

pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib

Run this or pyttsx3:

pip install pyttsx3

Run this for speech recognition:

pip install SpeechRecognition

Now there is a need to set up a Google Cloud console to interact with the Gmail API. This can be done very easily. You can find information on how to set up a Google cloud console.

After setting up the Google Cloud console, we just need to make an approach for building a voice assistant to speak up emails.

Approach:

• Import required modules.
• Create the below functions:
  • speak( ): This function will help our assistant to speak up.
  • get_audio( ): This function will help the assistant to get the input by the user.
  • check_mails( ): This function is used to check unread emails in the primary section of the inbox.
• Call the above-created function.

Below is the implementation in Python.

Python3

from __future__ import print_function
import datetime
import pickle
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
import os
import pyttsx3
import speech_recognition as sr
from datetime import date

# If modifying these scopes, delete the file token.pickle .
# if you run this for the first
# t time it will take you to gmail to choose your account
SCOPES = ["https://www.googleapis.com/auth/gmail.readonly"]


def speak(text):
    engine = pyttsx3.init()
    voices = engine.getProperty('voices')
    engine.setProperty('voice', voices[1].id)
    rate = engine.getProperty('rate')

    engine.setProperty('rate', rate-20)

    engine.say(text)
    engine.runAndWait()


speak("Welcome to mail service")


def get_audio():
    r = sr.Recognizer()
    with sr.Microphone() as source:
        r.pause_threshold = 1
        r.adjust_for_ambient_noise(source, duration=1)
        audio = r.listen(source)
        said = ""

    try:
        said = r.recognize_google(audio)
        print(said)

    except:
        speak("Didn't get that")

    return said.lower()


def authenticate_gmail():
    """Shows basic usage of the Gmail API.
    Lists the user's Gmail labels.
    """
    creds = None

    # The file token.pickle stores the user's
    # access and refresh tokens, and is
    # created automatically when the authorization
    # flow completes for the first
    # time.
    if os.path.exists('token.pickle'):
        with open('token.pickle', 'rb') as token:
            creds = pickle.load(token)

    # If there are no (valid) credentials available,
    # let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                'credentials.json', SCOPES)
            creds = flow.run_local_server(port=0)

        # Save the credentials for the next run
        with open('token.pickle', 'wb') as token:
            pickle.dump(creds, token)

    service = build('gmail', 'v1', credentials=creds)
    return service


def check_mails(service):

    # fetching emails of today's date
    today = (date.today())

    today_main = today.strftime('%Y/%m/%d')

    # Call the Gmail API
    results = service.users().messages().list(userId='me',
                                              labelIds=["INBOX", "UNREAD"],
                                              q="after:{0} and category:Primary".format(today_main)).execute()
    # The above code will get emails from primary
    # inbox which are unread
    messages = results.get('messages', [])

    if not messages:

        # if no new emails
        print('No messages found.')
        speak('No messages found.')
    else:
        m = ""

        # if email found
        speak("{} new emails found".format(len(messages)))

        speak("if you want to read any particular email just type read ")
        speak("and for not reading type leave ")
        for message in messages:

            msg = service.users().messages().get(userId='me',
                                                 id=message['id'], format='metadata').execute()

            for add in msg['payload']['headers']:
                if add['name'] == "From":

                    # fetching sender's email name
                    a = str(add['value'].split("<")[0])
                    print(a)

                    speak("email from"+a)
                    text = input()

                    if text == "read":

                        print(msg['snippet'])

                        # speak up the mail
                        speak(msg['snippet'])

                    else:
                        speak("email passed")


SERVICE2 = authenticate_gmail()
check_mails(SERVICE2)

Output:

Code Explanation:

1. The code begins by importing various Python programming modules.
2. These include the print_function and pickle module to store data in a pickled format, os.path to access path information, and the speech_recognition module to access speech recognition functionality.
3. Next, the code imports the GoogleAPIClient discovery library.
4. This library provides functions for building and managing Google APIs projects.
5. The build function is used to create a project structure for use with the GoogleAPIClient library.
6. The InstalledAppFlow class is used to authorize an application with Google credentials.
7. This class requires that you provide a set of app scopes (i.e., permissions).
8. The code first imports the InstalledAppFlow class, then uses it to create a new flow object named MyAppFlow .
9. Finally, the flow object is used to authorize an application with user credentials from the myuser account on behalf of developer@example.com .
10. The next section of code contains some basic configuration settings for using speech recognition in your applications.
11. First, you specify which language model should be used ( en-US ).
12. Then you specify which voice recognition services should be enabled ( SpeechRecognitionService ).
13. Finally, you set up some properties for controlling how speech
14. The code creates a file token.pickle which stores the access_token and refresh_token obtained from the Google Authentication flow.
15. The code also imports various Python modules, including pickle, os.path, speech_recognition, and datetime.
16. Next, the code creates a new instance of the InstalledAppFlow class.
17. This class is used to authorize an app with Google’s authentication system.
18. The flow object is then initialized with the necessary credentials (access_token and refresh_token).
19. Finally, the code calls the authorize() method on the InstalledAppFlow object to authorize the app with Google’s authentication system.
20. The code starts by importing the necessary libraries.
21. The library pyttsx3 provides the ability to speak text into a Google Home device.
22. The library sr provides access to the Recognizer object, which is used to identify and interpret spoken words.
23. Next, the code creates an instance of the pyttsx3 engine and sets some properties on it.
24. The engine has two voices assigned to it: one for normal speech and one for louder speech.
25. The rate property controls how fast these voices are played back.
26. The code then uses the engine to say a simple hello message.
27. You can try this out yourself by running the code and speaking “Hello, world!”
28. into your Google Home device.
29. Next, the code authenticates itself with Gmail using its credentials file (credentials.json in this case).
30. If everything goes well, you’ll see a message like this printed onscreen: {‘access_token’: ‘eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9’, ‘expires_in’: 3600, ‘
31. The code first imports the necessary libraries and sets up a few variables.
32. Next, it defines two functions: one to speak a welcome message and another to get the user’s current audio status.
33. The main function is check_mails().
34. This function will be used to check for new emails and will also authenticate the user if needed.
35. Next, the code creates a service object and sets some properties on it.
36. The properties that are set on this object include the Gmail account that the code wants to access (in this case, “gmail”), the scopes that should be used (in this case, “https://www.googleapis.com/auth/gmail.readonly”), and the voice ID of the first voice in the
37. The code first fetches all the emails for the user me today.
38. It uses the list method of the service object to do this.
39. The list method takes two arguments: first, a user ID; second, a list of label ids (e.g., “INBOX” for inboxes, “UNREAD” for unread messages).
40. The code then formats the date string using the strftime function and passes it as an argument to the after function.
41. This function looks at today’s date and returns a string that is formatted according to the format “%Y/%m/%d”.
42. Finally, the code calls the users() method of the service object to get all of my email addresses.
43. It uses my user ID (me) as well as my label ids (e.g., “INBOX” for inboxes, “UNREAD” for unread messages) to find my email addresses.
44. Then it uses q=”after:{0} and category:Primary”.format(today_main)) to filter out only those emails that are after today and in primary categories.
45. The code fetches all the emails in the user’s inbox and unread messages, as well as sorting them by date.
46. The q parameter is used to filter the results so that only those after today and with a Primary category are returned.
47. Finally, the list() method is called on the Gmail API object to return a list of email addresses and corresponding message ids.
48. The code first checks to see if there are any new emails.
49. If not, the code prints No messages found.
50. If there are new emails, the code gets the email addresses from the results object and creates a list of strings called messages.
51. The list has one string for each unread message in the inbox.
52. Next, the code checks to see if any of these messages is already spoken for.
53. If so, it just repeats that message.
54. Otherwise, it creates a new empty string called m and starts speaking those new emails.
55. The code will get emails from the primary inbox which are unread.
56. If no new emails are found, then the code will speak “No messages found.”
57. If an email is found, then the code will speak “{} new emails found”.
58. The format of this string will be based on how many emails were found.
59. The code starts by speaking the following: If you want to read any particular email, just type “read” and for not reading, type “leave”.
60. Then the code loops through all of the messages in the users database.
61. For each message, it gets the user’s ID (me in this example), retrieves the metadata for that message (including the sender’s email address), and prints it out.
62. The next line of code is important because it determines whether or not a given email should be read.
63. If text equals “read”, then the program will print out that email’s snippet.
64. Otherwise, if text does not equal “read”, then the program will leave that email alone.
65. The code will fetch the email address of the sender from the “From” header in the message.
66. Next, it will print out the email address.
67. If the user wants to read any particular email, they can simply type “read” and it will fetch that email for them.
68. If they do not want to read any emails, they can simply type “leave”.