Scraping weather data using Python to get umbrella reminder on email

In this article, we are going to see how to scrape weather data using Python and get reminders on email. If the weather condition is rainy or cloudy this program will send you an “umbrella reminder” to your email reminding you to pack an umbrella before leaving the house.  We will scrape the weather information from Google using bs4 and requests libraries in python. 

Then based on the weather conditions we will send the email using smtplib library. To run this program every day at a specified time we will use the schedule library in python.

Modules Needed

• bs4: Beautiful Soup (bs4) is a Python library for extracting data from HTML and XML files. To install this library, type the following command in IDE/terminal.

pip install bs4

• requests: This library allows you to send HTTP/1.1 requests very easily. To install this library, type the following command in IDE/terminal.

pip install requests

• smtplib: smtplib is a Python module that defines an SMTP client session object, which can be used to send mail to any machine on the Internet. To install this library, type the following command in the IDE/terminal.

pip install smtplib

• schedule: A schedule is a Python library used to schedule tasks at specific times of the day or specific days of the week. To install this library, type the following command in the IDE/terminal.

pip install schedule

Stepwise Implementation

Step 1: Import schedule, smtplib, requests, and bs4 libraries.

Python3

import schedule
import smtplib
import requests
from bs4 import BeautifulSoup

Step 2: Create a URL with the specified city name and create a requests instance bypassing this URL.

Python3

city = "Hyderabad"
url = "https://www.google.com/search?q=" + "weather" + city
html = requests.get(url).content

Step 3: Pass the retrieved HTML document into Beautiful Soup which will return a string stripped of  HTML tags and metadata.  Using the find function we will retrieve all the necessary data.

Python3

soup = BeautifulSoup(html,
                     'html.parser')
temperature = soup.find(
  'div', attrs={'class': 'BNeawe iBp4i AP7Wnd'}).text
time_sky = soup.find(
  'div', attrs={'class': 'BNeawe tAd8D AP7Wnd'}).text

# formatting data
sky = time_sky.split('\n')[1]

Step 4: If the weather condition turns out to be rainy or cloudy, this program will send an email with an “umbrella reminder” message to the recipient. To encapsulate an SMTP connection I have created an SMTP object with the parameters `smtp.gmail.com` and 587. After creating the SMTP object, you can log in with your email address and password.

Note: Different websites use different port numbers. In this article, we are using a Gmail account to send emails. The port number used here is “587”. If you want to send mail using a website other than Gmail, you need to obtain the corresponding information.

Python3

if sky == "Rainy" or sky == "Rain And Snow" or sky == "Showers" or sky == "Haze" or sky == "Cloudy":
    smtp_object = smtplib.SMTP('smtp.gmail.com', 587)

Step 5: For security reasons, the SMTP connection is now placed in TLS mode. Transport Layer Security encrypts all SMTP commands. After that, for authentication purposes, you need to pass your Gmail account credentials in the login instance. If you enter an invalid email ID or password, the compiler will display an authentication error.  

Save the message you need to send to a variable, like msg. Use the sendmail() instance to send your message. sendmail() uses three parameters: sender_email_id, receiver_email_id, and message_to_be_sent. The order of these three parameters must be the same. This will send an email from your account. After completing the task, use quit () to end the SMTP session.

Python3

# start TLS for security
smtp_object.starttls()
# Authentication
smtp_object.login("YOUR EMAIL", "PASSWORD")
subject = "Umbrella Reminder"
body = f"Take an umbrella before leaving the house.\
Weather condition for today is ", {
    sky}, " and temperature is {temperature} in {city}."
msg = f"Subject:{subject}\n\n{body}\n\nRegards,\
\nGeeksforGeeks".encode('utf-8')

# sending the mail
smtp_object.sendmail("FROM EMAIL ADDRESS",
                     "TO EMAIL ADDRESS", msg)
# terminating the session
smtp_object.quit()
print("Email Sent!")

Step 6:  Using the schedule library, Schedule the task every day at a specific time. schedule.run_pending() Checks whether a scheduled task is pending to run or not.

Python3

# Every day at 06:00AM time umbrellaReminder() is called.
schedule.every().day.at("06:00").do(umbrellaReminder)

while True:
    schedule.run_pending()

Note: When you execute this program it will throw you a smtplib.SMTPAuthenticationError and also sends you a Critical Security alert to your email because, In a nutshell, Google is not allowing you to log in via smtplib because it has flagged this sort of login as “less secure”, so what you have to do is go to this link while you’re logged in to your google account, and allow the access:

Below is the full implementation:

Python3

import schedule
import smtplib
import requests
from bs4 import BeautifulSoup


def umbrellaReminder():
    city = "Hyderabad"

    # creating url and requests instance
    url = "https://www.google.com/search?q=" + "weather" + city
    html = requests.get(url).content

    # getting raw data
    soup = BeautifulSoup(html, 'html.parser')
    temperature = soup.find('div',
                            attrs={'class': 'BNeawe iBp4i AP7Wnd'}).text
    time_sky = soup.find('div',
                         attrs={'class': 'BNeawe tAd8D AP7Wnd'}).text

    # formatting data
    sky = time_sky.split('\n')[1]

    if sky == "Rainy" or sky == "Rain And Snow" or sky == "Showers" or sky == "Haze" or sky == "Cloudy":
        smtp_object = smtplib.SMTP('smtp.gmail.com', 587)

        # start TLS for security
        smtp_object.starttls()

        # Authentication
        smtp_object.login("YOUR EMAIL", "PASSWORD")
        subject = "GeeksforGeeks Umbrella Reminder"
        body = f"Take an umbrella before leaving the house.\
        Weather condition for today is {sky} and temperature is\
        {temperature} in {city}."
        msg = f"Subject:{subject}\n\n{body}\n\nRegards,\nGeeksforGeeks".encode(
            'utf-8')

        # sending the mail
        smtp_object.sendmail("FROM EMAIL",
                             "TO EMAIL", msg)

        # terminating the session
        smtp_object.quit()
        print("Email Sent!")


# Every day at 06:00AM time umbrellaReminder() is called.
schedule.every().day.at("06:00").do(umbrellaReminder)

while True:
    schedule.run_pending()

Output:

Sample Umbrella Reminder Email