Skip to content
Related Articles

Related Articles

Build Fuel Price Tracker Using Python

View Discussion
Improve Article
Save Article
Like Article
  • Last Updated : 31 Mar, 2022

In this modern-day lifestyle, fuel has become a necessity for all human beings. It is a base for our life-style. So, we are going to write a script to track their price using Python.

Modules Needed

  • bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come built-in with Python. To install this type the below command in the terminal.
pip install bs4
  • requests: Request allows you to send HTTP/1.1 requests extremely easily. This module also does not come built-in with Python. To install this type the below command in the terminal.
pip install requests

Let’s see the stepwise execution of the script

Step 1: Import all dependence

Python3




# import module
import pandas as pd
import requests
from bs4 import BeautifulSoup

Step 2: Create a URL get function

Python3




# user define function
# Scrape the data
def getdata(url):
    r = requests.get(url)
    return r.text

Step 3: Now pass the URL into the getdata() function and Convert that data into HTML code

Python3




# link for extract html data
soup = BeautifulSoup(htmldata, 'html.parser')
result = soup.find_all("div", class_="gold_silver_table")
print(result)

Output :

[<div class=”gold_silver_table”> <table border=”0″ cellpadding=”1″ cellspacing=”1″ width=”100%”> <tr class=”first”> <td class=”heading” width=”200″>City</td> <td class=”heading” width=”200″>Today Price</td> <td class=”heading” width=”200″>Yesterday’s Price</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-new-delhi.html” title=”New Delhi”>New Delhi</a></td> <td> ₹ 82.08</td> <td> ₹ 82.03</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-kolkata.html” title=”Kolkata”>Kolkata</a></td> <td> ₹ 83.57</td> <td> ₹ 83.52</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-mumbai.html” title=”Mumbai”>Mumbai</a></td> <td> ₹ 88.73</td> <td> ₹ 88.68</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-chennai.html” title=”Chennai”>Chennai</a></td> <td> ₹ 85.04</td> <td> ₹ 85.00</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-gurgaon.html” title=”Gurgaon”>Gurgaon</a></td> <td> ₹ 79.92</td> <td> ₹ 79.84</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-noida.html” title=”Noida”>Noida</a></td> <td> ₹ 82.23</td> <td> ₹ 82.30</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-bangalore.html” title=”Bangalore”>Bangalore</a></td> <td> ₹ 84.75</td> <td> ₹ 84.70</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-bhubaneswar.html” title=”Bhubaneswar”>Bhubaneswar</a></td> <td> ₹ 82.47</td> <td> ₹ 82.59</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-chandigarh.html” title=”Chandigarh”>Chandigarh</a></td> <td> ₹ 78.96</td> <td> ₹ 78.92</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-hyderabad.html” title=”Hyderabad”>Hyderabad</a></td> <td> ₹ 85.30</td> <td> ₹ 85.25</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-jaipur.html” title=”Jaipur”>Jaipur</a></td> <td> ₹ 90.08</td> <td> ₹ 89.24</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-lucknow.html” title=”Lucknow”>Lucknow</a></td> <td> ₹ 82.20</td> <td> ₹ 82.09</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-patna.html” title=”Patna”>Patna</a></td> <td> ₹ 84.73</td> <td> ₹ 84.88</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-trivandrum.html” title=”Trivandrum”>Trivandrum</a></td> <td> ₹ 83.91</td> <td> ₹ 84.03</td> </tr> </table> </div>]

Note: These scripts will give you only Raw data in String format you have to print your data with your needs. 

Step 4: Now, Search your needed data into sting with soup.find_all().

Python3




# Declare string var
# Declare list
mydatastr = ''
result = []
 
# searching all tr in the html data
# storing as a string
for table in soup.find_all('tr'):
    mydatastr += table.get_text()
 
# set according to your required
mydatastr = mydatastr[1:]
itemlist = mydatastr.split("\n\n")
 
for item in itemlist[:-5]:
    result.append(item.split("\n"))
 
result

Output :

Step 4: Make a DataFrame for displaying your result.

Python3




# Calling DataFrame constructor on list
df = pd.DataFrame(result[:-8])
df

Complete code:

Python3




# import module
import requests
import pandas as pd
from bs4 import BeautifulSoup
 
# link for extract html data
 
 
def getdata(url):
    r = requests.get(url)
    return r.text
 
 
soup = BeautifulSoup(htmldata, 'html.parser')
 
# Declare string var
# Declare list
mydatastr = ''
result = []
 
# searching all tr in the html data
# storing as a string
for table in soup.find_all('tr'):
    mydatastr += table.get_text()
 
# set according to your required
mydatastr = mydatastr[1:]
itemlist = mydatastr.split("\n\n")
 
for item in itemlist[:-5]:
    result.append(item.split("\n"))
 
# Calling DataFrame constructor on list
df = pd.DataFrame(result[:-8])
df

Output :


My Personal Notes arrow_drop_up
Recommended Articles
Page :

Start Your Coding Journey Now!