Build Fuel Price Tracker Using Python

In this modern-day lifestyle, fuel has become a necessity for all human beings. It is a base for our life-style. So, we are going to write a script to track their price using Python.

Modules Needed

  • bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come built-in with Python. To install this type the below command in the terminal.
pip install bs4
  • requests: Request allows you to send HTTP/1.1 requests extremely easily. This module also does not come built-in with Python. To install this type the below command in the terminal.
pip install requests

Let’s see the stepwise execution of the script

Step 1: Import all dependence

Python3



filter_none

edit
close

play_arrow

link
brightness_4
code

# import module
import pandas as pd
import requests
from bs4 import BeautifulSoup

chevron_right


Step 2: Create a URL get function

Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

# user define function
# Scrape the data
def getdata(url):
    r = requests.get(url)
    return r.text

chevron_right


Step 3: Now pass the URL into the getdata() function and Convert that data into HTML code

Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

# link for extract html data
soup = BeautifulSoup(htmldata, 'html.parser')
result = soup.find_all("div", class_="gold_silver_table")
print(result)

chevron_right


Output :

[<div class=”gold_silver_table”>
<table border=”0″ cellpadding=”1″ cellspacing=”1″ width=”100%”>
<tr class=”first”>
<td class=”heading” width=”200″>City</td>
<td class=”heading” width=”200″>Today Price</td>
<td class=”heading” width=”200″>Yesterday’s Price</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-new-delhi.html” title=”New Delhi”>New Delhi</a></td>
<td> ₹ 82.08</td>
<td> ₹ 82.03</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-kolkata.html” title=”Kolkata”>Kolkata</a></td>
<td> ₹ 83.57</td>
<td> ₹ 83.52</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-mumbai.html” title=”Mumbai”>Mumbai</a></td>
<td> ₹ 88.73</td>
<td> ₹ 88.68</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-chennai.html” title=”Chennai”>Chennai</a></td>
<td> ₹ 85.04</td>
<td> ₹ 85.00</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-gurgaon.html” title=”Gurgaon”>Gurgaon</a></td>
<td> ₹ 79.92</td>
<td> ₹ 79.84</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-noida.html” title=”Noida”>Noida</a></td>
<td> ₹ 82.23</td>
<td> ₹ 82.30</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-bangalore.html” title=”Bangalore”>Bangalore</a></td>
<td> ₹ 84.75</td>
<td> ₹ 84.70</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-bhubaneswar.html” title=”Bhubaneswar”>Bhubaneswar</a></td>
<td> ₹ 82.47</td>
<td> ₹ 82.59</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-chandigarh.html” title=”Chandigarh”>Chandigarh</a></td>
<td> ₹ 78.96</td>
<td> ₹ 78.92</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-hyderabad.html” title=”Hyderabad”>Hyderabad</a></td>
<td> ₹ 85.30</td>
<td> ₹ 85.25</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-jaipur.html” title=”Jaipur”>Jaipur</a></td>
<td> ₹ 90.08</td>
<td> ₹ 89.24</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-lucknow.html” title=”Lucknow”>Lucknow</a></td>
<td> ₹ 82.20</td>
<td> ₹ 82.09</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-patna.html” title=”Patna”>Patna</a></td>
<td> ₹ 84.73</td>
<td> ₹ 84.88</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-trivandrum.html” title=”Trivandrum”>Trivandrum</a></td>
<td> ₹ 83.91</td>
<td> ₹ 84.03</td>
</tr>
</table>
</div>]

Note: These scripts will give you only Raw data in String format you have to print your data with your needs. 

Step 4: Now, Search your needed data into sting with soup.find_all().



Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

# Declare string var
# Declare list
mydatastr = ''
result = []
  
# searching all tr in the html data
# storing as a string
for table in soup.find_all('tr'):
    mydatastr += table.get_text()
  
# set accourding to your required
mydatastr = mydatastr[1:]
itemlist = mydatastr.split("\n\n")
  
for item in itemlist[:-5]:
    result.append(item.split("\n"))
  
result

chevron_right


Output :

Step 4: Make a DataFrame for displaying your result.

Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

# Calling DataFrame constructor on list
df = pd.DataFrame(result[:-8])
df

chevron_right


Complete code:

Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

# import module
import requests
import pandas as pd
from bs4 import BeautifulSoup
  
# link for extract html data
  
  
def getdata(url):
    r = requests.get(url)
    return r.text
  
  
soup = BeautifulSoup(htmldata, 'html.parser')
  
# Declare string var
# Declare list
mydatastr = ''
result = []
  
# searching all tr in the html data
# storing as a string
for table in soup.find_all('tr'):
    mydatastr += table.get_text()
  
# set accourding to your required
mydatastr = mydatastr[1:]
itemlist = mydatastr.split("\n\n")
  
for item in itemlist[:-5]:
    result.append(item.split("\n"))
  
# Calling DataFrame constructor on list
df = pd.DataFrame(result[:-8])
df

chevron_right


Output :


Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.

To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course.




My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.