Skip to content
Related Articles

Related Articles

Improve Article
Build Fuel Price Tracker Using Python
  • Last Updated : 29 Dec, 2020

In this modern-day lifestyle, fuel has become a necessity for all human beings. It is a base for our life-style. So, we are going to write a script to track their price using Python.

Modules Needed

  • bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come built-in with Python. To install this type the below command in the terminal.
pip install bs4
  • requests: Request allows you to send HTTP/1.1 requests extremely easily. This module also does not come built-in with Python. To install this type the below command in the terminal.
pip install requests

Let’s see the stepwise execution of the script

Step 1: Import all dependence

Python3






# import module
import pandas as pd
import requests
from bs4 import BeautifulSoup

Step 2: Create a URL get function

Python3




# user define function
# Scrape the data
def getdata(url):
    r = requests.get(url)
    return r.text

Step 3: Now pass the URL into the getdata() function and Convert that data into HTML code

Python3




# link for extract html data
soup = BeautifulSoup(htmldata, 'html.parser')
result = soup.find_all("div", class_="gold_silver_table")
print(result)

Output :

[<div class=”gold_silver_table”>
<table border=”0″ cellpadding=”1″ cellspacing=”1″ width=”100%”>
<tr class=”first”>
<td class=”heading” width=”200″>City</td>
<td class=”heading” width=”200″>Today Price</td>
<td class=”heading” width=”200″>Yesterday’s Price</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-new-delhi.html” title=”New Delhi”>New Delhi</a></td>
<td> ₹ 82.08</td>
<td> ₹ 82.03</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-kolkata.html” title=”Kolkata”>Kolkata</a></td>
<td> ₹ 83.57</td>
<td> ₹ 83.52</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-mumbai.html” title=”Mumbai”>Mumbai</a></td>
<td> ₹ 88.73</td>
<td> ₹ 88.68</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-chennai.html” title=”Chennai”>Chennai</a></td>
<td> ₹ 85.04</td>
<td> ₹ 85.00</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-gurgaon.html” title=”Gurgaon”>Gurgaon</a></td>
<td> ₹ 79.92</td>
<td> ₹ 79.84</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-noida.html” title=”Noida”>Noida</a></td>
<td> ₹ 82.23</td>
<td> ₹ 82.30</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-bangalore.html” title=”Bangalore”>Bangalore</a></td>
<td> ₹ 84.75</td>
<td> ₹ 84.70</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-bhubaneswar.html” title=”Bhubaneswar”>Bhubaneswar</a></td>
<td> ₹ 82.47</td>
<td> ₹ 82.59</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-chandigarh.html” title=”Chandigarh”>Chandigarh</a></td>
<td> ₹ 78.96</td>
<td> ₹ 78.92</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-hyderabad.html” title=”Hyderabad”>Hyderabad</a></td>
<td> ₹ 85.30</td>
<td> ₹ 85.25</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-jaipur.html” title=”Jaipur”>Jaipur</a></td>
<td> ₹ 90.08</td>
<td> ₹ 89.24</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-lucknow.html” title=”Lucknow”>Lucknow</a></td>
<td> ₹ 82.20</td>
<td> ₹ 82.09</td>
</tr>
<tr class=”even_row”>
<td><a href=”/petrol-price-in-patna.html” title=”Patna”>Patna</a></td>
<td> ₹ 84.73</td>
<td> ₹ 84.88</td>
</tr>
<tr class=”odd_row”>
<td><a href=”/petrol-price-in-trivandrum.html” title=”Trivandrum”>Trivandrum</a></td>
<td> ₹ 83.91</td>
<td> ₹ 84.03</td>
</tr>
</table>
</div>]

Note: These scripts will give you only Raw data in String format you have to print your data with your needs. 

Step 4: Now, Search your needed data into sting with soup.find_all().



Python3




# Declare string var
# Declare list
mydatastr = ''
result = []
  
# searching all tr in the html data
# storing as a string
for table in soup.find_all('tr'):
    mydatastr += table.get_text()
  
# set accourding to your required
mydatastr = mydatastr[1:]
itemlist = mydatastr.split("\n\n")
  
for item in itemlist[:-5]:
    result.append(item.split("\n"))
  
result

Output :

Step 4: Make a DataFrame for displaying your result.

Python3




# Calling DataFrame constructor on list
df = pd.DataFrame(result[:-8])
df

Complete code:

Python3




# import module
import requests
import pandas as pd
from bs4 import BeautifulSoup
  
# link for extract html data
  
  
def getdata(url):
    r = requests.get(url)
    return r.text
  
  
soup = BeautifulSoup(htmldata, 'html.parser')
  
# Declare string var
# Declare list
mydatastr = ''
result = []
  
# searching all tr in the html data
# storing as a string
for table in soup.find_all('tr'):
    mydatastr += table.get_text()
  
# set accourding to your required
mydatastr = mydatastr[1:]
itemlist = mydatastr.split("\n\n")
  
for item in itemlist[:-5]:
    result.append(item.split("\n"))
  
# Calling DataFrame constructor on list
df = pd.DataFrame(result[:-8])
df

Output :

 Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.  

To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course. And to begin with your Machine Learning Journey, join the Machine Learning – Basic Level Course

My Personal Notes arrow_drop_up
Recommended Articles
Page :