Create Cricket Score API using Web Scrapping in Flask
Cricket is one of the famous outdoor sport played worldwide. There are very few APIs providing live scoreboards and none of them are free to use. Using any of the scoreboards available we can create API for ourselves. This method not only works for Cricket Scoreboard but also for any information available online. Following is the flow in which this blog would guide to create an API and deploy it.
- Setting up the App Directory
- Web Scrape data from NDTV Sports.
- Beautiful Soup in Python would be used.
- Create an API.
- Flask would be used.
- Heroku would be used for deployment,
Setting up the App Directory
Step 1: Create a Folder (eg. CricGFG).
Step 2: Set up the virtual environment. Here we create an environment .env
python -m venv .env
Step 3: Activate the environment.
Getting the Data
Step 1: In Python, we have Beautiful Soup which is a library to pull out data from HTML files. To install Beautiful Soup, run a simple command;
pip install beautifulsoup4
Similarly, install the Requests module of Python.
pip install requests
We would use the NDTV Sports Cricket Scorecard to fetch the data.
Step 3: Following are the steps for Scraping data from the Web Page. To get the HTML text from the web page;
html_text = requests.get(‘https://sports.ndtv.com/cricket/live-scores’).text
To represent the parsed object as a whole we use the BeautifulSoup object,
soup = BeautifulSoup(html_text, "html.parser")
Note: It is recommended to run and check the code after each step to know about the difference and thoroughly understand the concepts.
We will further find all the required divs and other tags with their respective classes.
The next section of the code has our data that is our result. If for any of the reasons that code is not present in the HTML file, it would lead to an error, so including that part in a try and except block.
Live score England vs India 3rd Test,Pataudi Trophy, 2021
England lead by 223 runs
Day 2 | Post Tea Session
Creating the API
We will use Flask which is a micro web framework written in Python.
pip install Flask
Following is the starter code for our flask application.
Open localhost on your browser:
We would now add our code of Web Scrapping into this and some helper methods provided by Flask to properly return JSON data.
Now it’s time to merge all our codes. Let’s Start!
Output in the Browser:
Here we have created our own Cricket API.
Deploying API on Heroku
Step 1: You need to create an account on Heroku.
Step 2: Install Git on your machine.
Step 3: Install Heroku on your machine.
Step 4: Login to your Heroku Account
Step 5: Install gunicorn which is a pure-Python HTTP server for WSGI applications. It allows you to run any Python application concurrently by running multiple Python processes.
pip install gunicorn
Step 6: We need to create a procfile which is a text file in the root directory of our application, to explicitly declare what command should be executed to start our app.
web: gunicorn CricGFG:app
Step 7: We further create a requirements.txt file that includes all the necessary modules which Heroku needs to run our flask application.
pip freeze >> requirements.txt
Step 8: Create an app on Heroku, click here.
Step 9: We now initialize a git repository and add our files to it.
git init git add . git commit -m "Cricket API Completed"
Step 10: We will now direct Heroku towards our git repository.
heroku git:remote -a cricgfg
Step 11: We will now push our files on Heroku.
git push heroku master
Finally, our API is now available on https://cricgfg.herokuapp.com/
Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.
To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course. And to begin with your Machine Learning Journey, join the Machine Learning – Basic Level Course