In this post, we will discuss how to get information about a GeeksforGeeks user using web scraping and serve the information as an API using Python’s micro-framework, Flask.
Step #1: Visit the auth profile
To scrape a website, the first step is to visit the website.
Step #2: Inspect the Page Source
In the above image, you can spot the
descDiv div which contains user Data. In the following image, spot the four
mdl-grid divs for four pieces of information.
Dive deep, spot the two blocks for attribute and its corresponding value. That’s your user-profile data.
Find all this in the following function which returns all the data in the form of a dictionary.
Step #3: Articles and Improvements list
This time, try to spot the various tags yourself.
If you could spot the various elements of the HTML, you can easily write the code for scraping it too.
If you couldn’t, here’s the code for your help.
Step #4: Setup Flask
The code for web scraping is completed. Now it’s time to set-up our Flask Server. Here’s the setup for the Flask app, along with all the necessary libraries needed for the entire script.
Step #5: Setting Up for API
Now that we’ve got the appropriate functions, our only task is to combine both of their results, convert the dictionary to JSON and then serve it on the server.
Here’s the code for an endpoint which serves the API on the basis of user handle it receives. Remember, we need to take care of the improper user handle, our endpoint can receive any time.
Combining all the code, you’ve a got a fully functioning server serving dynamic APIs.
- Using JWT for user authentication in Flask
- Retrieving HTML From data using Flask
- Dockerize your Flask App
- Subdomain in Flask | Python
- Differences Between Django vs Flask
- Python | Using for loop in Flask
- Python | Introduction to Web development using Flask
- Sending Emails Using API in Flask-Mail
- Python | Build a REST API using Flask
- Deploy Python Flask App on Heroku
- Deploying a TensorFlow 2.1 CNN model on the web with Flask
- Create a Weather app using Flask | Python
- Python | 404 Error handling in Flask
- Flask - (Creating first simple application)
- Todo list app using Flask | Python
- Deploy Machine Learning Model using Flask
- ES6 | Environment Setup
- Environment setup for JSP
- p5.js | setup() Function
- Environment Setup for Matplotlib
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to email@example.com. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.