Approach:
- Fetch the entire HTML page and store it as string using request package.
- Load HTML into cheerio and find the CSS selectors to extract repositories details.
Using request package:
request package: The request is designed to be the simplest way possible to make http calls. It supports HTTPS and follows redirects by default.
Installing request package:
$ npm install request
Note: Run this command in current folder, where ‘app.js’ file located.
Syntax for request:
request(url, (error, response, body) => { if(!error && response.statusCode == 200) { statements to be executed. } }
Parameters:
- url: API endpoint to which request is made.
- response: HTTP response status codes indicate whether a specific HTTP request has been successfully completed.
- body: Response data.
Using cheerio package:
cheerio package: Fast, flexible & lean implementation of core jQuery designed specifically for the server.
Installing cheerio package:
$ npm install cheerio
Note: Run this command in current folder, where ‘app.js’ file located.
Syntax for cheerio:
const cheerio = require('cheerio') const $ = cheerio.load(HTMLString) $(CSS Selector).text('Hello there!') $.html()
// Import request package const request = require( 'request' );
// Import cheerio package const cheerio = require( 'cheerio' );
// Github Trending Page URL // Get request to the URL request.get(url, (error, response, body) => { // If the response code is 200 and
// there is no error
if (!error && response.statusCode == 200) {
// Load HTML string into cheerio
const $ = cheerio.load(body);
// Below are the CSS selectors to
// fetch the data required
let temp = $( '.Box-row' )
let repos = $( '.h3.lh-condensed a' );
let data = [];
for (let i = 0; i < repos.length; i++) {
let reponame = $(temp[i])
.find( '.h3.lh-condensed a' )
.text().replace(
/[\n\r]+|[\s]{2, }/g, ' ' ).trim();
let repolanguage = $(temp[i])
.find(
'.f6.text-gray.mt-2 span span[itemprop="programmingLanguage"]' )
.text()
.replace(/[\n\r]+|[\s]{2, }/g, ' ' ).trim();
let repostars = $(temp[i])
.find(
'.f6.text-gray.mt-2 .muted-link.d-inline-block.mr-3' )
.text()
.replace(/[\n\r]+|[\s]{2, }/g, ' ' ).trim();
// Push the fetched data into an object
data.push({
'Repository' : reponame,
'Language' : repolanguage,
'Stars/Forks' : repostars
})
}
// Display the Object created using console.table
console.table(data)
}
else {
console.log( "Unable to fetch data from github" );
}
}); |
Output: