Run Python Script using PythonShell from Node.js

Nowadays Node.js is the most attractive technology in the field of backend development for developers around the globe. And if someone wishes to use something like Web Scraping using python modules or run some python scripts having some machine learning algorithms, then one need to know how to integrate these two. 

We are going to get some data of a user through web scraping of leetcode . So, let’s get started.

Now, setup the Node.js server code first.

Javascript

filter_none

edit
close

play_arrow

link
brightness_4
code

//Import express.js module and create its variable.
const express=require('express');
const app=express();
  
//Import PythonShell module.
const {PythonShell} =require('python-shell');
  
//Router to handle the incoming request.
app.get("/", (req, res, next)=>{
    //Here are the option object in which arguments can be passed for the python_test.js.
    let options = {
        mode: 'text',
        pythonOptions: ['-u'], // get print results in real-time
          scriptPath: 'path/to/my/scripts', //If you are having python_test.py script in same folder, then it's optional.
        args: ['shubhamk314'] //An argument which can be accessed in the script using sys.argv[1]
    };
      
  
    PythonShell.run('python_test.py', options, function (err, result){
          if (err) throw err;
          // result is an array consisting of messages collected 
          //during execution of script.
          console.log('result: ', result.toString());
          res.send(result.toString())
    });
});
  
//Creates the server on default port 8000 and can be accessed through localhost:8000
const port=8000;
app.listen(port, ()=>console.log(`Server connected to ${port}`));

chevron_right


Now, Python Script. (Make sure you have installed the bs4 module and csv module.)

Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

# code
import sys
import requests
from bs4 import BeautifulSoup 
from csv import writer
# bs4 module for web scraping and requests for making HTTPS requests using Python.
response = requests.get('https://leetcode.com / shubhamk314')
soup = BeautifulSoup(response.text, 'html.parser')
main_content = soup.select(
    '# base_content>div>div>div.col-sm-5.col-md-4>div:nth-child(3)>ul')
list_items = main_content[0].select('li')
items = ['Solved Question', 'Accepted Submission', 'Acceptance Rate']
n = 0
  
# It will create csv files named progress.csv in root folder once this is script is called.
with open('progress.csv', 'w') as csv_file:
  csv_writer = writer(csv_file)
  headers = ['Name', 'Score']
  csv_writer.writerow(headers)
  while(n < 3):
    name = items[n]
    score = list_items[n].find('span').get_text().strip()
    csv_writer.writerow([name, score])
    n = n + 1
print("csv file created for leetcode")

chevron_right


After saving both the files, run the following command from its root folder : 



node test.js

Now, send request through localhost:8000 in the browser.

If everything goes right, then the outout will be :

Message: csv file created for leetcode.

Conclusion

This is the simple implementation of a how-to run python script with Node.js which can be useful in situations where you have a stack of Node.js application and you want to run a simple python script. If you want to know moe about PythonShell module, then go through the given link.

Link :  https://github.com/extrabacon/python-shell




My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.


Article Tags :

Be the First to upvote.


Please write to us at contribute@geeksforgeeks.org to report any issue with the above content.