Open In App

Python subprocess module

Improve
Improve
Like Article
Like
Save
Share
Report

The subprocess module present in Python(both 2.x and 3.x) is used to run new applications or programs through Python code by creating new processes. It also helps to obtain the input/output/error pipes as well as the exit codes of various commands. In this tutorial, we’ll delve into how to effectively use subprocessing modules to execute external functions from your Python code, pass data to them, and get results Whether you’re a beginner or an experienced Python user developer, this guide will prove useful with your skills and abilities.

An Introduction to Subprocess in Python

The subprocess module in Python is a built-in module that allows us to create new child processes. We can get exit codes, output, and error messages from these child processes. It is a valuable tool for executing external functions or commands in your Python code. Think of it as a way to programmatically interact with your computer using Python instead of manually entering commands in the terminal. This module simplifies the implementation process and seamlessly integrates external programming into your Python workflow.

For example, we can use a subprocessing module to initiate shell commands such as “ls” or “ping” and retrieve their output directly from your Python script. It also makes it easy to execute other Python scripts or executable files, such as .exe files on Windows. Also, the subprocess module gives you control over the input and output of the executed process. This means that you can accurately monitor the data sent to the application and capture the data it returns.

Uses of Subprocess Module

  • Run system commands or external programs from your Python script.
  • They can handle input, output, and errors caused by child programs in your Python code
  • Capture the output (stdout) and error messages (stderr) generated by these commands.
  • Communicate with the running process by sending input (stdin).
  • Wait for the command to complete and obtain its return code to check for success or failure.

What is subprocess Popen in Python?

`subprocess.Popen` is a low-level interface for running subprocesses, while subprocess.run is a high-level wrapper around Popen intended for ease of use. Popen allows you to start a new process and interact with its standard input, output, and error streams. It returns a handle to the running procedure that can be used to wait for the procedure to complete, check its return code, or abort it. run is a more flexible function that allows you to run a command and capture its results in a single call, allowing you to specify options for a command without having to create a Popen object and manage the streams yourself, such as whether to raise or exception if the command fails.

In general, you should use run if you just want to run the command and capture its output and Popen if you need more control over the process, such as interacting with its input and output streams. The Popen class takes the same arguments as run(), including args specifying which commands to run and other optional arguments such as stdin, stdout, stderr, shell, cwd, and env

Python Subprocess check_output

In the subprocess module, check_output is a function that closely resembles run(). It has the specific behavior of returning only the standard output of the command. However, it differs in how it handles command execution outcomes: if the command’s return code is non-zero, it triggers a CalledProcessError exception. The check_output function shares the same set of arguments as the run() function. This includes the ‘args’ parameter, which defines the command to be executed, and it also accepts additional optional arguments like ‘stdin’, ‘stderr’, ‘shell’, ‘cwd’, and ‘env’.

Python3




import subprocess
try:
    ans = subprocess.check_output(["python", "--version"], text=True)
    print(ans)
  
except subprocess.CalledProcessError as e:
    print(f"Command failed with return code {e.returncode}")


Output

Python 3.7.17

Python Subprocess Pipe

The Python subprocess module empowers the creation and interaction with child processes, which enables the execution of external programs or commands. A standout feature of this module is the capacity to establish pipes, facilitating communication between the parent and child processes. To create such pipes, you can employ the subprocess module in combination with the Popen class. By specifying the ‘stdout’ or ‘stdin’ argument as ‘subprocess.PIPE’, you establish this interprocess communication channel. For instance, the code snippet below illustrates a pipe that connects the output of the ‘ls’ command to the input of the ‘grep’ command, which filters the output to display only the lines containing the word “file”.

Python3




import subprocess
  
lsProcess = subprocess.Popen(["ls"], stdout=subprocess.PIPE, text=True)
grepProcess = subprocess.Popen(
    ["grep", "sample"], stdin=ls_process.stdout,
    stdout=subprocess.PIPE, text=True)
output, error = grepProcess.communicate()
  
print(output)
print(error)


sample_file

Python Subprocess Call

subprocess.call() is a Python function within the subprocess module. It is employed to execute a command in a separate process and wait for its completion. The function returns a return code, which is typically zero for success and non-zero for failure. This function shares its arguments with subprocess.run(), such as ‘args’ for specifying the command to be executed, along with optional parameters like ‘stdin’, ‘stdout’, ‘stderr’, ‘shell’, ‘cwd’, and ‘env’.

By default, the command’s standard output and error are directed to the same output streams as the parent process unless you explicitly redirect them using the ‘stdout’ and ‘stderr’ arguments.

Python3




import subprocess
ans = subprocess.call(["python", "--version"])
if ans == 0:
    print("Command executed.")
else:
    print("Command failed.", return_code)


Output

Python 3.7.17
Command executed.

Compile and Run Different Languages in Python using Subprocess

To execute different programs using Python two functions of the subprocess module are used:

1.subprocess.check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False)
Parameters:
args=The command to be executed.Several commands can be passed as a string by separated by “;”.
stdin=Value of standard input stream to be passed as (os.pipe()).
stdout=Value of output obtained from standard output stream.
stderr=Value of error obtained(if any) from standard error stream.
shell=Boolean parameter.If True the commands get executed through a new shell environment.
Return Value:
The function returns the return code of the command.If the return code is zero, the function simply returns(command executed successfully) otherwise CalledProcessError is being raised.

2.subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)
Parameters:
args=The command to be executed. Several commands can be passed as a string by separated by “;”.
stdin=Value of standard input stream to be passed as pipe(os.pipe()).
stdout=Value of output obtained from standard output stream.
stderr=Value of error obtained(if any) from standard error stream.
shell=boolean parameter.If True the commands get executed through a new shell environment.
universal_newlines=Boolean parameter.If true files containing stdout and stderr are opened in universal newline mode.
Return Value:
The function returns the return code of the command.If the return code is zero, the function simply returns the output as a byte string(command executed successfully) otherwise CalledProcessError is being raised.

Example: Let us consider the following code example in different languages.

C++




#include <iostream> 
using namespace std; 
int main() 
    int a, b; 
    cin >> a >> b; 
    cout << "Hello World from C++.Values are:" << a << " " << b; 
    return 0; 


C




#include<stdio.h> 
int main() 
    printf("Hello World from C"); 
    return 0; 


Java




class HelloWorld { 
    public static void main(String args[]) 
    
        System.out.print("Hello World from Java."); 
    


Output

Hello World from C++.Values are:0 4196176

Python Code to Execute the Above Programs

In this code there are three functions created separate each for executing C,C++ and JAVA program respectively. ‘executeC()’ is used to execute the C program file.‘executeCpp()’ is used to execute the C++ program file.‘executeJava()’ is used to execute the JAVA program file.

In each method It employs the ‘subprocess.check_call’ method to execute the necessary compilation and execution commands. After execution, it prints the return code, which can indicate whether the program ran successfully.

Python3




# Python 3 program to demonstrate subprocess
import subprocess
import os
  
def excuteC():
    # store the return code of the c program(return 0)
    # and display the output
    s = subprocess.check_call("gcc HelloWorld.c -o out1;./out1", shell=True)
    print(", return code", s)
  
def executeCpp():
    # create a pipe to a child process
    data, temp = os.pipe()
    # write to STDIN as a byte object(convert string
    # to bytes with encoding utf8)
    os.write(temp, bytes("5 10\n", "utf-8"))
    os.close(temp)
  
    # store output of the program as a byte string in s
    s = subprocess.check_output(
        "g++ HelloWorld.cpp -o out2;./out2", stdin=data, shell=True)
  
    # decode s to a normal string
    print(s.decode("utf-8"))
  
def executeJava():
    # store the output of
    # the java program
    s = subprocess.check_output(
        "javac HelloWorld.java;java HelloWorld", shell=True)
    print(s.decode("utf-8"))
  
  
# Driver function
if __name__ == "__main__":
    excuteC()
    executeCpp()
    executeJava()


Output

Hello World from C, return code 0
Hello World from C++. Values are:5 10
Hello World from Java.



Last Updated : 01 Nov, 2023
Like Article
Save Article
Previous
Next
Share your thoughts in the comments
Similar Reads