Skip to content
Related Articles

Related Articles

Shell Scripting – Standard Input, Output and Error

View Discussion
Improve Article
Save Article
Like Article
  • Last Updated : 28 Jan, 2022

Working on Linux applications we have several ways to get information from outside and to put it inside: command line args, environment variables, files. All of these sources are legal and good. But it has a finite size. Another way to establish communication is standard streams: input stream (stdin) used for getting data from outside of the app, output stream (stdout) to put data outside of the app, and error to put data outside of the app (stderr).

standard Linux application stream at sh

Each stream acts like a pipe: It has the same buffer to write and read the data. This buffer is available for reading from one application and available for writing from another one. On reading, the occupied buffer size will be reduced and it will be increased on writing. If the average rate of reading and writing is equal – then data passed over the stream can be any number of bytes long.

Input/output streams operating in the example

Let’s look at an example of the “grep” application describing how it interacts with it. 

script body:

#!/usr/bin/env bash
echo pattern | grep pattern
echo pattern | grep pattern

The script running at the console:

script body in the yellow frame, output next

This time we ran 2 commands: “cat stdio1.sh” – to list script body, “./stdio1.sh” to run the script. “echo pattern” forms the standard output stream that is connected to the standard input stream of grep. So all of the data comes to grep. grep is an app that writes from input stream to output stream if it matches the pattern. The first input does not match – no output expected. The Second-string of the script “echo pattern | grep pattern” has appropriate input – it has been passed to grep output and displayed at the console next.

Error stream operating in the example

Stderr is very similar to stdout: with only one difference – it will contain an error report if it will take a place

script body:

#!/usr/bin/env bash

ls > /tmp/log
cat /tmp/log

ls /not/existed_path > /tmp/log
cat /tmp/log
cat /tmp/log

rm /tmp/log

The script running at the console:

script body in the yellow frame, output next

At this script, we make a listing of files in the current directory (“ls”) and redirect standard output to the file (“/tmp/log”). Then we display this file data at the console from the standard output stream of the cat utility that read the file. Next, we try to list files in the not existing folder and store results into the file as above, and display it two times. But there is no effect. Just one output about the error in the console. That is because this error comes from the standard errors stream of ls. We removed the file to clear some space “/tmp/log“.

Streams redirecting

It is possible to manipulate the standard streams: redirect them or use pipes to process them.

script body:

#!/usr/bin/env bash

ls /not/existed_path 2>/dev/null
ls * /not/existed_path > /tmp/log 2> /tmp/err_log
cat /tmp/log
rm /tmp/log /tmp/err_log

The script running at the console:

script body in the yellow frame, output next

We use the “>” redirect action in this command. /dev/null – is a stream termination file to drop any input with no processing. “2>” clause redirects standard output stream to file. Now it is possible to look at output and errors later by calling “cat /tmp/log” or “cat /tmp/err_log”.

Discard the output

In some cases, it is useful to drop all of the application output (to save storage space or remove the unimportant sources of data to analyze). We should remember that we should mute not just the standard output stream but the standard error stream also. Let’s look at an example of how does it work.

script body:

#!/usr/bin/env bash

ls . unexisted_file > /dev/null
ls . unexisted_file > /dev/null 2>&1

The script running at the console:

 script body in the yellow frame, output next

In both lines of script standard output stream redirected to file /dev/null. This file is not a file itself: but a special Linux device with a file interface. It has a write function available from shell (that does nothing). So all of the input is just dropped. But the second string of the script has a difference from the first one: “2>&1” at the end of the command. These symbols tell that the second standard stream ( standard error stream ) needs to be directed at the first standard stream (standard output stream) permanently for this command. That is why we see only one error message displayed in the console – at the second command string of the script standard error stream redirected to terminating file device /dev/null in the end.

Pipelined streams

It also can be useful to modify the stream.  is able to run several applications connected by the streams. 

script body:

#!/usr/bin/env bash

touch /tmp/a
ls /tmp/* 2> /dev/null | grep -w a
rm /tmp/a

The script running at the console:

script body in the yellow frame, output next

Here we collect the listing and set “ls standard output stream as grep standard input stream by using “|”. On “2>standard errors stream is redirected to the termination file.  The output stream of grep contains the “/tmp/a” filename from listing that matches by grep pattern. 

Here document

Here the document is redirection option to fill input stream of application by information native way:

application << delimiter
some useful text
delimiter

script body:

#!/usr/bin/env bash

grep perfect << EOF
    PERFECT STORY
    past perfect
    is it Perfect?
EOF

The script running at console

script body in the yellow frame, output next

As we can see only the second string from 3 of the strings of text for the grep standard input stream delimited by “EOF” match the grep pattern: that is why it passed to grep standard output stream and be displayed in the console next.

Conclusion    

There are several standard utilities to work with streams (cat, cut, grep, sed, tr, …). You can use it or write your own to make a data stream processing pipeline that will match your needs. Under the hood, it will work with stdin, stdout, stderr.

My Personal Notes arrow_drop_up
Recommended Articles
Page :

Start Your Coding Journey Now!