uniq command in Linux with examples

The linux uniq command is basically used to remove all repeated lines in a file. This command is used when a line is repeated multiple times and replace these multiple lines with one line. This command is designed to work on sorted files.

Purpose : Remove repetitious lines from the sorted ‘input-file’ and send unique lines to the ‘output-file’. If no ‘output-file’ is specified, the output of the command is send to standard output. if no ‘input-file’ is specified, the command takes input from standard output.

Syntax :

$ uniq [option] [input-file] [output-file]

Options :

-c : Precede each output line by the number of times it occur.
-d : Display the repeated lines.
-u : Display the lines that are not repeated.

Examples :
Create a sample input file.

$ cat sample

Input :
This is a test file for the uniq command.
It contains some repeated lines.
It contains some repeated lines.
And some are different.
It contains some repeated lines.
It contains some repeated lines.

1. Removing duplicate lines when file is not sorted.

$ uniq sample

Output :
This is a test file for the uniq command.
It contains some repeated lines.
And some are different.
It contains some repeated lines.

2. Removing duplicate files after sorting.

$ sort sample -> sample1
$ uniq sample1

Output :
And some are different.
It contains some repeated lines.
This is a test file for the uniq command.

3. Finding number of times the lines are repeated.

$ uniq -c sample1

Output :

1 And some are different.
4 It contains some repeated lines.
1 This is a test file for the uniq command.

4. Finding repeated lines.

$ uniq -d sample1

Output :

It contains some repeated lines.

5. Finding lines which are unique and store in another file.

$ uniq -u sample1 out
$ cat out

Output :

And some are different.
This is a test file for the uniq command.


My Personal Notes arrow_drop_up

Check out this Author's contributed articles.

If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.




Article Tags :

1


Please write to us at contribute@geeksforgeeks.org to report any issue with the above content.