Java program to delete duplicate lines in text file
Prerequisite : PrintWriter , BufferedReader
Given a file input.txt . Our Task is to remove duplicate lines from it and save the output in file say output.txt
Naive Algorithm :
1. Create PrintWriter object for output.txt
2. Open BufferedReader for input.txt
3. Run a loop for each line of input.txt
3.1 flag = false
3.2 Open BufferedReader for output.txt
3.3 Run a loop for each line of output.txt
-> If line of output.txt is equal to current line of input.txt
-> flag = true
-> break loop
4. Check flag, if false
-> write current line of input.txt to output.txt
-> Flush PrintWriter stream
5. Close resources.
To successfully run the below program input.txt must exits in same folder OR provide full path for it.
import java.io.*;
public class FileOperation
{
public static void main(String[] args) throws IOException
{
PrintWriter pw = new PrintWriter( "output.txt" );
BufferedReader br1 = new BufferedReader( new FileReader( "input.txt" ));
String line1 = br1.readLine();
while (line1 != null )
{
boolean flag = false ;
BufferedReader br2 = new BufferedReader( new FileReader( "output.txt" ));
String line2 = br2.readLine();
while (line2 != null )
{
if (line1.equals(line2))
{
flag = true ;
break ;
}
line2 = br2.readLine();
}
if (!flag){
pw.println(line1);
pw.flush();
}
line1 = br1.readLine();
}
br1.close();
pw.close();
System.out.println( "File operation performed successfully" );
}
}
|
Output:
File operation performed successfully
Note : If output.txt exist in cwd(current working directory) then it will be overwritten by above program otherwise new file will be created.
A better solution is to use HashSet to store each line of input.txt. As set ignores duplicate values, so while storing a line, check if it already present in hashset. Write it to output.txt only if not present in hashset.
To successfully run the below program input.txt must exits in same folder OR provide full path for them.
import java.io.*;
import java.util.HashSet;
public class FileOperation
{
public static void main(String[] args) throws IOException
{
PrintWriter pw = new PrintWriter( "output.txt" );
BufferedReader br = new BufferedReader( new FileReader( "input.txt" ));
String line = br.readLine();
HashSet<String> hs = new HashSet<String>();
while (line != null )
{
if (hs.add(line))
pw.println(line);
line = br.readLine();
}
pw.flush();
br.close();
pw.close();
System.out.println( "File operation performed successfully" );
}
}
|
Output:
File operation performed successfully
Note : If output.txt exist in cwd(current working directory) then it will be overwritten by above program otherwise new file will be created.
Last Updated :
30 May, 2018
Like Article
Save Article
Share your thoughts in the comments
Please Login to comment...