Python Program To Remove Duplicates From A Given String

Given a string S, the task is to remove all the duplicates in the given string. 
Below are the different methods to remove duplicates in a string.

Python

string="geeksforgeeks"
k2=[]
for ele in k:
    if ele not in k2:
        k2.append(ele)
for i in range(0,len(k2)):
    print(k2[i],end="")

The time complexity of the given code is O(n), where n is the length of the input string. This is because the for loop iterates over each character in the string once and the append method of the list k2 takes constant time.

The space complexity of the code is also O(n), where n is the length of the input string. This is because the list k2 can store at most n unique characters from the input string.

METHOD 1 (Simple)

Python3

string="geeksforgeeks"
p=""
for char in string:
    if char not in p:
        p=p+char
print(p)
k=list("geeksforgeeks")

Output

geksfor

Time Complexity : O(n * n) 
Auxiliary Space : O(1) 
Keeps order of elements the same as input. 

METHOD 2 (Use BST)
use set which implements a self-balancing Binary Search Tree. 

Python3

# Python program to remove duplicate character
# from character array and print in sorted
# order
def removeDuplicate(str, n):
    s = set()

    # Create a set using String characters
    for i in str:
        s.add(i)

    # Print content of the set
    st = ""
    for i in s:
        st = st+i
    return st


# Driver code
str = "geeksforgeeks"
n = len(str)
print(removeDuplicate(list(str), n))

# This code is contributed by rajsanghavi9.

Output

ogerfsk

Time Complexity: O(n Log n) 
Auxiliary Space: O(n)

Thanks to Anivesh Tiwari for suggesting this approach.

It does not keep the order of elements the same as the input but prints them in sorted order.

METHOD 3 (Use Sorting)
Algorithm:

  1) Sort the elements.
  2) Now in a loop, remove duplicates by comparing the 
      current character with previous character.
  3)  Remove extra characters at the end of the resultant string.

Example:

Input string:  geeksforgeeks
1) Sort the characters
   eeeefggkkorss
2) Remove duplicates
    efgkorskkorss
3) Remove extra characters
     efgkors

Note that, this method doesn’t keep the original order of the input string. For example, if we are to remove duplicates for geeksforgeeks and keep the order of characters the same, then the output should be geksfor, but the above function returns efgkos. We can modify this method by storing the original order.

Implementation:

Python

# Python program to remove duplicates, the order of
# characters is not maintained in this program

# Utility function to convert string to list
def toMutable(string):
    temp = []
    for x in string:
        temp.append(x)
    return temp

# Utility function to convert string to list
def toString(List):
    return ''.join(List)

# Function to remove duplicates in a sorted array
def removeDupsSorted(List):
    res_ind = 1
    ip_ind = 1

    # In place removal of duplicate characters
    while ip_ind != len(List):
        if List[ip_ind] != List[ip_ind-1]:
            List[res_ind] = List[ip_ind]
            res_ind += 1
        ip_ind+=1

    # After above step string is efgkorskkorss.
    # Removing extra kkorss after string
    string = toString(List[0:res_ind])

    return string

# Function removes duplicate characters from the string
# This function work in-place and fills null characters
# in the extra space left
def removeDups(string):
    # Convert string to list
    List = toMutable(string)

    # Sort the character list
    List.sort()

    # Remove duplicates from sorted
    return removeDupsSorted(List)

# Driver program to test the above functions
string = "geeksforgeeks"
print removeDups(string)

# This code is contributed by Bhavya Jain

Output:

efgkors

Time Complexity: O(n log n) If we use some nlogn sorting algorithm instead of quicksort.

Auxiliary Space: O(1)

METHOD 4 (Use Hashing )

Algorithm:

1: Initialize:
    str  =  "test string" /* input string */
    ip_ind =  0          /* index to  keep track of location of next
                             character in input string */
    res_ind  =  0         /* index to  keep track of location of
                            next character in the resultant string */
    bin_hash[0..255] = {0,0, ….} /* Binary hash to see if character is 
                                        already processed or not */
2: Do following for each character *(str + ip_ind) in input string:
              (a) if bin_hash is not set for *(str + ip_ind) then
                   // if program sees the character *(str + ip_ind) first time
                         (i)  Set bin_hash for *(str + ip_ind)
                         (ii)  Move *(str  + ip_ind) to the resultant string.
                              This is done in-place.
                         (iii) res_ind++
              (b) ip_ind++
  /* String obtained after this step is "the stringing" */
3: Remove extra characters at the end of the resultant string.
  /*  String obtained after this step is "te string" */

Implementation:

Python

# Python program to remove duplicate characters from an
# input string
NO_OF_CHARS = 256

# Since strings in Python are immutable and cannot be changed
# This utility function will convert the string to list
def toMutable(string):
    List = []
    for i in string:
        List.append(i)
    return List

# Utility function that changes list to string
def toString(List):
    return ''.join(List)

# Function removes duplicate characters from the string
# This function work in-place and fills null characters
# in the extra space left
def removeDups(string):
    bin_hash = [0] * NO_OF_CHARS
    ip_ind = 0
    res_ind = 0
    temp = ''
    mutableString = toMutable(string)

    # In place removal of duplicate characters
    while ip_ind != len(mutableString):
        temp = mutableString[ip_ind]
        if bin_hash[ord(temp)] == 0:
            bin_hash[ord(temp)] = 1
            mutableString[res_ind] = mutableString[ip_ind]
            res_ind+=1
        ip_ind+=1

     # After above step string is stringiittg.
     # Removing extra iittg after string
    return toString(mutableString[0:res_ind])

# Driver program to test the above functions
string = "geeksforgeeks"
print(removeDups(string))

# A shorter version for this program is as follows
# import collections
# print ''.join(collections.OrderedDict.fromkeys(string))

# This code is contributed by Bhavya Jain

Output

geksfor

Time Complexity: O(n)

Space Complexity: O(1)

Important Points:

• Method 2 doesn’t maintain the characters as original strings, but method 4 does.
• It is assumed that the number of possible characters in the input string is 256. NO_OF_CHARS should be changed accordingly.
• calloc() is used instead of malloc() for memory allocations of a counting array (count) to initialize allocated memory to ‘�’. the malloc() followed by memset() could also be used.
• The above algorithm also works for integer array inputs if the range of the integers in the array is given. An example problem is to find the maximum occurring number in an input array given that the input array contains integers only between 1000 to 1100

Method 5 (Using IndexOf() method) : 
Prerequisite : Java IndexOf() method

Python3

# Python 3 program to create a unique string

# Function to make the string unique


def unique(s):

    st = ""
    length = len(s)

    # loop to traverse the string and
    # check for repeating chars using
    # IndexOf() method in Java
    for i in range(length):

        # character at i'th index of s
        c = s[i]

        # if c is present in str, it returns
        # the index of c, else it returns - 1
        # print(st.index(c))
        if c not in st:
            # adding c to str if -1 is returned
            st += c

    return st


# Driver code
if __name__ == "__main__":

    # Input string with repeating chars
    s = "geeksforgeeks"

    print(unique(s))

    # This code is contributed by ukasp.

Output

geksfor

Thanks debjitdbb for suggesting this approach.

Method #6: Using Counter() function and Hashing Logic

Python3

from collections import Counter
string = "geeksforgeeks"
freq = Counter(string)
for char in string:
    if freq[char] != 0:
        print(char, end='')
        freq[char] = 0

Output

geksfor

Please refer complete article on Remove duplicates from a given string for more details!

Method 7:  using operator.countOf() method

Python3

import operator as op
string="geeksforgeeks"
k2=[]
for ele in string:
    if op.countOf(k2,ele)==0 :
        k2.append(ele)
for i in range(0,len(k2)):
    print(k2[i],end="")

Output

geksfor

Time Complexity: O(N)

Auxiliary Space : O(N)

 Method 8: Using pandas.unique() and join():

1. First, we define the input string string = “geeksforgeeks”.
2. Then, we create an empty list k2 = [].
3. We loop through each character ele in the input string string.
4. We use the countOf() method from the operator module to check if the current character ele is already present in the list k2.
5. If the count is 0, it means the character is not already present in the list, so we append it to the list k2.
6. After the loop completes, we have a list k2 that contains all the unique characters in the input string.
7. We loop through each index i in the range 0 to len(k2) – 1.
8. For each index i, we print the character at that index in the list k2.
9. Finally, the output is the string geksfor.

Python3

import pandas as pd

# Define the input string
string = "geeksforgeeks"

# Convert the string to a list of characters, and use pd.unique() to remove duplicates
unique_chars = pd.unique(list(string))

# Join the unique characters back into a string using the str.join() method
result = ''.join(unique_chars)

# Print the result
print(result)
#This code is contributed by Vinay Pinjala

Output:
geksfor

Time complexity: O(n^2)
Auxiliary Space: O(n)