Open In App

Python – Remove all duplicate occurring tuple records

Improve
Improve
Improve
Like Article
Like
Save Article
Save
Share
Report issue
Report

Sometimes, while working with records, we can have a problem of removing those records which occur more than once. This kind of application can occur in web development domain. Let’s discuss certain ways in which this task can be performed. 

Method #1 : Using list comprehension + set() + count() Initial approach that can be applied is that we can iterate on each tuple and check it’s count in list using count(), if greater than one, we can remove them, converting then to set. 

Step-by-step approach :

  • Create a list comprehension that iterates over each element ele in test_list
  • Check if the count of ele in test_list is greater than 1 (meaning it is a duplicate)
  • If it is not duplicate, add it to a set
  • Convert the set back into a list and assign it to res
  • Print the resulting list of non-duplicate tuples

Below is the implementation of the above approach:

Python3




# Python3 code to demonstrate working of
# Remove all Duplicate occurring records
# Using list comprehension + set() + count()
 
# initialize list
test_list = [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
 
# printing original list
print("The original list : " + str(test_list))
 
# Remove all Duplicate occurring records
# Using list comprehension + set() + count()
res = list(set([ele for ele in test_list if not test_list.count(ele) > 1]))
 
# printing result
print("All the non Duplicate from list are : " + str(res))


Output : 

The original list : [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
All the non Duplicate from list are : [(6, 7), (3, 6)]

Time complexity: O(n^2), where n is the length of the input list test_list.
Auxiliary space: O(n), where n is the length of the input list test_list. 

Method #2 : Using Counter() + items() + list comprehension The combination of above functions can also be used to perform this particular task. In this, we just get the count of each occurrence of element using Counter() as dictionary and then extract all those whose value is equal to 1. 

Python3




# Python3 code to demonstrate working of
# Remove all Duplicate occurring records
# Using list comprehension + Counter() + items()
from collections import Counter
 
# initialize list
test_list = [(3, 4), (4, 5), (3, 6), (3, 4), (4, 5), (6, 7)]
 
# printing original list
print("The original list : " + str(test_list))
 
# Remove all Duplicate occurring records
# Using list comprehension + Counter() + items()
res = [ele for ele, count in Counter(test_list).items() if count == 1]
 
# printing result
print("All the non Duplicate from list are : " + str(res))


Output : 

The original list : [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
All the non Duplicate from list are : [(6, 7), (3, 6)]

Time complexity: O(n), where n is the length of the input list.
Auxiliary space: O(n), where n is the length of the input list. 

Method #3: Using operator.countOf() method

Python3




# Python3 code to demonstrate working of
# Remove all Duplicate occurring records
import operator as op
 
# initialize list
test_list = [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
 
# printing original list
print("The original list : " + str(test_list))
 
# Remove all Duplicate occurring records
# Using list comprehension + set() + count()
res = list(
    set([ele for ele in test_list if not op.countOf(test_list, ele) > 1]))
 
# printing result
print("All the non Duplicate from list are : " + str(res))


Output

The original list : [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
All the non Duplicate from list are : [(6, 7), (3, 6)]

Time Complexity: O(N)
Auxiliary Space: O(N)

Method #4: Using a dictionary to track occurrences

This method uses a dictionary to track the occurrences of each element in the list. We iterate through the list and increment the count for each element in the dictionary. Then we use a list comprehension to extract all elements with count 1, which are the non-duplicate elements.

Python3




# Python3 code to demonstrate working of
# Remove all Duplicate occurring records
 
# initialize list
test_list = [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
 
# printing original list
print("The original list : " + str(test_list))
 
# Remove all Duplicate occurring records
# Using dictionary to track occurrences
dict_count = {}
for ele in test_list:
    dict_count[ele] = dict_count.get(ele, 0) + 1
res = [ele for ele in test_list if dict_count[ele] == 1]
 
# printing result
print("All the non Duplicate from list are : " + str(res))


Output

The original list : [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
All the non Duplicate from list are : [(3, 6), (6, 7)]

Time complexity: O(n) because it iterates through the list once to create the dictionary, and then once more to extract the non-duplicate elements using a list comprehension.
Auxiliary space: O(n) because it uses a dictionary to track the occurrences of each element in the list, which can potentially store all the elements of the list. 

Method #5: Using a loop and a set to track occurrences

Iterate through the list and use a dictionary to keep track of the items that have already appeared. If an item has not appeared before, we add it to the output list.

Python3




# initialize list
test_list = [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
 
# printing original list
print("The original list : " + str(test_list))
 
# Remove all Duplicate occurring records
# Using a loop and a dictionary to track occurrences
occurrences = {}
for ele in test_list:
    if ele not in occurrences:
        occurrences[ele] = 1
    else:
        occurrences[ele] += 1
 
res = [ele for ele in test_list if occurrences[ele] == 1]
 
# printing result
print("All the non Duplicate from list are : " + str(res))


Output

The original list : [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
All the non Duplicate from list are : [(3, 6), (6, 7)]

Time complexity: O(n), where n is the length of the input list.
Auxiliary space: O(n), since we use a dictionary to keep track of occurrences.

Method 6: Using list.count() method and list slicing to find non-duplicate items

  1. Create an empty result list called res.
  2. Iterate through each item in the original list test_list.
  3. If the count of the item in test_list is equal to 1, append the item to the result list res.
  4. Print the result list res.

Python3




# initialize list
test_list = [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
 
# printing original list
print("The original list : " + str(test_list))
 
# Remove all Duplicate occurring records
# Using list.count() method and list slicing to find non-duplicate items
res = [ele for ele in test_list if test_list.count(ele) == 1]
 
# printing result
print("All the non Duplicate from list are : " + str(res))


Output

The original list : [(3, 4), (4, 5), (3, 4), (3, 6), (4, 5), (6, 7)]
All the non Duplicate from list are : [(3, 6), (6, 7)]

Time complexity: O(n^2)
Auxiliary space: O(n)



Last Updated : 05 Apr, 2023
Like Article
Save Article
Previous
Next
Share your thoughts in the comments
Similar Reads