Python | Unique dictionary filter in list
Last Updated :
24 Mar, 2023
While working with Python dictionaries, we can always come across a problem in which we have list of dictionary with possible repetitions and we wish to remove the duplicates. This is common problem one can face in Machine Learning Domain. Let’s discuss certain way in which this problem can be solved.
Method : Using map() + set() + items() + sorted() + tuple() In this method, we just combination of functions to perform this task. The map function performs the task of extending logic to all elements. The set function does the actual filtering on pairs accessed by items(). The sorted function is required by set to remove duplicate.
Python3
test_list = [{ 'gfg' : 1 , 'is' : 2 , 'best' : 3 },
{ 'gfg' : 1 , 'is' : 2 , 'best' : 3 },
{ 'gfg' : 9 , 'is' : 8 , 'best' : 6 }]
print ( "The original list is : " + str (test_list))
res = list ( map ( dict , set ( tuple ( sorted (sub.items())) for sub in test_list)))
print ( "The list after removing duplicates : " + str (res))
|
Output
The original list is : [{'gfg': 1, 'is': 2, 'best': 3}, {'gfg': 1, 'is': 2, 'best': 3}, {'gfg': 9, 'is': 8, 'best': 6}]
The list after removing duplicates : [{'best': 3, 'gfg': 1, 'is': 2}, {'best': 6, 'gfg': 9, 'is': 8}]
Time Complexity: O(n*nlogn) where n is the number of elements in the list “test_list”.
Auxiliary Space: O(n), where n is the number of elements in the new res list
Method #2 : Using list comprehension + dict()
This method uses list comprehension to filter unique dictionaries and the dict() function to convert the filtered list of tuples to a list of dictionaries.
Python3
test_list = [{ 'gfg' : 1 , 'is' : 2 , 'best' : 3 },
{ 'gfg' : 1 , 'is' : 2 , 'best' : 3 },
{ 'gfg' : 9 , 'is' : 8 , 'best' : 6 }]
print ( "The original list is : " + str (test_list))
res = [ dict (t) for t in { tuple (d.items()) for d in test_list}]
print ( "The list after removing duplicates : " + str (res))
|
Output
The original list is : [{'gfg': 1, 'is': 2, 'best': 3}, {'gfg': 1, 'is': 2, 'best': 3}, {'gfg': 9, 'is': 8, 'best': 6}]
The list after removing duplicates : [{'gfg': 1, 'is': 2, 'best': 3}, {'gfg': 9, 'is': 8, 'best': 6}]
Time complexity: O(n)
Auxiliary Space: O(n)
Like Article
Suggest improvement
Share your thoughts in the comments
Please Login to comment...