Python – Remove duplicate values across Dictionary Values

Sometimes, while working with Python dictionaries, we can have a problem in which we need to remove all the duplicate values across all the dictionary value lists. This problem can have applications in data domains and web development domains. Let’s discuss certain ways in which this task can be performed.

Input: test_dict = {‘Manjeet’: [1], ‘Akash’: [1, 8, 9]} 
Output: {‘Manjeet’: [], ‘Akash’: [8, 9]} 

Input: test_dict = {‘Manjeet’: [1, 1, 1], ‘Akash’: [1, 1, 1]} 
Output: {‘Manjeet’: [], ‘Akash’: []}

Method #1: Using Counter() + list comprehension The combination of the above functions can be used to solve this problem. In this, we use Counter() to extract all frequencies and list comprehension to assign the value with a single occurrence in the value list. 

Python3

# Python3 code to demonstrate working of
# Remove duplicate values across Dictionary Values
# Using Counter() + list comprehension
from collections import Counter

# initializing dictionary
test_dict = {'Manjeet': [1, 4, 5, 6],
             'Akash': [1, 8, 9],
             'Nikhil': [10, 22, 4],
             'Akshat': [5, 11, 22]}

# printing original dictionary
print("The original dictionary : " + str(test_dict))

# Remove duplicate values across Dictionary Values
# Using Counter() + list comprehension
cnt = Counter()
for idx in test_dict.values():
    cnt.update(idx)
res = {idx: [key for key in j if cnt[key] == 1]
       for idx, j in test_dict.items()}

# printing result
print("Uncommon elements records : " + str(res))

Output

The original dictionary : {'Manjeet': [1, 4, 5, 6], 'Akash': [1, 8, 9], 'Nikhil': [10, 22, 4], 'Akshat': [5, 11, 22]}
Uncommon elements records : {'Manjeet': [6], 'Akash': [8, 9], 'Nikhil': [10], 'Akshat': [11]}

Method #2 : Using extend(),count(),keys() methods and for loops

Python3

# Python3 code to demonstrate working of
# Remove duplicate values across Dictionary Values

# initializing dictionary
test_dict = {'Manjeet': [1, 4, 5, 6],
             'Akash': [1, 8, 9],
             'Nikhil': [10, 22, 4],
             'Akshat': [5, 11, 22]}

# printing original dictionary
print("The original dictionary : " + str(test_dict))

# Remove duplicate values across Dictionary Values
x = []
for i in test_dict.keys():
    x.extend(test_dict[i])
y = []
for i in x:
    if(x.count(i) == 1):
        y.append(i)
res = dict()

for i in test_dict.keys():
    a = []
    for j in test_dict[i]:
        if j in y:
            a.append(j)
        res[i] = a

# printing result
print("Uncommon elements records : " + str(res))

Output

The original dictionary : {'Manjeet': [1, 4, 5, 6], 'Akash': [1, 8, 9], 'Nikhil': [10, 22, 4], 'Akshat': [5, 11, 22]}
Uncommon elements records : {'Manjeet': [6], 'Akash': [8, 9], 'Nikhil': [10], 'Akshat': [11]}

Method #3 : Using extend(),operator.countOf(),keys() methods and for loops

Python3

# Python3 code to demonstrate working of
# Remove duplicate values across Dictionary Values
import operator as op
# initializing dictionary
test_dict = {'Manjeet': [1, 4, 5, 6],
             'Akash': [1, 8, 9],
             'Nikhil': [10, 22, 4],
             'Akshat': [5, 11, 22]}

# printing original dictionary
print("The original dictionary : " + str(test_dict))

# Remove duplicate values across Dictionary Values
x = []
for i in test_dict.keys():
    x.extend(test_dict[i])
y = []
for i in x:
    if(op.countOf(x,i) == 1):
        y.append(i)
res = dict()

for i in test_dict.keys():
    a = []
    for j in test_dict[i]:
        if j in y:
            a.append(j)
        res[i] = a

# printing result
print("Uncommon elements records : " + str(res))

Output

The original dictionary : {'Manjeet': [1, 4, 5, 6], 'Akash': [1, 8, 9], 'Nikhil': [10, 22, 4], 'Akshat': [5, 11, 22]}
Uncommon elements records : {'Manjeet': [6], 'Akash': [8, 9], 'Nikhil': [10], 'Akshat': [11]}

Time Complexity: O(n*n)

Auxiliary Space:  O(n)