Python | Remove duplicate dictionaries from nested dictionary

Given a nested dictionary, the task is to remove duplicate dictionaries from the dictionary. Given below are few methods to complete the given task.

Method #1: Using Naive Method

Python3

# Python code to demonstrate
# for removing duplicate values from dictionary

# initialising dictionary
ini_dict = {'a':{'b':1, 'c':2}, 'b':{'b':1, 'c':2},
            'c':{'a':2, 'b':3}, 'd':{'a':2, 'b':7}}

# printing initial_dictionary
print ("initial dictionary", str(ini_dict))

# code to remove duplicates
result = {}

for key, value in ini_dict.items():
    if value not in result.values():
        result[key] = value

# printing result
print ("result", str(result))

Output:

initial dictionary {‘c’: {‘a’: 2, ‘b’: 3}, ‘d’: {‘a’: 2, ‘b’: 7}, ‘a’: {‘c’: 2, ‘b’: 1}, ‘b’: {‘c’: 2, ‘b’: 1}} result {‘c’: {‘a’: 2, ‘b’: 3}, ‘d’: {‘a’: 2, ‘b’: 7}, ‘a’: {‘c’: 2, ‘b’: 1}}

Time complexity: O(n), where n is the number of key-value pairs in the dictionary.
Auxiliary space: O(n), to store the keys and values in dictionary.

Method #2: Using sorted and set

Python3

# Python code to demonstrate
# for removing duplicate values from dictionary

# initialising dictionary
ini_dict = {'a':{'b':1, 'c':2}, 'b':{'b':1, 'c':2},
            'c':{'a':2, 'b':3}, 'd':{'a':2, 'b':7}}

# printing initial_dictionary
print ("initial dictionary", str(ini_dict))

# code to remove duplicates
keep = set({repr(sorted(value.items())):key
          for key, value in ini_dict.items()}.values())

for key in list(ini_dict):
    if key not in keep:
        del ini_dict[key]

# printing result
print ("result", str(ini_dict))

Output:

initial dictionary {‘a’: {‘b’: 1, ‘c’: 2}, ‘b’: {‘b’: 1, ‘c’: 2}, ‘c’: {‘a’: 2, ‘b’: 3}, ‘d’: {‘a’: 2, ‘b’: 7}} result {‘b’: {‘b’: 1, ‘c’: 2}, ‘c’: {‘a’: 2, ‘b’: 3}, ‘d’: {‘a’: 2, ‘b’: 7}}

Method #3: Using OrderedDict

This code converts a nested dictionary ini_dict into another dictionary called result, with the original keys as the keys and the dictionaries as the values where duplicates are removed. It does this by first creating an OrderedDict called odict, where the keys are tuples containing the items of the dictionaries in ini_dict, and the values are the keys of the dictionaries in ini_dict. It then converts odict back into a regular dictionary called result, with the original keys as the keys and the dictionaries as the values.

Python3

from collections import OrderedDict

# Initialize the nested dictionary
ini_dict = {'a': {'b': 1, 'c': 2}, 'b': {'b': 1, 'c': 2},
            'c': {'a': 2, 'b': 3}, 'd': {'a': 2, 'b': 7}}
# printing initial_dictionary
print ("initial dictionary: "+str(ini_dict))
# Create an OrderedDict with the values as the keys and the original keys as the values
odict = OrderedDict((tuple(value.items()), key) for key, value in ini_dict.items())

# Convert the OrderedDict back into a regular dictionary, with the original keys as the keys
# and the dictionaries as the values
result = {value: dict(key) for key, value in odict.items()}
print("Result : "+str(result))
#This code is contributed by Edula Vinay Kumar Reddy

Output

initial dictionary: {'a': {'b': 1, 'c': 2}, 'b': {'b': 1, 'c': 2}, 'c': {'a': 2, 'b': 3}, 'd': {'a': 2, 'b': 7}}
Result : {'b': {'b': 1, 'c': 2}, 'c': {'a': 2, 'b': 3}, 'd': {'a': 2, 'b': 7}}

In terms of time complexity, the most time-consuming step is likely the creation of the OrderedDict, which has a time complexity of O(n), where n is the number of dictionaries in ini_dict. The iteration over the keys and values of odict has a time complexity of O(n), and the conversion back to a regular dictionary has a time complexity of O(n). Therefore, the overall time complexity of this code is O(n). Space complexity is O(n)

Method #4: Using list comprehension and lambda function

One way to remove duplicate values from a dictionary is to use a list comprehension with a lambda function to check if each value has already been added to the result dictionary.

Step-by-step approach:

• Initialize the dictionary
• Use a list comprehension to create a list of unique values from the dictionary using a lambda function that checks if the current value is already in the list of values that have been added to the result dictionary
• Create a new dictionary using the list of unique values and their corresponding keys
• Print the original dictionary and the result dictionary

Below is the implementation of the above approach:

Python3

# Python code to demonstrate
# for removing duplicate values from dictionary

# initialising dictionary
ini_dict = {'a':{'b':1, 'c':2}, 'b':{'b':1, 'c':2},
            'c':{'a':2, 'b':3}, 'd':{'a':2, 'b':7}}

# printing initial dictionary
print("initial dictionary", str(ini_dict))

# code to remove duplicates
values = set()
result = {}

# create a list of unique values using a temporary set
for key, value in ini_dict.items():
    item = tuple(value.items())
    if item not in values:
        values.add(item)
        result[key] = value

# printing result
print("result", str(result))

Output

initial dictionary {'a': {'b': 1, 'c': 2}, 'b': {'b': 1, 'c': 2}, 'c': {'a': 2, 'b': 3}, 'd': {'a': 2, 'b': 7}}
result {'a': {'b': 1, 'c': 2}, 'c': {'a': 2, 'b': 3}, 'd': {'a': 2, 'b': 7}}

Time complexity: O(n^2) because the approach involves iterating over the dictionary twice.
Auxiliary space: O(n) because the set of unique values may be as large as the original dictionary.

Method #5: Using reduce method:

Algorithm:

1. Initialize an empty dictionary called result.
2. Iterate over each key-value pair in the initial dictionary ini_dict.
3. For each key-value pair, check if the value is already present in the result dictionary’s values.
4. If it is not, add the key-value pair to the result dictionary.
5. Return the result dictionary as the final output.

Python3

from functools import reduce

ini_dict = {'a': {'b': 1, 'c': 2}, 'b': {'b': 1, 'c': 2},
            'c': {'a': 2, 'b': 3}, 'd': {'a': 2, 'b': 7}}
# printing initial dictionary
print("initial dictionary", str(ini_dict))

result = reduce(lambda r, k: ({**r, k: ini_dict[k]} if ini_dict[k] not in r.values() else r), ini_dict, {})

print("result", str(result))
#This code is contributed by Rayudu.

Output

initial dictionary {'a': {'b': 1, 'c': 2}, 'b': {'b': 1, 'c': 2}, 'c': {'a': 2, 'b': 3}, 'd': {'a': 2, 'b': 7}}
result {'a': {'b': 1, 'c': 2}, 'c': {'a': 2, 'b': 3}, 'd': {'a': 2, 'b': 7}}

Time Complexity:
The time complexity of this algorithm is O(n^2), where n is the number of key-value pairs in the initial dictionary. This is because for each key-value pair, we are iterating over all the values in the result dictionary to check for duplicates.

Auxiliary Space:
The space complexity of this algorithm is O(n), where n is the number of key-value pairs in the initial dictionary. This is because we are creating a new dictionary result to store the unique key-value pairs, which can contain at most n key-value pairs.