Python | Merging two list of dictionaries

Given two list of dictionaries, the task is to merge these two lists of dictionaries based on some value. 

Merging two list of dictionaries

Using defaultdict and extend to merge two list of dictionaries based on school_id. 

Python3

# Python code to  merge two list of dictionaries
# based on some value.

from collections import defaultdict

# List initialization
Input1 = [{'roll_no': ['123445', '1212'], 'school_id': 1},
          {'roll_no': ['HA-4848231'], 'school_id': 2}]

Input2 = [{'roll_no': ['473427'], 'school_id': 2},
          {'roll_no': ['092112'], 'school_id': 5}]


# Using defaultdict
temp = defaultdict(list)

# Using extend
for elem in Input1:
    temp[elem['school_id']].extend(elem['roll_no'])

for elem in Input2:
    temp[elem['school_id']].extend(elem['roll_no'])

Output = [{"roll_no":y, "school_id":x} for x, y in temp.items()]

# printing
print(Output)

Output:

[{‘school_id’: 1, ‘roll_no’: [‘123445’, ‘1212’]}, {‘school_id’: 2, ‘roll_no’: [‘HA-4848231’, ‘473427’]}, {‘school_id’: 5, ‘roll_no’: [‘092112’]}]

Merging two list of dictionaries Using extend() only.

Python3

# Python code to  merge two list of dictionaries
# based on some value.

# List initialization
Input1 = [{'roll_no': ['123445', '1212'], 'school_id': 1},
          {'roll_no': ['HA-4848231'], 'school_id': 2}]
Input2 = [{'roll_no': ['473427'], 'school_id': 2},
          {'roll_no': ['092112'], 'school_id': 5}]

# Iterating and using extend to convert
for elm2 in Input2:

    for elm1 in Input1:
        if elm2['school_id'] == elm1['school_id']:
            elm1['roll_no'].extend(elm2['roll_no'])
            break
    else:
        Input1.append(elm2)

# printing
print(Input1)

Output:

[{‘school_id’: 1, ‘roll_no’: [‘123445’, ‘1212’]}, {‘school_id’: 2, ‘roll_no’: [‘HA-4848231’, ‘473427’]}, {‘school_id’: 5, ‘roll_no’: [‘092112’]}]

Merging two list of dictionaries Using a simple for loop and if-else statements. 

Step-by-step approach:

• Initialize an empty list called merged_list.
• Loop through each dictionary in Input1:
  • Extract the school_id from the dictionary.
    • Loop through each dictionary in Input2:
      •  Check if the school_id of the current dictionary in Input2 matches with the school_id of the current dictionary in Input1.
      • If there’s a match, extract the roll_no from the current dictionary in Input2 and extend it to the roll_no list of the current dictionary in Input1.
      • Break out of the loop and continue to the next dictionary in Input1.
    • If there’s no match found in Input2, append the current dictionary from Input1 to merged_list.
• Loop through each dictionary in Input2:
  • Extract the school_id from the dictionary.
  • Loop through each dictionary in merged_list:
    • Check if the school_id of the current dictionary in merged_list matches with the school_id of the current dictionary in Input2.
    • If there’s a match, continue to the next dictionary in Input2.
    • If there’s no match found in merged_list, append the current dictionary from Input2 to merged_list.
• Print the final merged_list.

Below is the implementation of the above approach:

Python3

# List initialization
Input1 = [{'roll_no': ['123445', '1212'], 'school_id': 1},
          {'roll_no': ['HA-4848231'], 'school_id': 2}]
Input2 = [{'roll_no': ['473427'], 'school_id': 2},
          {'roll_no': ['092112'], 'school_id': 5}]

# Initialize an empty dictionary to store the merged dictionaries
merged_dict = {}

# Loop through each dictionary in Input1 and add them to the merged_dict
for dict1 in Input1:
    merged_dict[dict1['school_id']] = {'school_id': dict1['school_id'], 'roll_no': dict1['roll_no']}

# Loop through each dictionary in Input2 and update the corresponding dictionary in merged_dict or add a new dictionary if it doesn't exist
for dict2 in Input2:
    if dict2['school_id'] in merged_dict:
        merged_dict[dict2['school_id']]['roll_no'].extend(dict2['roll_no'])
    else:
        merged_dict[dict2['school_id']] = {'school_id': dict2['school_id'], 'roll_no': dict2['roll_no']}

# Convert the merged_dict to a list of dictionaries and sort it based on the school_id
merged_list = sorted(merged_dict.values(), key=lambda x: x['school_id'])

# Print the merged_list
print(merged_list)

Output

[{'school_id': 1, 'roll_no': ['123445', '1212']}, {'school_id': 2, 'roll_no': ['HA-4848231', '473427']}, {'school_id': 5, 'roll_no': ['092112']}]

Time complexity: O(n log n) due to the sorting step, where n is the total number of dictionaries in both Input1 and Input2. 
Auxiliary space: O(n) because we’re creating a new dictionary to store the merged dictionaries.

Merging two list of dictionaries Using  merge dictionary

1.Merge both input lists.
2.Create a dictionary to store the merged data.
3.Iterate over the dictionaries in the merged list.
4.If the school ID is already present in the dictionary, append the roll numbers to the existing list.
5.If the school ID is not present in the dictionary, create a new entry with the school ID and roll numbers.

Python3

Input1 = [{'roll_no': ['123445', '1212'], 'school_id': 1},
          {'roll_no': ['HA-4848231'], 'school_id': 2}]
Input2 = [{'roll_no': ['473427'], 'school_id': 2},
          {'roll_no': ['092112'], 'school_id': 5}]

merged_list = Input1 + Input2
merged_dict = {}

for entry in merged_list:
    if entry['school_id'] in merged_dict:
        merged_dict[entry['school_id']]['roll_no'].extend(entry['roll_no'])
    else:
        merged_dict[entry['school_id']] = {'school_id': entry['school_id'], 'roll_no': entry['roll_no']}

output_list = list(merged_dict.values())

print(output_list)

Output

[{'school_id': 1, 'roll_no': ['123445', '1212']}, {'school_id': 2, 'roll_no': ['HA-4848231', '473427']}, {'school_id': 5, 'roll_no': ['092112']}]

Time complexity: O(n), where n is the total number of entries in the input lists.

Space complexity: O(n), where n is the total number of entries in the input lists.

Merging two list of dictionaries Using Pandas library

Using the inbuilt merge() and to_dict() methods of Pandas library we can merge a list of dictionaries into 1 in Python by first converting them into DataFrames.

Step – 1 : Firstly we will require two variables to store two lists of dictionaries.

Step – 2 : Then we will need to use two more variables to convert each of those lists into DataFrames.

Step – 3 : Then using another variable we will use the concat() method of Pandas to concatenate those two lists of dictionaries

Step – 4 : Then we will use the groupby method by passing the “school_id” as a parameter to group together all the roll_no for a single school_id, we will also use the list() function on roll_no column for each of the groups by using the apply() method.

Finally we will use the reset_index() method to convert the grouped DataFrame into a normal DataFrame with school_id and the roll_id keys as columns.

Step – 5 : Here we will convert the result we got in the last step into a list of dictionaries using the to_dict() method and we need to pass another argument inside it which is “records”, now if we don’t pass any argument then it will return a dictionary of dictionaries, but the “record” parameter is used to tell it convert it to list of dictionaries.

Step – 6 : Finally we will print the result.

Below is the implementation –

Python3

import pandas as pd

lst1 = [{'roll_no': ['123445', '1212'], 'school_id': 1},
        {'roll_no': ['HA-4848231'], 'school_id': 2}]
lst2 = [{'roll_no': ['473427'], 'school_id': 2},
        {'roll_no': ['092112'], 'school_id': 5}]

lst1_df = pd.DataFrame(lst1)
lst2_df = pd.DataFrame(lst2)
lst_concat_df = pd.concat([lst1_df, lst2_df])
lst_grouped_res_df = lst_concat_df.groupby('school_id')['roll_no'].apply(list).reset_index()
result = lst_grouped_res_df.to_dict('records')
print(result)

Output – 

[{'school_id': 1, 'roll_no': ['123445', '1212']},
 {'school_id': 2, 'roll_no': ['HA-4848231', '473427']},
 {'school_id': 5, 'roll_no': ['092112']}]

Time Complexity – O(n log n) # n is the total number of roll_no in both lists.

Auxiliary Space – O(n) # n is the total number of roll_no