Dict Of Dicts Of Dicts To Dataframe

When working with computer programming, you often encounter situations where data is organized in complex structures, like nested dictionaries. One common task is to transform these nested dictionaries into a format that’s easy to work with, such as a Pandas DataFrame. In this article, we will see how we can convert dict of dicts of dicts into Pandas DataFrame.

Dict Of Dicts Of Dicts To Dataframe in Python

Below are some examples of Dict Of Dicts Of Dicts To Pandas Dataframe in Python:

• Using pd.DataFrame Constructor
• Using pd.DataFrame.from_dict()
• Using a Custom Function
• Using pd.concat() method

Create Nested Dictionary

Here, we will create a nested dictionary that we will use in our all examples.

Python3

import pandas as pd
data = {
    'student1': {
        'math': {'score': 90},
        'science': {'score': 85},
        'history': {'score': 88}
    },
    'student2': {
        'math': {'score': 78},
        'science': {'score': 92},
        'history': {'score': 95}
    },
    'student3': {
        'math': {'score': 85},
        'science': {'score': 89},
        'history': {'score': 78}
    }
}

Python Nested Dictionaries to DataFrame Using pd.DataFrame()

In this example, a Pandas DataFrame is created where each row corresponds to a subject, and each column contains a dictionary representing the student’s score for that subject. using pd.DataFrame() constructor.

Python3

df = pd.DataFrame([{**{'Student': student},
                    **subjects} for student,
                   subjects in data.items()])
print(df)

Output:

   Student            math           science           history
0  student1  {'score': 90}  {'score': 85}  {'score': 88}
1  student2  {'score': 78}  {'score': 92}  {'score': 95}
2  student3  {'score': 85}  {'score': 89}  {'score': 78}

Nested Dictionaries to Pandas DataFrame Using pd.DataFrame.from_dict()

In this example, a pandas DataFrame is created using a list comprehension and the DataFrame.from_dict() method. Each row corresponds to a student, and columns include ‘Student’ and subject scores as dictionaries extracted from the original nested dictionary structure.

Python3

df = pd.DataFrame.from_dict([{**{'Student': student},
                              **subjects} for student,
                             subjects in data.items()])
print(df)

Output:

   Student            math           science           history
0  student1  {'score': 90}  {'score': 85}  {'score': 88}
1  student2  {'score': 78}  {'score': 92}  {'score': 95}
2  student3  {'score': 85}  {'score': 89}  {'score': 78}

Dict Of Dicts Of Dicts To Dataframe Using a Custom Function

Here, we will define a function `dict_of_dicts_to_dataframe` that converts a dictionary of nested dictionaries into a Pandas DataFrame, incorporating a ‘Student’ column, and then apply the function to the given data.

Python3

def dict_of_dicts_to_dataframe(data):
    flat_data = [{**{'Student': student}, **subjects} for student, subjects in data.items()]
    return pd.DataFrame.from_dict(flat_data)


df = dict_of_dicts_to_dataframe(data)
print(df)

Output:

   Student            math           science           history
0  student1  {'score': 90}  {'score': 85}  {'score': 88}
1  student2  {'score': 78}  {'score': 92}  {'score': 95}
2  student3  {'score': 85}  {'score': 89}  {'score': 78}

Nested Dicts to Pandas DataFrame Using pd.concat() Method

In this example, a pandas DataFrame is created using the pd.concat() function. The function pd.concat() combines a list of pandas Series along the columns (axis=1), forming a tabular structure where each column corresponds to a student, and the rows contain subject scores.

Python3

dfs = [pd.Series(subjects, name=student) for student, subjects in data.items()]
df = pd.concat(dfs, axis=1)
print(df)

Output:

               student1       student2       student3
math     {'score': 90}  {'score': 78}  {'score': 85}
science  {'score': 85}  {'score': 92}  {'score': 89}
history  {'score': 88}  {'score': 95}  {'score': 78}