Open In App

Python | Convert list of nested dictionary into Pandas dataframe

Improve
Improve
Like Article
Like
Save
Share
Report

Given a list of the nested dictionary, write a Python program to create a Pandas dataframe using it. We can convert list of nested dictionary into Pandas DataFrame. Let’s understand the stepwise procedure to create a Pandas Dataframe using the list of nested dictionary.

Convert Nested List of Dictionary into Pandas Dataframe

Below are the methods that will be used

  • Using from_dict(orient=’index’)
  • Native Method

Nested Dictionary to Pandas DataFrame Using orient=’index’

In this example, the Panda’s library is used to create a DataFrame named df from a dictionary (countries) where each key represents a unique identifier for a country, and the values contain information about the country’s name, capital, and population. The resulting DataFrame is printed, displaying the modified country details with columns for “Country,” “Capital,” and “Population.

Python3




# importing pandas
import pandas as pd
 
countries = {
    "1": {"Country": "New Country 1",
          "Capital": "New Capital 1",
          "Population": "123,456,789"},
    "2": {"Country": "New Country 2",
          "Capital": "New Capital 2",
          "Population": "987,654,321"},
    "3": {"Country": "New Country 3",
          "Capital": "New Capital 3",
          "Population": "111,222,333"}
}
 
df = pd.DataFrame.from_dict(countries, orient='index')
print(df)


Output:

       Country         Capital    Population
1 New Country 1 New Capital 1 123,456,789
2 New Country 2 New Capital 2 987,654,321
3 New Country 3 New Capital 3 111,222,333

Nested Dictionary to Pandas DataFrame using Native Method

Below are the steps by which we can convert list of nested dictionary into Pandas DataFrame:

Step 1: Create a List of Nested Dictionary

In this step, a list of dictionaries represents students with associated exam scores and grades. The data is structured such that each dictionary includes a “Student” key containing a list of exam details, and a “Name” key indicating the student’s name.

Python3




# importing pandas
import pandas as pd
 
# List of nested dictionary initialization
list = [{
        "Student": [{"Exam": 90, "Grade": "a"},
                    {"Exam": 99, "Grade": "b"},
                    {"Exam": 97, "Grade": "c"},
                    ],
        "Name": "Paras Jain"
        },
        {
        "Student": [{"Exam": 89, "Grade": "a"},
                    {"Exam": 80, "Grade": "b"}
                    ],
        "Name": "Chunky Pandey"
        }
        ]
 
print(list)


Output:

[{'Student': [{'Exam': 90, 'Grade': 'a'}, {'Exam': 99, 'Grade': 'b'}, {'Exam': 97, 'Grade': 'c'}],
'Name': 'Paras Jain'},
{'Student': [{'Exam': 89, 'Grade': 'a'}, {'Exam': 80, 'Grade': 'b'}],
'Name': 'Chunky Pandey'}]

Step 2: Adding Dict Values to Rows

In this step, a new list named “rows” is initialized to store flattened data. The code iterates through the original list of dictionaries, extracting each student’s exam details along with their name and appends the flattened rows to create a Pandas DataFrame, which is then displayed.

Python3




# rows list initialization
rows = []
 
# appending rows
for data in list:
    data_row = data['Student']
    time = data['Name']
 
    for row in data_row:
        row['Name'] = time
        rows.append(row)
 
# using data frame
df = pd.DataFrame(rows)
 
print(df)


Output:

   Exam Grade           Name
0 90 a Paras Jain
1 99 b Paras Jain
2 97 c Paras Jain
3 89 a Chunky Pandey
4 80 b Chunky Pandey

Step 3: Pivoting DataFrame and Assigning Column Names

In this step, the Pandas pivot_table function is utilized to reshape the DataFrame. The “Name” column becomes the index, “Grade” values are transformed into separate columns (‘Maths’, ‘Physics’, ‘Chemistry’), and corresponding ‘Exam’ values fill the table. The resulting DataFrame is then displayed with the updated column names.

Python3




# using pivot_table
df = df.pivot_table(index='Name', columns=['Grade'],
                    values=['Exam']).reset_index()
# Defining columns
df.columns = ['Name', 'Maths', 'Physics', 'Chemistry']
 
# print dataframe
print(df)


Output:

            Name  Maths  Physics  Chemistry
0 Chunky Pandey 89.0 80.0 NaN
1 Paras Jain 90.0 99.0 97.0

Below is the complete code:

Python3




# importing pandas
import pandas as pd
 
# List of list of dictionary initialization
list = [
    {
        "Student": [{"Exam": 90, "Grade": "a"},
                    {"Exam": 99, "Grade": "b"},
                    {"Exam": 97, "Grade": "c"},
                    ],
        "Name": "Paras Jain"
    },
    {
        "Student": [{"Exam": 89, "Grade": "a"},
                    {"Exam": 80, "Grade": "b"}
                    ],
        "Name": "Chunky Pandey"
    }
]
 
# rows list initialization
rows = []
 
# appending rows
for data in list:
    data_row = data['Student']
    time = data['Name']
 
    for row in data_row:
        row['Name'] = time
        rows.append(row)
 
# using data frame
df = pd.DataFrame(rows)
 
# using pivot_table
df = df.pivot_table(index='Name', columns=['Grade'],
                    values=['Exam']).reset_index()
 
# Defining columns
df.columns = ['Name', 'Maths', 'Physics', 'Chemistry']
 
# print dataframe
print(df)


Output:

            Name  Maths  Physics  Chemistry
0 Chunky Pandey 89 80 NaN
1 Paras Jain 90 99 97

 



Last Updated : 06 Dec, 2023
Like Article
Save Article
Previous
Next
Share your thoughts in the comments
Similar Reads