Copying Csv Data Into Csv Files Using Python

CSV files, the stalwarts of information exchange, can be effortlessly harnessed to extract specific data or merge insights from multiple files. In this article, we unveil five robust approaches, transforming you into a virtuoso of CSV data migration in Python. Empower your data-wrangling endeavors, mastering the art of copying and organizing information with finesse.

Copy Csv Data Into Csv Files in Python

Below are some of the ways by which we can copy data between CSV files in Python:

The CSV file used is StudentPerformance.CSV.Contents of the file are

gender,race/ethnicity,parental level of education,lunch,test preparation course,math score,reading score,writing score
female,group B,bachelor's degree,standard,none,72,72,
female,group C,some college,standard,completed,69,90,88
female,group B,master's degree,standard,none,90,95,93
male,group A,associate's degree,free/reduced,none,47,57,44
male,group C,some college,standard,none,76,78,75
female,group B,associate's degree,standard,none,71,83,78
female,group B,some college,standard,completed,88,95,92
male,group B,some college,free/reduced,none,40,43,39

Copy Csv Data Into Csv Files Using CSV Module

This fundamental approach leverages the built-in csv module, offering fine-grained control over data movement. You’ll open both files, iterate through each row in the source file, and selectively write desired data to the target file.

This code iterates through each row in student_scores2.csv, extracts elements at specific indices (0 and 2), modifies the second element by adding 10, and writes the modified row to target.csv. This showcases basic data transfer and manipulation within the loop.

Python3

import csv

with open("StudentPerformance.csv", "r") as source, open("target.csv", "w") as target:
    reader = csv.reader(source)
    writer = csv.writer(target)

    for row in reader:
        if row[2].isdigit():
            new_row = [row[0], str(int(row[2]) + 10)]
        else:
            new_row = row  # Handle rows with non-numerical data at index 2

        # Print the selected or manipulated row to the console
        print(new_row)

        # Write the row to the target CSV file
        writer.writerow(new_row)
    print('\ncsv Data copied to target csv files')

Output:

['gender', 'race/ethnicity', 'parental level of education', 'lunch', 'test preparation course', 'math score', 'reading score', 'writing score']
['female', 'group B', "bachelor's degree", 'standard', 'none', '72', '72', '']
['female', 'group C', 'some college', 'standard', 'completed', '69', '90', '88']
['female', 'group B', "master's degree", 'standard', 'none', '90', '95', '93']
['male', 'group A', "associate's degree", 'free/reduced', 'none', '47', '57', '44']
['male', 'group C', 'some college', 'standard', 'none', '76', '78', '75']
['female', 'group B', "associate's degree", 'standard', 'none', '71', '83', '78']
['female', 'group B', 'some college', 'standard', 'completed', '88', '95', '92']
['male', 'group B', 'some college', 'free/reduced', 'none', '40', '43', '39']

csv Data copied to target csv files

Python Copy Csv Data Into Csv Files Using Pandas

In this example, we are using Pandas to copy CSV data into CSV files. Here reads StudentPerformance.csv into a Pandas dataframe, filters rows based on a condition in a specific column, and writes the filtered data to target.csv without including the dataframe index.

Python3

import pandas as pd
# Read the dataset
url= "https://media.geeksforgeeks.org/wp-content/uploads/20240206150437/StudentPerformance.csv"
df = pd.read_csv(url)

# Define the threshold value for "Hours"
threshold = 90  # Replace with your desired threshold
filtered_data = df[df["reading score"] > threshold]

# Print the filtered DataFrame to the console
print(filtered_data)

# Write the filtered data to the target CSV file
filtered_data.to_csv("target.csv", index=False)

print('\n Filtered csv Data copied to target csv files')

Output:

   gender race/ethnicity parental level of education     lunch  \
2  female        group B             master's degree  standard   
6  female        group B                some college  standard   

  test preparation course  math score  reading score  writing score  
2                    none          90             95           93.0  
6               completed          88             95           92.0  

 Filtered csv Data copied to target csv files

Merging Multiple Files with glob and Pandas

Need to combine the insights from multiple CSV files? Enter the dynamic duo of glob and Pandas. glob finds files matching a pattern, while Pandas seamlessly merges them into one DataFrame for further analysis or writing to a single CSV.

Python3

import pandas as pd
import os

directory = 'Data'
all_files = os.listdir(directory)

# Get the full file paths
file_paths = [os.path.join(directory, file) for file in all_files if file.split('.')[-1]=='csv']

# Print the file paths
print('All csv files path:\n',file_paths)
combined_data = pd.concat([pd.read_csv(f) for f in file_paths])
# Reset the index
combined_data.reset_index(inplace=True, drop=True)
# Print the combined DataFrame to the console
print(combined_data)

# Save the combined data to the CSV file
combined_data.to_csv("master_data.csv", index=False)

Output:

All csv files path:
 ['Data/StudentPerformance.csv', 'Data/target.csv']
   gender race/ethnicity parental level of education         lunch  \
0  female        group B           bachelor's degree      standard   
1  female        group C                some college      standard   
2  female        group B             master's degree      standard   
3    male        group A          associate's degree  free/reduced   
4    male        group C                some college      standard   
5  female        group B          associate's degree      standard   
6  female        group B                some college      standard   
7    male        group B                some college  free/reduced   
8  female        group B             master's degree      standard   
9  female        group B                some college      standard   

  test preparation course  math score  reading score  writing score  
0                    none          72             72            NaN  
1               completed          69             90           88.0  
2                    none          90             95           93.0  
3                    none          47             57           44.0  
4                    none          76             78           75.0  
5                    none          71             83           78.0  
6               completed          88             95           92.0  
7                    none          40             43           39.0  
8                    none          90             95           93.0  
9               completed          88             95           92.0 

master_data.csv