Hadoop copyFromLocal command is used to copy the file from your local file system to the HDFS(Hadoop Distributed File System). copyFromLocal command has an optional switch –f which is used to replace the already existing file in the system, means it can be used to update that file. -f switch is similar to first delete a file and then copying it. If the file is already present in the folder then copy it into the same folder will automatically throw an error.
Syntax to copy a file from your local file system to HDFS is given below:
hdfs dfs -copyFromLocal /path 1 /path 2 .... /path n /destination
The copyFromLocal local command is similar to the -put command used in HDFS. we can also use hadoop fs as a synonym for hdfs dfs. The command can take multiple arguments where all the paths provided are of the source from where we want to copy the file except the last one which is the destination, where the file is copied. Make sure that the destination should be a directory.
Our objective is to copy the file from our local file system to HDFS. In my case, I want to copy the file name Salaries.csv which is present at /home/dikshant/Documents/hadoop_file directory.
Steps to execute copyFromLocal Command
Let’s see the current view of my Root directoey in HDFS.
Step 1: Make a directory in HDFS where you want to copy this file with the below command.
hdfs dfs -mkdir /Hadoop_File
Step 2: Use copyFromLocal command as shown below to copy it to HDFS /Hadoop_File directory.
hdfs dfs -copyFromLocal /home/dikshant/Documents/hadoop_file/Salaries.csv /Hadoop_File
Step 3: Check whether the file is copied successfully or not by moving to its directory location with below command.
hdfs dfs -ls /Hadoop_File
Overwriting or Updating the File In HDFS with -f switch
From below Image, you can observe that copyFromLocal command itself does not copy the same name file at the same location. it says that the file already exists.
To update the content of the file or to Overwrite it, you should use -f switch as shown below.
hdfs dfs -copyFromLocal -f /home/dikshant/Documents/hadoop_file/Salaries.csv /Hadoop_File
Now you can easily observe that using copyFromLocal with -f switch does not produce any error or it will easily update or modify your file in HDFS.
- Difference between Hadoop 1 and Hadoop 2
- Difference Between Hadoop 2.x vs Hadoop 3.x
- Hadoop - HDFS (Hadoop Distributed File System)
- Hadoop - Features of Hadoop Which Makes It Popular
- Hadoop - getmerge Command
- Hadoop - Python Snakebite CLI Client, Its Usage and Command References
- Introduction to Hadoop
- Hadoop - Introduction
- Introduction to Hadoop Distributed File System(HDFS)
- Hadoop | History or Evolution
- Hadoop YARN Architecture
- Hadoop Ecosystem
- Map Reduce in Hadoop
- Sum of even and odd numbers in MapReduce using Cloudera Distribution Hadoop(CDH)
- How to Execute WordCount Program in MapReduce using Cloudera Distribution Hadoop(CDH)
- Distributed Cache in Hadoop MapReduce
- Volunteer and Grid Computing | Hadoop
- Data with Hadoop
- RDMS vs Hadoop
- How Does Namenode Handles Datanode Failure in Hadoop Distributed File System?
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to email@example.com. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.