Hadoop copyFromLocal command is used to copy the file from your local file system to the HDFS(Hadoop Distributed File System). copyFromLocal command has an optional switch –f which is used to replace the already existing file in the system, means it can be used to update that file. -f switch is similar to first delete a file and then copying it. If the file is already present in the folder then copy it into the same folder will automatically throw an error.
Syntax to copy a file from your local file system to HDFS is given below:
hdfs dfs -copyFromLocal /path 1 /path 2 .... /path n /destination
The copyFromLocal local command is similar to the -put command used in HDFS. we can also use hadoop fs as a synonym for hdfs dfs. The command can take multiple arguments where all the paths provided are of the source from where we want to copy the file except the last one which is the destination, where the file is copied. Make sure that the destination should be a directory.
Our objective is to copy the file from our local file system to HDFS. In my case, I want to copy the file name Salaries.csv which is present at /home/dikshant/Documents/hadoop_file directory.
Steps to execute copyFromLocal Command
Let’s see the current view of my Root directoey in HDFS.
Step 1: Make a directory in HDFS where you want to copy this file with the below command.
hdfs dfs -mkdir /Hadoop_File
Step 2: Use copyFromLocal command as shown below to copy it to HDFS /Hadoop_File directory.
hdfs dfs -copyFromLocal /home/dikshant/Documents/hadoop_file/Salaries.csv /Hadoop_File
Step 3: Check whether the file is copied successfully or not by moving to its directory location with below command.
hdfs dfs -ls /Hadoop_File
Overwriting or Updating the File In HDFS with -f switch
From below Image, you can observe that copyFromLocal command itself does not copy the same name file at the same location. it says that the file already exists.
To update the content of the file or to Overwrite it, you should use -f switch as shown below.
hdfs dfs -copyFromLocal -f /home/dikshant/Documents/hadoop_file/Salaries.csv /Hadoop_File
Now you can easily observe that using copyFromLocal with -f switch does not produce any error or it will easily update or modify your file in HDFS.
- Difference Between Hadoop 2.x vs Hadoop 3.x
- Hadoop - HDFS (Hadoop Distributed File System)
- Introduction to Hadoop
- Hadoop - Introduction
- Hadoop | History or Evolution
- Hadoop Ecosystem
- Map Reduce in Hadoop
- Data with Hadoop
- Difference Between RDBMS and Hadoop
- Basics of Hadoop Cluster
- Hadoop - Different Modes of Operation
- Hadoop - Pros and Cons
- Hadoop - Architecture
- Hadoop - Rack and Rack Awareness
- Hadoop - Cluster, Properties and its Types
- Hadoop - File Permission and ACL(Access Control List)
- Hadoop - Schedulers and Types of Schedulers
- Hadoop - A Solution For Big Data
- Top 10 Hadoop Analytics Tools For Big Data
- Hadoop - Daemons and Their Features
If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to firstname.lastname@example.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.
Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.