How To Copy Files Recursively Using AWS S3 CP?

S3 (Simple Storage Service) is a storage service provided by AWS to store any kind of data in File/Folder format. In this article, we will see how easily we can access S3 to store our data in buckets which are containers to store your data.

The AWS Command Line Interface (AWS CLI) is an open-source tool that allows you to interact with AWS services using syntax in your shell. It provides you with the ability to quickly and easily access the same functionality as the browser-based AWS Management Console from the comfort of your terminal program.

Prerequisites

• AWS Account: Before starting this tutorial, you must have an AWS account. Read this article in case you don’t have an AWS account.
• AWS CLI: You should have AWS CLI set in your local machine with access. Refer to this article if you want to know how to set up AWS CLI.
• S3 Bucket: You should have a bucket set up in your S3. Refer to this article if case you haven’t made it yet.

Step-By-Step Guide To Copy Multiple Files into S3 Bucket

Step 1: Login into AWS -> https://aws.amazon.com/(Amazon Web Services (AWS) – Free Tier Account Set up)

Step 2: After signing in you will land on the AWS Management Console page and search for S3 as shown below.

Step 3: From the sidebar go to Buckets. A bucket is a container to store the objects (Files and Folders).

Step 4: On the buckets page all of your buckets will be listed. We will copy multiples files into ‘gfg-s3-test-bucket‘.

Step 5: To add multiple files/folders in your bucket just add ‘–recursive’ at the end of the single copy syntax. This will upload all the files from the given path.

Syntax :

aws s3 cp 'your local file path' s3://bucket-name --recursive

Use file path as dot ( . ) if you to copy from present directory else ‘ %USERPROFILE%\path-to-folder ‘

Step 6: Following is output logs that shows you have successfully copied files from you directory to S3 bucket.

Conclusion

In this article, we explored AWS CLI commands to perform copy operations on the AWS S3 bucket. CLI makes it easy to perform tasks using simple commands and arguments.

S3 Bucket Copy Files – FAQ’s

How do I exclude specific files or folders from the copy operation?

The –exclude flag allows you to specify filenames or paths to exclude from the copy process.

Can I copy files from one S3 bucket to another?

The aws s3 cp command works itself for bucket to bucket transfers.

How can I verify if the copy operation was successful?

You can use the aws s3 ls command to list the objects in your bucket.

What happens if the internet connection is interrupted during the copy process?

By using aws s3 sync command you can detect and resume interrupted transfers under specific conditions.