How To Download Floder From AWS S3 CLI & UI ?

The AWS Simple Storage Service (S3) is a cloud service provided by Amazon Web Services (AWS) to store your data securely. There are different approaches to storing and retrieving data from AWS S3; one of them is by using AWS CLI provided by Amazon Web Services. In this article, we will provide you with step-by-step instructions on how to use AWS CLI download folders from Amazon S3 securely.

What Is Amazon S3?

Amazon S3 is a simple storage service provided by Amazon Web Services. It is highly durable and provides security and scalability to any kind of data. This data is stored as objects within resources called buckets.

Downloading Folder From Amazon S3

Create an AWS account and an IAM Role. Identity and Access Management (IAM) helps you secure the services you want to use by providing authentication and authorization as per your team’s requirements. Use the IAM role for accessing the S3 bucket data instead of directly using your main or root account.

Step 1: Create An Amazon S3 Bucket

Step 2: Configure The Amazon S3 Bucket

Select an AWS Region and enter a unique bucket name to create a new bucket.

Upon successful creation, you can see the list of buckets in the AWS Management Console.

Step 3: Create Folder On Your Amazon S3 bucket

Click on upload and create a new folder on the S3 bucket.
On successful creation, you’ll see the folder in your bucket’s object list.

Step 4: Add files To Your Folder

Add few files to your folder by clicking on the upload button.
You can see 3 files are added below.

Step 4: Configure AWS CLI

Install AWS CLI on your command line using the below command.

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg AWSCLIV2.pkg -target /

Verify successful installation by checking the version of aws using below command.

aws --version

Configure your IAM user on the AWS CLI using below command

aws configure

You’ll have to enter your AWS Access Key ID, AWS Secret Access Key and default region name.

Execute below command to list the buckets.

aws s3 ls

Step 5: Download Folder From Amazon S3

Download Entire S3 bucket

Execute below command to download all data on your S3 bucket. This command will download your S3 bucket data on the current location.

aws s3 sync s3://<bucket_name> .

We had one folder on the S3 bucket. Check if the folder is downloaded at the current location.
List the files under the downloaded folder to verify if all files are downloaded successfully.

Download A Specific Folder From The Amazon S3 bucket

Add a new folder to the mygfgbucket on AWS S3.

Add files to this new folder – NEW_GFG_FOLDER.

Execute below command to download NEW_GFG_FOLDER on the current location.

aws s3 sync s3://<bucket_name>/<folder_name> .

As you can see below, files from the folder NEW_GFG_FOLDER are directly downloaded on the current location.

Now let’s download this folder into a new folder instead of directly downloading the files inside.
Execute below command to download NEW_GFG_FOLDER into new_downloaded_folder.
You can name the new folder as per your requirement. It doesn’t have to match the original folder.

aws s3 sync s3://<bucket_name>/<folder_name> <download_path>

Verify if new folder is created on the current location with the provided name in the command.

Verify if the content matches with the one available on the AWS S3 bucket folder.

In this way, you can downloaded the entire S3 bucket or specific folders from the S3 bucket using AWS CLI.

Amazon S3 – FAQ’s

What Operations Can You Perform On Aws S3 Using AWS CLI?

You can perform several operations on AWS S3 using AWS CLI including:

mb: Create a S3 bucket from the AWS CLI

cp: Copy a local file or S3 object to another location locally or in S3.

mv: Move a local file or S3 object to another location locally or in S3.

presign: Generate a pre-signed URL for an Amazon S3 object

rb: Delete an empty S3 bucket.

rm: Delete an S3 object.