How To Download Floder From AWS S3 CLI & UI ?

The AWS Simple Storage Service (S3) is a cloud service provided by Amazon Web Services (AWS) to store your data securely. There are different approaches to storing and retrieving data from AWS S3; one of them is by using AWS CLI provided by Amazon Web Services. In this article, we will provide you with step-by-step instructions on how to use AWS CLI download folders from Amazon S3 securely.

What Is Amazon S3?

Amazon S3 is a simple storage service provided by Amazon Web Services. It is highly durable and provides security and scalability to any kind of data. This data is stored as objects within resources called buckets.

Downloading Folder From Amazon S3

Create an AWS account and an IAM Role. Identity and Access Management (IAM) helps you secure the services you want to use by providing authentication and authorization as per your team’s requirements. Use the IAM role for accessing the S3 bucket data instead of directly using your main or root account.

Step 1: Create An Amazon S3 Bucket

• Sign in to your Amazon console and navigate to Amazon S3. Click on Create bucket to proceed.

Step 2: Configure The Amazon S3 Bucket

• Select an AWS Region and enter a unique bucket name to create a new bucket.

• Upon successful creation, you can see the list of buckets in the AWS Management Console.

Step 3: Create Folder On Your Amazon S3 bucket

• Click on upload and create a new folder on the S3 bucket.
• On successful creation, you’ll see the folder in your bucket’s object list.

Step 4: Add files To Your Folder

• Add few files to your folder by clicking on the upload button.
• You can see 3 files are added below.

Step 4: Configure AWS CLI

• Install AWS CLI on your command line using the below command.

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg AWSCLIV2.pkg -target /

• Verify successful installation by checking the version of aws using below command.

aws --version

• Configure your IAM user on the AWS CLI using below command

aws configure

• You’ll have to enter your AWS Access Key ID, AWS Secret Access Key and default region name.

• Execute below command to list the buckets.

aws s3 ls

Step 5: Download Folder From Amazon S3

Download Entire S3 bucket

• Execute below command to download all data on your S3 bucket. This command will download your S3 bucket data on the current location.

aws s3 sync s3://<bucket_name> .

• We had one folder on the S3 bucket. Check if the folder is downloaded at the current location.
• List the files under the downloaded folder to verify if all files are downloaded successfully.

Download A Specific Folder From The Amazon S3 bucket

• Add a new folder to the mygfgbucket on AWS S3.

• Add files to this new folder – NEW_GFG_FOLDER.

• Execute below command to download NEW_GFG_FOLDER on the current location.

aws s3 sync s3://<bucket_name>/<folder_name> .

• As you can see below, files from the folder NEW_GFG_FOLDER are directly downloaded on the current location.

• Now let’s download this folder into a new folder instead of directly downloading the files inside.
• Execute below command to download NEW_GFG_FOLDER into new_downloaded_folder.
• You can name the new folder as per your requirement. It doesn’t have to match the original folder.

aws s3 sync s3://<bucket_name>/<folder_name> <download_path>

• Verify if new folder is created on the current location with the provided name in the command.

• Verify if the content matches with the one available on the AWS S3 bucket folder.

In this way, you can downloaded the entire S3 bucket or specific folders from the S3 bucket using AWS CLI.

Amazon S3 – FAQ’s

What Operations Can You Perform On Aws S3 Using AWS CLI?

You can perform several operations on AWS S3 using AWS CLI including:

• mb: Create a S3 bucket from the AWS CLI
• cp: Copy a local file or S3 object to another location locally or in S3.
• mv: Move a local file or S3 object to another location locally or in S3.
• presign: Generate a pre-signed URL for an Amazon S3 object
• rb: Delete an empty S3 bucket.
• rm: Delete an S3 object.

How Do You Delete All Files Under A Specified Directory On The Amazon S3?

By using below command you can delete all files under a specified directory:

  aws s3 rm s3://my-bucket/path --recursive

How Do I Upload File From My Local Directory To Aws S3 Using AWS CLI?

On using below command you can upload files from my local directory:

  aws s3 cp /path/to/source s3://bucket-name/ --recursive

Which Command Returns List Of All Objects In A Bucket Recursively?

The following commands helps in returning list of all objects in a bucket recursively:

 aws s3 ls s3://<bucket_name> --recursive