How to Upload Files to AWS S3 using CLI?

The AWS Simple Storage Service (S3) from Amazon Web Services (AWS) is a widely used cloud storage option for securely overseeing and storing your data. The AWS Command Line Interface (CLI) offers a robust and effective means of engaging with various AWS services, including S3. Within this article, we will provide you with step-by-step instructions on how to utilize the AWS CLI to upload files to AWS S3.

What is AWS?

Amazon provides Amazon Web Services (AWS), a highly utilized and all-encompassing platform and infrastructure for cloud computing. With AWS, individuals, businesses, and organizations have access to an extensive range of cloud services, such as computing power, storage, databases, machine learning, analytics, content delivery, and more. These services can be easily accessed and utilized via the internet, offering a flexible, scalable, and cost-efficient solution to meet various computing requirements.

What is S3?

Amazon S3, also known as Simple Storage Service, is a key feature provided by Amazon Web Services (AWS) that functions as a flexible, expandable, and reliable cloud-based solution for storing and accessing data.

Its main function is to securely store and retrieve various types of content, including documents, images, videos, application data, and backups. This service is crucial for a wide range of cloud-based applications and services as it forms the foundation for data organization, with objects being stored and organized in buckets.

What is AWS CLI?

AWS CLI (Amazon Web Services Command Line Interface) is a command-line tool provided by Amazon Web Services to interact with and manage AWS services. It allows users to perform various operations such as creating and managing AWS resources, configuring services, and automating tasks using scripts. It’s a powerful tool for developers, administrators, and anyone who needs to work with AWS resources via the command line.

So, like shown in the above image, we can just type in some of the commands from our terminal and just access the information from our S3 bucket of AWS account without the need to login to AWS website and do things manually. This is best used for faster development process.

Tools Required

• AWS CLI configured
• AWS account
• Python Installed (AWS CLI depends on it)

How to Install AWS CLI?

1. Visit the AWS Website and select the Operating System you have. ex – Windows.
2. Click on the relevant download link or copy-paste the command to download and install the Command Line Interface on your system.

Configuring CLI

After the CLI is installed, it needs to be configured with your AWS account credentials to use it. Follow the below steps to configure it.

• Login and open your AWS account console.

• Click on your account username present at the top right corner and select the “Security credentials” option.
• Scroll down to the “Access keys” section and create on “Create access key”.
• After creating the key on the next page, copy the Access key and Secret Access Key, which we will be using later.
• Then, open your terminal and enter the following command

aws configure

• It will prompt you to enter the Access Key and Secret Access Key and enter those previously copied keys respectively.
• For the Default region name enter us-east-1 and for the Output format, just press enter to keep the default value. Your CLI is now configured!

Upload Files to S3 Bucket using CLI

Step 1: Go to AWS Console, search for S3 and open it.

Step 2:

• Create a new S3 Bucket (Skip this step if you already have one)
• Type in your bucket name and choose your AWS region, scroll down to the button and click on “Create bucket”

Step 3: Execute the below command from the terminal to upload files to the S3 bucket.

aws s3 cp /local/file/path s3://bucket-name

Replace /local/file/path with the file or folder you want to upload and bucket-name with your s3 bucket name.

After executing the above command, the file will get uploaded to S3 Bucket.

How to Upload Multiple Files to AWS S3 Using the CLI?

To upload multiple files to an Amazon S3 bucket using AWS CLI, use the following command from your terminal.

aws s3 cp my-files s3://my-bucket/ --recursive

• my-files is the local directory or folder containing the files you want to upload.
• The –recursive flag is used to upload all files and subdirectories within the specified directory.

How to Upload a Directory to AWS S3 Using the CLI?

To upload an entire directory to an Amazon S3 bucket using AWS CLI, use the use the following command from your terminal.

aws s3 cp dir-name s3://my-bucket/ --recursive

View Files present in S3 Bucket

To list all the files and folders present in your S3 bucket, you can execute the following command in your terminal.

aws s3 ls s3://bucket-name

To show the content present in the file, you can use the following command.

aws s3 cp s3://bucket-name/filename.txt -

Upload everything to S3

To upload every file, folder and files present in each folder of the present working directory, use the following command.

aws s3 cp . s3://bucket-name --recursive

This copies every file, folder, and sub-folder present in the current directory to the S3 bucket recursively.

Common use cases for uploading files to AWS S3 using the CLI

• Backing up Data: Using the AWS CLI we you securely store and automate the backup of your important data in Amazon S3, ensuring data backup and recovery.
• Sharing Files with Others: Upload files to S3 with the CLI, and generate temporary access links to easily share documents and media with external users.
• Serving Static Content for Websites: You can host static assets like HTML, CSS, and media files on S3, using the CLI to provide efficient content delivery for websites.
• Hosting Images and Videos: Using the AWS CLI you can upload images and videos to S3 for reliable storage and seamless access by the end user.

Frequently Asked Questions (FAQ’s)

Q1 : Is Amazon S3 a secure storage option for sensitive data?

Amazon S3 provides robust security features, including access control lists, bucket policies, and server-side encryption. It’s a secure option for storing sensitive data.

Q2 : How does Amazon S3 pricing work?

Amazon S3 pricing is based on factors such as storage volume, data transfer. It offers a pay-as-you-go model, and you can find detailed pricing information on the AWS website.

Q3: How can I track and monitor file uploads to S3?

AWS provides tools like Amazon CloudWatch and AWS CloudTrail for tracking and monitoring S3 activities, including file uploads. These tools can help you gain insights into usage and access patterns.

Q4: Can I make my S3 files and buckets publicly accessible for website hosting?

Yes, you can configure S3 buckets and objects to be publicly accessible if you want to host a website.

Conclusion

To summarize, Amazon Web Services (AWS) and its Simple Storage Service (S3) provide a strong and dependable option for cloud storage and data handling. Using the AWS Command Line Interface (CLI) simplifies the tasks of uploading, managing, and retrieving data in AWS S3. This article presented a detailed guide on configuring the AWS CLI, establishing an S3 bucket, uploading files and directories to it, and viewing the contents of your S3 bucket.