How To Read File From Amazon S3 Bucket Using Node.Js ?

The AWS Simple Storage Service (S3) is a cloud service provided by Amazon Web Services (AWS) to store your data securely. There are different approaches to storing and retrieving data from AWS S3; one of them is by using aws-sdk provided by Amazon Web Services. In this article, we will provide you with step-by-step instructions on how to use aws-sdk to upload and retrieve files from Amazon S3 securely.

What Is Amazon S3?

Amazon S3 is a simple storage service provided by Amazon Web Services. It is highly durable and provides security and scalability to any kind of data. This data is stored as objects within resources called buckets.

There are several approaches to storing and retrieving data from the S3 bucket. One of them is using the aws-sdk provided by AWS. This can be installed on any node application to communicate with the S3 bucket.

Reading A File From Amazon S3 Bucket Using Node Js

Create an AWS account and an IAM Role. Identity and Access Management (IAM) helps you secure the services you want to use by providing authentication and authorization as per your team’s requirements. Use the IAM role for accessing the S3 bucket data instead of directly using your main or root account.

Step 1: Create An Amazon S3 Bucket

Step 2: Configure The Amazon S3 Bucket

Select an AWS Region and enter a unique bucket name to create a new bucket.

Upon successful creation, you can see the list of buckets in the AWS Management Console.

Now, your bucket is ready to be integrated into the node application.

Setting Up AWS SDK On Your Node Server

Step 4: Installation And Set Up

Install aws-sdk to communicate with the AWS S3.
Install the fs library for read and write operations of the file.

npm install aws-sdk fs

Step 5: Connect Node Service To The Amazon S3 Bucket

Create an object using aws-sdk by passing accessKeyID, secretAccessKey and region in your environment variables. Make sure you’ve created an IAM role with appropriate permission.
Update the region where the bucket is stored.
Check the below code for better understanding.

Upload A File On The Amazon S3 bucket

Step 6: Prepare The File To Be Uploaded

Add a file in your project which needs to be uploaded on the S3 bucket. Store the file content in a variable using readFileSync() by passing the path of the file as parameter.
Create parameters to be passed to the S3 function to upload the file.
- Bucket: Name of the bucket in which you want to store the file.
- Key: Name of the file you want to be reflected once it’s uploaded on the S3 Bucket.
- Body: The content of the file.
In the below code, you can see we have added the file image-to-s3.png in our root folder and stored it in the fileContent variable.

Step 7: Write The Function To Upload File To Amazon S3 Bucket

Use the upload() provided by aws-sdk to upload the file by passing the parameter object created in the previous step.
Implement error handling to evaluate if the file has been uploaded successfully.

Step 8: Execution

Run the following command to execute the function.

node index.js

Step 9: Verify if file is visible on the S3 bucket.

Go to the Amazon S3 console and check if the file is visible on the Objects tab.
You can see that the file image-to-s3 is visible in the list of files.
You can view and update the permissions of the file by navigating to the Permissions tab as per below image.

There is one more file Node_to_aws_GFG.drawio.png available in the list. Let’s check how can we download that file from our Node js application.

Retrieve The File From Amazon S3 bucket

Step 10: Create a Write Stream to download the file.

Create a write stream to store the file which will be downloaded by using createWriteStream() provided by fs.
Add the path where you want this file to be downloaded.
We have created an uploads folder in our project root folder and named the new file as newFile.png. Once the file is downloaded, it should be visible in the uploads folder.

Step 11: Create The Parameter Object

Create a parameter object to download the file from the S3 bucket.
- Bucket: The name of the bucket from where you want to retrieve the file.
- Key: Name of the file in the S3 bucket you want to download.

Step 12: Execution

Run the following command to execute the function.

node index.js

The downloaded file is visible in the uploads folder.
Open the file to check the content and verify it with your original file

In this way we can retrieve a file from the AWS S3 Bucket.

AWS S3 Bucket With Node Js – FAQ’s

Is The Data Uploaded On Amazon S3 Accessible To Everyone?

You can add permissions to your file/data which you want to upload and provide access to specific users. You can also keep it public, available to everyone.

How Much Data Can You Store On Amazon S3?

As per the AWS document, the total volume of data and number of objects you can store in Amazon S3 are unlimited. Individual Amazon S3 objects can range in size from a minimum of 0 bytes to a maximum of 5 TB.