Pre-requisite: AWS Kinesis
A Kinesis data stream is a collection of data records that are continually and sequentially added to over time, like a stream. It is a real-time data streaming service offered by Amazon Web Services (AWS) that enables us to collect, process, and analyze large streams of data records in real time.
Following are a few key terms related to the Kinesis data stream:
- Stream: A stream is a sequence of data records in real-time. A stream can have one or more shards.
- Shard: A shard is a unit of capacity in a stream. Each shard can support up to 1,000 PUT records per second and up to 2 MiB of data per second.
- Producer: A producer is an application that sends data records to a stream.
- Consumer: A consumer is an application that reads data records from a stream.
- Checkpoint: A checkpoint is a marker that a consumer application uses to track its progress through a stream.
- Streams API: A set of API actions that enable you to create, read, and manage streams and their shards.
- Data Retention Period: The amount of time that records are kept in a stream.
Steps to Create a Kinesis Data Stream
To create a Kinesis data stream in AWS (Amazon Web Services), you can use the AWS Management Console, the AWS Command Line Interface (CLI), or the AWS SDKs.
Here are the general steps to create a Kinesis data stream using the AWS Management Console:
Step 1: Open the Amazon Kinesis console in AWS and click on Get Started.
Step 2: Click on the “Create data stream” button.
Step 3: Enter a name for your data stream in the “Name” field.
Step 4: Choose the number of shards for your data stream in the “Number of shards” field. The number of shards determines the capacity and cost of your data stream. Click on the “Create kinesis stream” button to create the data stream.