How To Use AWS Step Functions For Serverless Workflows?

AWS step functions process that calls the AddNumbers service, First, we will add input numbers, and then it will be decided whether the output number is greater than or less than the given constraints and the final state will be decided upon the results. While we could have one Lambda function call the other, we worry that managing all of those connections will become challenging as the AddNumbers application becomes more sophisticated. Plus, any change in the flow of the application will require changes in multiple places, and we could end up writing the same code over and over again.

What are AWS Step Functions?

Step Functions are a visual workflow service offered by AWS. It has a drag-and-drop visual console that helps the users create workflows of business-critical processes. It is mainly used in helping organizations to blend more than one AWS service while allowing them to also manage each of the microservices separately.

Features of Step Functions

1. State Machine

A state machine is a basic idea that symbolizes a workflow in AWS Step Functions. It is mainly composed of several states joined together by transitions. Each state carries out a particular action or task, such as calling a Lambda function, picking a course of action, anticipating an event, or running multiple tasks simultaneously. State machines make it simple to manage, define, and monitor complex applications by giving a visual representation of the workflow. A state machine enables the creation of scalable and resilient applications by enabling the coordination and serverless orchestration of various AWS services.

2. State

Step Functions provide different types of states, each serving a specific purpose:

Task State: Representing the execution of a specific task or action, a Task state serves as the primary means to run custom code or interact with external systems within a workflow. It can perform a range of functions, such as invoking an AWS Lambda function, executing activities in Amazon Simple Workflow Service, running an AWS Batch job, or interacting with AWS services.

Choice State: One or more conditions based on input data are evaluated in a Choice state. This allows for branching and decision-making within the workflow. Depending on the evaluated conditions, the workflow execution is directed to different paths or states.

Pass State: Performing any substantial work is not a task of a Pass state. Instead, it simply passes its input data to its output data. This state is often utilized to modify or transform the data between states. For example, it can extract specific values or change data structures.

Wait State: Before moving to the next state, a Wait state is implemented to introduce a timed delay or wait for a specific event or condition. This delay or pause in workflow execution allows for the introduction of a timed delay or the waiting for a specific event or condition.

Parallel State: A Parallel state allows multiple branches of states to be executed concurrently. This means that different paths of execution can happen at the same time, enabling parallel processing or simultaneous actions.

Fail State: An error or failure is indicated by a Fail state, which represents a terminal state in the workflow. This state occurs when the workflow execution encounters an error or failure. Information about the failure is provided in the form of an error message or cause.

Within a given workflow, AWS Step Functions employ states to encapsulate distinct actions, decisions, or behaviors. This permits the definition of the workflow’s flow and logic, empowering the construction of intricate and coordinated systems through the interconnection of various states. Within this system, each state processes its specific input data, generating output data that can then be transmitted to succeeding states. Consequently, a sequence of interconnected actions is formed, effectively representing the flow of execution within the workflow.

3. Transition

Defining the flow and progression of workflows in a state machine relies heavily on transitions within Step functions. We can say that these transitions determine the movement between states based on specific conditions or events.

Describing rules or conditions, transitions explain the evaluation of the output of a state and determine the next state to transition to. The evaluation of these conditions considers factors like a combination of input or output data of the state and enables the workflow execution to progress to a different state.

Transitions are established by evaluating the conditions specified in the state definition in a Choice state, for example. Each condition is associated with a potential next state, and the transition occurs to the state where the condition evaluates to true.

We can build resilient and adaptable workflows in AWS Step Functions using transitions that control execution flow based on earlier state results. Dynamic and flexible workflows that implement branching, decision-making, looping, and error handling are enabled by transitions.

4. Tasks

Tasks in AWS Step Functions are the units of work performed within a state machine. You can execute them in a Task state, one of the available state types in Step Functions. With tasks, you can invoke AWS Lambda functions, interact with different AWS services, execute activities in Amazon Simple Workflow Service (SWF), or run AWS Batch jobs.

Before moving on to the next state, the state machine will call the specified task and wait for it to finish. This task can be either an AWS Lambda function or an activity in SWF, and its ARN (Amazon Resource Name) must be specified when defining the Task state.

AWS Step functions examples

In this example, the algorithm takes two input numbers and add them, then :

• If the result is larger than 500 the user should receive a SMS with a random number between 500-1000.

• If it’s less, then the random number in the SMS should be from 0-500.

• Otherwise, it will fail.

Steps to Implement AWS Step functions

Step 1. Create an IAM Role

First, we need to create an IAM Role with the permissions that allow Step functions to get access to lambda.

1. Create a role under the IAM tab from the AWS Management Console.

Step 2: Select AWS Service under the Trusted Entity Type, under Use Case make sure to select Step Functions in the dropdown.

Step 3: Under the Add Permissions, leave it as default(Make sure AWSLambdaRole is added in the permissions policies). Then click on next to proceed.

Step 4: Under the Name, Review, and create section, enter the name of the Role according to you.

the Finally, click on create the role.

Now, before creating the step function, we will need to create the lambda functions since we need the ARN of the lambda functions in our step function code.

Steps To Create AWS Lambda Functions

Step 1: Create the Lambda Functions
We will create the lambda functions which will be used to connect to our state machine which will be created further. Here we will have 4 Lambda functions in this example:

• add_numbers – It takes { key1 : X, key2 : Y} as input and returns { number : Z }
• greater_than – It returns { final_number : rand(500,1000) }
• less_than – It returns { final_number : rand(0,500) }
• final_state – It takes { final_number : C }

Step 2: Click on Create Function under Lambda Console. Select Author from Scratch, and next give add_numbers as the name of your function, under Runtime select Python from the dropdown (or any other preferred Runtime language of your choice). Then click on Create Function.

Step 3: When the function is created, hover down to the Code section and replace the Function Code with the following code and then click on Deploy.

Python3

import json

def lambda_handler(event, context):
   number_1 = int(event['key1'])
   number_2 = int(event['key2'])
   return {"number" : abs(number_1 + number_2)}

After this, we need to make 3 more Lambda Functions named as greater_than, less_than, and final_state using the above-mentioned steps.

Below are the three different codes for three different functions:

Greater_than

Python3

import json
import random

def lambda_handler(event,context):
final_number = random.randint(500,1000)
   return {"final_number" : final_number}

Less_than

Python3

import json
import random

def lambda_handler(event,context):
final_number = random.randint(0,500)
   return {"final_number" : final_number}

Final_state

Python3

import json
import boto3
import subprocess
import os
import sys
import subprocess

# pip install custom package to /tmp/ and add to path
subprocess.call('pip install inflect -t /tmp/ --no-cache-dir'.split(), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
sys.path.insert(1, '/tmp/')
import inflect
def lambda_handler(event,context):
inflect_engine = inflect.engine()
   number = inflect_engine.number_to_words(event['final_number'])
   message = {"DelMeMessage": "The StepFunctions Result is %r" %number}
   client = boto3.client('sns')
   response = client.publish(
TargetArn="SNS-TOPIC-ARN",
Message=json.dumps({'default': json.dumps(message)}),
MessageStructure='json'
   )

After completing these steps, all our functions will be created successfully.

Steps to Create SNS Topic

Step 1. Create an SNS Topic

Go to Create Topic under SNS in the AWS Management console. Under the type select Standard, also give a suitable name for your topic. Then click on Create Topic.

Step 2: Then we need to create a Subscription for the SNS Topic, under protocol select Email and in the endpoint add the E-mail ID as per your need. Finally, click on create subscription.

Also, make sure to add the target ARN in the state_machine function code.

Step 3: Create a State Machine and Serverless Workflow. Navigate to the Step Functions under the AWS Management Console.

Step 4: Click on State Machine and then click Create State Machine.

Step 5: Select Write your Workflow in Code and type as Standard.

Step 7: Under the definition window, change the code with Amazon States Language. This state machine uses a series of Task states to add numbers. Then, a Choice state is used to determine if the returned number is greater than , less than, or equal to a specified number. Then there is also a final state to conclude the task. Make sure under the resource in the code, change the ARN with the necessary ARN of the lambda functions. Then click on next.

Javascript

{
"Comment": "A simple Step Functions state machine",
"StartAt": "AddNumbers",
"States": {
"AddNumbers": {
"Type": "Task",
"Resource": "ARN:OF:add-numbers",
"Next": "AnswerState"
   },
"AnswerState": {
"Type" : "Choice",
"Choices": [
       {
"Variable": "$.number",
"NumericGreaterThan": 500,
"Next": "GreateThan"
       },
       {
"Variable": "$.number",
"NumericLessThan": 500,
"Next": "LessThan"
       }
     ],
"Default": "EqualTo"
   },

"GreateThan": {
"Type" : "Task",
"Resource": "ARN:OF:greater-than-lambda",
"Next": "FinalState"
   },

"LessThan": {
"Type" : "Task",
"Resource": "ARN:OF:less-than-lambda",
"Next": "FinalState"
   },

"EqualTo": {
"Type": "Fail",
"Cause": "No Matches!"
   },

"FinalState": {
"Type": "Task",
"Resource": "ARN:OF:final-state-lambda",
"End": true
   }
 }
}

Step 8: Give a suitable name to the State Machine, under the permission click on Choose an existing role, and add the role which we made in the starting.

Step 9: Finally click on Create State Machine.

Step 10. Execution of Workflow. Click on Start Execution.

2. A new dialog box appears in the screen, in the input section enter the following content. and then finally click on start execution.

Javascript

{
 "key1" : 100,
 "key2" : 300
}

Once the execution is completed, we can see the visual workflow pane and many other things as depicted in the below images.

Finally, we can check the Email which we provided as an endpoint in SNS Topic Subscription creation, you would have received a random number if the execution is successful.

Benefits

Using Step Functions for building serverless workflows offers several benefits:

1. Orchestration and Coordination: Providing a visual and intuitive method for orchestrating and coordinating multiple AWS services, Step Functions allow for the creation of cohesive workflows. With the ability to define various states and transitions, it simplifies the management and maintenance of distributed systems.
2. Scalability and Resilience: High volumes of concurrent workflow executions are easily handled by Step Functions with its automatic scaling. Efficient management of individual states ensures reliable and efficient workflow processing. Additionally, built-in error handling and retries are provided, improving the resilience of your workflows.
3. Serverless Architecture: By utilizing the serverless computing model, Step Functions eliminate the burdensome tasks of infrastructure provisioning and management. With Step Functions, you are solely responsible for paying for the actual usage of both the functions and the services they rely on, resulting in reduced operational expenses and overhead.
4. Fault Tolerance: With Step Functions, you’ll find a wonderful fault tolerance feature. For any workflow hiccups, Step Functions will effortlessly manage retries based on your custom retry policy. You’re free to set the intervals, backoff tactics, and max attempts for each stage, guaranteeing that fleeting mishaps are expertly resolved. All thanks to this innate fault tolerance mechanic, you can bask in the unwavering reliability and toughness of your workflows, and wave goodbye to concerns of data loss or workflow disturbance.
5. Visibility and Monitoring: You have the option to monitor workflow execution progress, status, and performance using the AWS Management Console, SDKs, or API, thanks to Step Functions. This service provides comprehensive visibility into your workflows’ execution.

How Step Functions can be Integrated with Other AWS Services 

Integrating seamlessly with a range of AWS services such as Amazon SNS, AWS Lambda, and Amazon DynamoDB, AWS Step Functions allows you to take full advantage of these services’ functionalities within your workflow. Below, you will find a quick summary of how Step Functions integrates with these services:

1. AWS Lambda: As part of its workflow execution, the Step Function has the ability to directly invoke AWS Lambda functions. This integration enables the execution of custom code and the performance of complex computations or business logic within your workflows. By utilizing the Task state, you can pass input data to the Lambda function and receive the output data seamlessly.
2. Amazon SNS: Using the Task state, Step Functions have the capability to send messages to specified SNS topics and trigger SNS notifications. This allows for smooth integration of event-driven workflows with external systems and the activation of actions based on SNS notifications.
3. Amazon DynamoDB: A fully managed NoSQL database service called Amazon DynamoDB can be interacted with by Step Functions. With the Task state, you are able to perform operations like reading, writing, updating, or deleting data from DynamoDB tables within your workflows. It is possible to incorporate database operations into your workflows and develop data-driven applications with this integration.
4. Stepping it up, Step Functions demonstrate their ability to work alongside other AWS services. Integrations with Amazon SQS, AWS Glue, AWS Batch, as well as AWS Step Functions Dataflow, are just a preview of the possibilities. By linking with these services, Step Functions pave the way for the efficient execution of intricate tasks using a combination of serverless workflows and AWS services.

Conclusion

We have successfully implemented Serverless Workflow using AWS Step Functions. Don’t forget to delete all the services which you have created to avoid incurring any charges.