Using AWS Step Functions as Art
Jenn Bergstrom
Posted on June 29, 2023
AWS Step Functions are a service I find is undervalued in my organization. But they are a powerful tool for managing workflows and automation, especially in situations where many different tools, services, and possible flow paths need to be brought into the mix. According to the documentation here
you can incorporate AWS Lambda, AWS Batch, Amazon DynamoDB, Amazon ECS, Amazon SNS, Amazon SQS, AWS Glue, Amazon SageMaker, Amazon EMR, and even other AWS Step Functions into your step function workflow (along with many more; look at the table at the bottom of the linked page).I was feeling a bit restive last week and decided to see what else I could do with AWS Step Functions. As I was playing with the capabilities, I realized that what I was forming looked vaguely tree-like. So I decided to build on that theme.
By using a Lambda function invocation that is designed to always fail as an output generator for a step, I was able to add colors other than the blue of an unexecuted step and the green of a successful step. Specifically, I could add orange.
In my code poem below, you can see the structure of the tree. It is a combination of Pass states, Task states, and Parallel states, all working together to create a nice pine-tree-like form.
{
"Comment": "Building a tree is fun!",
"StartAt": "Time to plant a seed",
"States": {
"Time to plant a seed": {
"Type": "Pass",
"Next": "To grow a beautiful tree"
},
"To grow a beautiful tree": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:xxxxxxxxxxxx:function:Fail",
"Catch": [
{
"ErrorEquals": [
"States.TaskFailed"
],
"Next": "As tall as it can be, so wild and free"
}
],
"Next": "As tall as it can be, so wild and free"
},
"As tall as it can be, so wild and free": {
"Type": "Pass",
"Next": "My needles are green my trunk is brown"
},
"My needles are green my trunk is brown": {
"Type": "Parallel",
"Branches": [
{
"StartAt": "Im the tallest",
"States": {
"Im the tallest": {
"Type": "Succeed"
}
}
},
{
"StartAt": "Healthiest",
"States": {
"Healthiest": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:xxxxxxxxxxxx:function:Fail",
"Catch": [
{
"ErrorEquals": [
"States.TaskFailed"
],
"Next": "Pine cones are my future"
}
],
"End": true
},
"Pine cones are my future": {
"Type": "Pass",
"Result": "Love em",
"End": true
}
}
},
{
"StartAt": "Tree around",
"States": {
"Tree around": {
"Type": "Succeed"
}
}
}
],
"Next": "Whether growing up or growing down"
},
"Whether growing up or growing down": {
"Type": "Parallel",
"Next": "My trunk roots me to the ground",
"Branches": [
{
"StartAt": "I reach for the sun",
"States": {
"I reach for the sun": {
"Type": "Pass",
"End": true
}
}
},
{
"StartAt": "PC",
"States": {
"PC": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:xxxxxxxxxxxx:function:Fail",
"Catch": [
{
"ErrorEquals": [
"States.TaskFailed"
],
"Next": "Squirrels and birds"
}
],
"End": true
},
"Squirrels and birds": {
"Type": "Pass",
"Result": "Beautiful",
"End": true
}
}
},
{
"StartAt": "I burrow deep underground",
"States": {
"I burrow deep underground": {
"Type": "Pass",
"End": true
}
}
},
{
"StartAt": "!PC",
"States": {
"!PC": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-1:xxxxxxxxxxxx:function:Fail",
"Catch": [
{
"ErrorEquals": [
"States.TaskFailed"
],
"Next": "Home for many critters"
}
],
"End": true
},
"Home for many critters": {
"Type": "Pass",
"Result": "Good company",
"End": true
}
}
},
{
"StartAt": "I fill the air with scent profound",
"States": {
"I fill the air with scent profound": {
"Type": "Pass",
"End": true
}
}
}
]
},
"My trunk roots me to the ground": {
"Type": "Pass",
"End": true
}
}
}
Lessons learned from my creative endeavor:
- Unless you want to implement a large-scale parallel workload, a single uncaught failure (even a Fail state) leads to the entire step function failing. Since I wanted to use failures as a way to inject some color for the pine cones, this was a little frustrating. I was very happy when I realized that I could use a Task and catch the error off of that Task failure to get the color pop without the failed step function.
- Every name for every state in your step function has to be unique. If you try to reuse a name anywhere, the system will yell at you and refuse to go through its states until you correct the duplicated name. Generally not a problem, just something to note.
- You CAN, however, reuse external sources within your step function. If, for example, you want to be super lazy and not create unique Lambda functions that will each always fail, you can use a single one and just call it multiple times. And that works really well.
- Make sure that your State names are 100% consistent with the execution names you call. Down to the spacing and punctuation! Or your step function will fail to execute.
- You don't have to pass an input into your step function if you don't want to. Even though the default json string into the function has an input specified, it's not actually required.
- For larger, more complex artworks, consider moving into a Map Run instead of a basic Step Function. Using a Map Run allows you to set a tolerated failure for your stages, which in turn gives you more flexibility in your artistic construction of the system.
Interested in learning more about how to work with Step Functions? Check out the tutorials in the documentation. There are quite a few there. I'd love to see the art you create!
Here's my super fancy schmancy tree art:
Posted on June 29, 2023
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related