CDK Patterns at 20! Let's Walk Through all 20 Serverless Patterns for AWS
Matt Coulter
Posted on July 14, 2020
There are now 20 fully deployable serverless architecture patterns at cdkpatterns.com, all built with AWS CDK TypeScript/Python and all including the vanilla CloudFormation Template.
This seemed like a good time to stop and reflect on what has been built showcasing the use cases for each pattern in one article. I will simply walk through them alphabetically but if you want to view them by AWS Component used you can use the component filter or if you want to view them by the AWS Well Architected pillar they relate to you can use the well architected pattern matcher
For the patterns that exist in Jeremy Dalys new Serverless Reference Architecture collection I will also include a link to there for users of other frameworks than AWS CDK (SAM, Serverless, Pulumi etc).
Polly
This is a pattern that integrates the Amazon Polly service into an AWS Lambda Function so that you can synthesise text into speech using a serverless stack. It also integrates with Amazon Translate to allow you to choose the language for the spoken text.
When To Use
If you are publishing content you could use this to support international readers/listeners in their native language
Alternatives
I am sure external vendors exist, this is the only serverless AWS native solution I know.
S3 Deploy
This is a simple pattern for taking a website built in Angular or React and deploying it to an S3 bucket for a super cheap and scalable hosting solution. The construct bundled with the pattern is flexible and allows you to expand on the bundled pattern by adding CloudFront, a custom domain name and SSL.
Deconstructing S3 Deploy
When To Use
You need to deploy a website on AWS through CDK
Alternatives
AWS Amplify
The Big Fan
This is a pattern from Heitor Lessa that allows you to fan out your messages to multiple different consumers. The beauty is that you are filtering the messages based on properties in the message so consumers only receive messages relevant to them saving compute time / cost.
Heitor Lessa at re:Invent
When To Use
If you are building an asynchronous event driven flow that needs to fan out to multiple consumers based on different message types
Alternatives
Amazon EventBridge - look at The EventBridge ATM
The CloudWatch Dashboard
When you start building serverless applications the big piece that feels missing compared to a traditional application is observability. You have all these small pieces running asynchronously and when something goes wrong it can feel like logs are your only friend. This pattern shows you have to generate the metrics needed to build a custom dashboard and add Alerts publishing to SNS for a serverless stack containing API Gateway HTTP API, Lambda and DynamoDB
When To Use
Anyone building production serverless applications should be using CloudWatch dashboards and alerts to understand their application state
Alternatives
You can use other third party tools but this is the only AWS option
The Destined Lambda
This pattern combines Lambda Destinations with Amazon EventBridge to show you that with EventBridge rules you can decouple your components in an event driven architecture and by combining it with lambda destinations you can strip out EventBridge specific code from your lambda functions themselves and decouple further.
Deconstructing The Destined Lambda
When To Use
Lambda destinations are a brilliant way of removing boilerplate from your orchestration logic but they only work when executed asynchronously so if your flow is synchronous then this is not for you yet.
They also forward the full context of an event which means that when you send an error to a Dead Letter Queue through destinations you get everything including the event that caused the error so you have everything you need to implement automatic retry or error hospital functionality.
Alternatives
If you don't want to use destinations, you can still use Amazon EventBridge just with more logic in your Lambda Function or you could use AWS Step Functions.
The Dynamo Streamer
This is a variation on The Simple Webservice pattern that has been created by Eric Johnson only instead of a lambda being connected to the apigateway, the dynamodb is connected directly and the api gateway uses templates to transform the incoming message to insert the data. A lambda then listens for events coming from dynamodb streams and can be used to do data transforms after insertion meaning if an error occurs you lose no data.
"When thinking about #Serverless architectures, consider how much of your processing can happen AFTER the data is saved. Thinking asynchronously can lead to greater resiliency and often, less code."
When To Use
I would use a direct API Gateway integration with DynamoDB as much as you can. Unfortunately you cannot use VTL yet with the new HTTP API so you need to use the more expensive REST API but removing a Lambda Function from the flow reduces latency and execution costs. If cold starts are an issue for your use case this eliminates them from the flow.
Alternatives
The simple webservice pattern
The EFS Lambda
This is a pattern from Danilo Poccia that attaches an EFS file system to your lambda function to give it expandable, persistent storage. Having this level of storage in a Lambda Function opens the door to many new possibilities (multiple functions can even use the same file system).
When To Use
If you need to share state between multiple Lambda Functions or if you need the Lambda to be able to access something bigger than the max in memory allows
Alternatives
Convert from using a Lambda Function to using an AWS Fargate container of you could go all the way to EC2
The EventBridge ATM
This is an example CDK stack to deploy the code from this blogpost by James Beswick
In this example, a banking application for automated teller machine (ATM) produces events about transactions. It sends the events to EventBridge, which then uses rules defined by the application to route accordingly. There are three downstream services consuming a subset of these events.
When To Use
If you need to build an asynchronous flow where the contents of the message dictate which consumer(s) should receive the message
Alternatives
You could use The Big Fan pattern with SNS
The EventBridge Circuit Breaker
This is an example CDK stack to deploy my own interpretation of The Circuit Breaker stack described by Jeremy Daly in this article and by Martin Fowler all the way back in 2014 here
In this example, we have a lambda behind an API gateway that is supposed to integrate with an external webservice (www.google.com). The problem is that Google is down and it takes 10 seconds for your lambda to return that error. You pay for every ms of execution with Lambda so this is bad if lots of consumers hit your service.
Don't worry, we have integrated a circuit breaker into this system. When a call to google fails an error event is pushed to EventBridge where it is routed to a lambda that inserts a record into DynamoDB with a 60 second lifespan.
When a consumer calls our lambda we check if there have been 3 failure events in the last 60 seconds and if so we fail immediately, this saves over 9 seconds of execution costs. As the error events expire after 60 seconds our failure events should gradually drop below 3 where we call the service again and check status.
When To Use
If you integrate with external webservices you should probably integrate a circuit breaker so that your system is more resilient in the event of that service going down
Alternatives
AWS ElastiCache could be used to cache successful calls/responses but this won't help if you haven't called before and the service is down.
The EventBridge ETL
This is an example stack showing how you can use EventBridge to orchestrate events through an ETL process. This pattern was insired by Vyas Sarangapani and Hervé Nivon. The CDK Patterns README details their implementations.
Note - This is a learning pattern, if I was implementing this in a production system I would make a couple of changes:
- Use batching of events where possible to reduce costs
- KMS Encryption of sensitive data in events
- SQS between EventBridge and the Lambdas for resiliency
- Add in error events to EventBridge and error event rules
Deconstructing The EventBridge ETL
When To Use
This pattern is more a demonstration of what is possible with EventBridge in terms of building complex decoupled flows. I would use this to learn EventBridge.
Alternatives
AWS Glue for complex flows or you could use a Lambda Function streaming larger files from S3.
The Lambda Power Tuner
This is an AWS CDK project that deploys the awesome AWS Lambda Power Tuning project by Alex Casalboni.
AWS Lambda Power Tuning is an AWS Step Functions state machine that helps you optimize your Lambda functions in a data-driven way.
When To Use
If you are using Lambda Functions in your application you will want to use the Lambda Power Tuning step function to optimize performance vs cost.
Alternatives
You could tune it by hand with enough effort
The Lambda Trilogy
This is three patterns in one representing the three states of AWS Lambda.
The three states of AWS Lambda are something that has been discussed by many serverless heroes since their invention. This is probably the most controversial subject in all of serverless so I am not going to tell you which of the three is the best because like everything you need to adapt the right implementation to fit your context!
This pattern includes examples from Paul Swail, Yan Cui, Jeremy Daly and others
Deconstructing The Lambda Trilogy
When To Use
If you are using Lambda Functions in your application you will want to use one of the three states
The RDS Proxy
This is a project that has been configured with a MySQL RDS DB, an RDS Proxy, a Lambda Function to run queries and an API Gateway HTTP API to trigger the lambda function. The RDS Proxy is designed to bridge the gap between the massively scalable, serverless Lambda function and the less scalable RDS Instance
When To Use
If you need to have a synchronous flow where a serverless component communicates with a non-serverless RDS Instance
Alternatives
If you can go asynchronous you can use the scalable webhook
The SAGA Step Function
This is a pattern that demonstrates how use handle distributed transactions within a serverless system. It also demonstrates single table DynamoDB design inspired by Alex Debrie
Deconstructing The Saga Step Function
When To Use
If you need to make multiple smaller transactions pass or fail in one atomic unit like in the example where you are booking a holiday and need flights and hotel booked together.
Alternatives
This is a SAGA design pattern implemented with Step Functions, you could try implementing it with other components but it will probably be less efficient.
The Scalable Webhook
This is a pattern from Jeremy Daly for integrating serverless resources like Lambda with non serverless ones like RDS in an asynchronous flow.
Deconstructing The Scalable Webhook
When To Use
If you cannot predict your volume of traffic and you can use an asynchronous process to process the events
Alternatives
The RDS Proxy but the scalable webhook can be applied to any resource, not just RDS.
The Simple GraphQL Service
This is a simple MVP implementation of AWS AppSync to use GraphQL to query a Lambda resolver and a DynamoDB table.
When To Use
GraphQL allows your consumer to specify what pieces of information they need from your API which is really powerful. It also is helpful as an extra unifying wrapper on top of disparate APIs
Alternatives
You could deploy Apollo in a Lambda Function
The Simple Webservice
The most basic pattern on cdkpatterns, the start of most peoples serverless journey. This is an API Gateway HTTP API, connected to a Lambda Function that queries DynamoDB
When To Use
If you need to expose your DynamoDB table (or other serverless component) with a RESTful endpoint.
Alternatives
The Simple GraphQL service and The Dynamo Streamer
The State Machine
This is a simple State Machine implementation using Step Functions to implement the pattern of the same name described by Jeremy Daly
When To Use
If you have complex orchestration logic between multiple Lambda Functions
Alternatives
It is possible to also build event driven flows with EventBridge and Rules. You get a lot for free with Step Functions though like complete insights into every step executed
The XRay Tracer
Learn about using AWS X-Ray for tracing events through your system. This pattern has X-Ray enabled on API Gateway, Lambda, DynamoDB, External HTTP calls, SNS and SQS
When To Use
If you use serverless components in your system you should be using a tracing system to understand its behaviour
Alternatives
You can instrument some third party alternatives like Thundra
Posted on July 14, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
