Containers can be serverless, too, at least to some degree. I’ve decided to try Fargate and see how easy it is to deploy a web application while learning about its advantages and developer experience.

In this post, you will learn how to build a Fargate ECS cluster with an application load balancer and a web application using Python CDK code. We will build a production-grade and secure cluster that passes security scanning tools like CDK-nag and hosts a ChatBot web application that connects to OpenAI.

You can find the complete deployable GitHub repository here.

• If you find any misconfiguration or mistake, let me know.

This blog post was originally published on my website, “Ran The Builder.”

Table of Contents

1. AWS Fargate Quick Introduction

2. Building a Serverless Chatbot Web Application on Fargate

3. High Level Architecture

4. CDK Code

5. Web Application Code & Dockerfile

6. Deployment Result

7. Summary

AWS Fargate Quick Introduction

You can create a web application on Lambda, App Runner, EC2, ECS, or EKS. In this post, we will focus on ECS and containerized web applications.

To deploy a containerized web application, you need an ECS cluster and an Application load balancer that routes traffic to it.

AWS Fargate is a service that takes containerization services, either ECS or EKS, to the next level. It adds another layer of welcomed abstraction and ease of management.

AWS Fargate is a technology that you can use with Amazon ECS to run containers without having to manage servers or clusters of Amazon EC2 instances. With AWS Fargate, you no longer have to provision, configure, or scale clusters of virtual machines to run containers. — AWS documentation

You don’t need to think about EC2 machine types, manage them, install security updates, or manage their scaling up or down ; it’s all done for you. AWS claims it is a serverless service; however, in my book, it’s “almost” serverless since, as you will see in the code examples, we still need to create VPCs, and we pay for the service even when it’s idle, and no customers are accessing our cluster. However, it’s still a blessed abstraction, and I do agree with the following AWS statement:

With AWS Fargate, you can focus on building applications. You manage less — AWS documentation

You manage less but pay extra, so keep that in mind. If you are okay with managing the infrastructure and the EC2 machines under the hood, you can use ECS clusters without Fargate, which will cost you DevOps/infra team time.

Building a Serverless Chatbot Web Application on Fargate

We will write a CDK construct that will deploy a Fargate-managed ECS cluster that runs a web application with an Application load balancer routing traffic to it. I purchased a custom domain and created a certificate, but it is not mandatory.

The entire solution will be secure and audited with production-grade settings. I used CDK-nag to scan my CDK’s CloudFormation output and ensure it was not missing any important auditing or security configurations.

The web application will be a simple chatbot (yes, one of those) that empowers OpenAI. The chatbot is an ECS task container running a Python-based Streamlit application.

Streamlit, an open-source tool, allowed me to create a chatbot interface with less than 10 lines of code. It’s designed to handle user sessions and responses in the backend while providing an attractive frontend user interface.

As a side note, the CDK code can deploy any web application, not just a chatbot.

High Level Architecture

Let’s review the architecture of our Fargate solution below.

AWS Fargate will manage the Application load balancer and an ECS cluster. The ECS cluster deploys on a VPC across two AZs and creates ECS task containers (the web application container) with their security groups. The container image is uploaded to ECR.

Fargate also creates a load balancer. It has AWS WAF to protect it with an S3 bucket that logs traffic data and another bucket that tracks access to the first S3 bucket.

We also created a Route53 custom domain and DNS records that point to the ALB public address.

Let’s review the CDK code and the web application’s code.

CDK Code

There’s a lot of code, so follow my comments.

In line 19, we get a WAF object (see its definition here) and network assets. The network assets are the custom domain and its certificate. Be aware that these are optional. Their definition is here.

In lines 25–31, we build the ECR docker image for our web application container image. CDK will run Docker and expect a Dockerfile in the directory we provide. More on that later.

In lines 34–78, we define a VPC for our ECS cluster. Fargate is not “truly” serverless; we must still use VPCs like regular ECS. We also enable VPC flows and all the required permissions (yes, it’s a lot of code just for logs!).

In line 81, we define the ECS cluster and enable container insights. There are many moving parts here, so we should enable as many logs and metrics as possible to debug issues.

In lines 35–51, we define the Fargate task definition. Notice that we don’t select EC2 machine types. Instead, we define the amount of memory and virtual CPUs we require. Fargate will find the appropriate machine types according to its cost-efficient strategies (you can control them and add Spot instances capacity provider, too). In line 42, we select the ARM64 platform since I use an ARM-based laptop to build the Docker image.

In lines 84–103, we define the container used on the ECS task. We set the Docker image and enabled logging for the container. Again, another log type that we should enable to help debug the web application.

In line 103, we set the container open port to 8501, which is the default for Streamlit as our web application.

Let’s move on to the ALB and other production readiness configurations.

In lines 105–129, we create the S3 buckets for ALB and bucket access logs; we enable the security-recommended configs.

In lines 132–163, we define the Fargate ALB task. It creates an ECS service with our task definition (and container) and sets up an ALB that handles the routing to those containers. We provide it with the certificate and hosted zone created in the network assets construct. We don’t set a public IP; we enable HTTP redirect to HTTPS. Advanced features such as circuit breakers are recommended to detect failures during deployment and start a rollback to a previously stable state.

In lines 154–160, we add IAM permissions to the running process in the container. In our case, we need to access the secrets manager to fetch the OpenAI API key that was stored there.

In line 166, we enable WAF ACL on our ALB for extra security.

In lines 172–182, we tell Fargate how and when to scale its service. We provide both CPU and memory thresholds.

Lastly, line 184 enables health checks on our web application. Streamlit provides health checks endpoint support on the ‘/healthz’ path.

Web Application Code & Dockerfile

Let’s start with the Dockerfile.

When you run CDK deploy, it will eventually use this Dockerfile to create and upload an ECR image. It requires a requirements.txt file to reside in the same folder. We also require app.py of our web application.

I created it automatically by running the following command:

poetry export --without=dev --format=requirements.txt > cdk/service/docker/requirements.txt

Let’s review the file below:

As for the web application code, its mostly based on this example with added code that fetches the OpenAI API key secret from Secrets Manager.

If you look closely at line 21, you will see that I did a simple “prompt engineering” and asked the bot to answer every question with some predefined context. Obviously, this is not an optimized or good chat bot, but just a POC. The main idea was to learn how to deploy a Fargate web application.

Deployment Result

Once deployed, you will access the URL and be greeted by this chatbot interface.

I asked him if Ran the builder, prefers K8s or serverless and got a pretty reasonable answer.

Summary

In this post, we learned a bit about Fargate. We created a Chatbot on Fargate ECS and improved its security and auditing best practices using CDK. You can use this code as a template and deploy any web application.

Please review this code — have it checkd for misconfigurations and security issues. Also, make sure you add CloudWatch dashboards and alerts in the same manner I did for my serverless service (see this post).

If you find any security issues, please contact me.