KISS with Docker Compose

KISS - Keep It Simple Stupid - is a design mantra we live by in software engineering. But too often we fail to apply it to AWS infrastructure. I struggle with this because I want to do the right thing. The “right thing” is to follow AWS guidance. Guidance which sells you on replacing open source software with AWS native versions. Yet the more AWS services you use, the harder it is to manage all aspects of those services - necessitating yet more services. It becomes necessary to undergo extensive training to properly understand these services and measure risk, and the complexity of important tasks like disaster recovery balloons.

KISS with Docker Compose takes a different approach. It takes ~~Bikini Bottom and pushes it somewhere else~~ a Docker Compose app from your development machine and copies it into an EC2 instance with ports 80 and 443 open (this, and everything else, is configurable). It then pulls your Docker images and starts Docker-Compose.

The result? Stupid simple infrastructure which works the same way in the cloud as it does on your development machine.

I’ve created an AWS CDK package and published it to Github to help you try this approach: https://github.com/Gregory-Ledray/kiss-docker-compose-on-aws and give a tutorial on how it works below under “Try It Out”.

Evaluating KISS Docker Compose

Evaluation needs to be split into three parts:

Why Bother?
Can Docker Compose meet your requirements on an infinitely powerful and reliable machine?
Is an EC2 Instance sufficient to handle the Docker Compose app’s needs?

Why Bother?

Cheap: All code runs on one small EC2. Compare this to running one small EC2 for your code + a small RDS instance for your database + a NAT gateway. It’s also far less complex and labor intensive to set up and maintain.
Simple: It runs the same way on your machine as it runs in the cloud.
Fast: Works by default.

Can Docker Compose Meet Your Requirements?

Docker supports running Compose on a single server for production, so probably yes. It is production ready with support for health checks, restarting, secrets, environment configuration, networking, GPUs, etc. If the EC2 instance is infinitely powerful and reliable then Docker Compose is probably enough for you. But therein lies the rub.

Is an EC2 Instance Sufficient?

Therein lies the issues, and some good reasons to not use this approach:

Is the 99.99% uptime of an EC2 instance sufficient?
Is your application going to exceed the RAM of the EC2 instance? We are able to provide swap space to help, but that only goes so far.
Is your application going to exceed the file storage of the EC2 instance? By default, we only launch the EC2 instance with 8GB of storage, although this is fully configurable.
Is your application going to run out of CPU? This configuration doesn’t allow for auto scaling.
Disaster recovery requires backups. Doing this safely means stopping the instance, creating a backup of the file storage, and then restarting the instance, which means downtime.
Deployments are simple but cause downtime. Whenever you restart the instance it re-pulls all the Docker Images used by Docker Compose, so to deploy new images you first push those images to the repository and then restart the VM.

When thinking about drawbacks though, we need to consider how solvable these issues are and discount those which are insolvable:

No single instance of ECS or Fargate will have more than 99.99% uptime because they are all built on EC2, so to get more than 99.99% uptime you’ll need to run several instances. If your application containers are stateful, that requires code changes.
If your app exceeds RAM limits + Swap in ECS you’ll also see a node failure. Fargate doesn’t even have swap space. If this is a concern, then (1) seems relevant again.
Exceeding storage is a risk with every approach, but in some recommended architectures some or all of the risk is offloaded to another AWS service like RDS. If you expect to see large growth in storage it’s better to run at least your DB on RDS.
Running out of CPU is a mostly mitigated risk when you use ECS or Fargate.
Creating backups of databases is relatively easy and safe with RDS.
Services like ECS and Fargate deploy changes without downtime.

The Verdict

KISS Docker Compose works for production systems which tolerate 2-3 minutes of downtime during code deployments and system backups. You must also size your EC2 instance so it can handle all traffic spikes.

If you have a fairly predictable or small workload, conduct load testing, and tolerate downtime, then it’s a great low-cost way to run a full stack application on AWS.

Try KISS Docker Compose

First, follow “Getting Started with the AWS CDK” to be able to create CDK applications. Then:

mkdir kissdc
cd kissdc
cdk init app --language typescript
npm i aws-cdk-lib
npm i kiss-docker-compose
curl -O https://raw.githubusercontent.com/Gregory-Ledray/kiss-docker-compose-on-aws/main/test/docker-compose.yml

Now open lib/kissdc-stack.ts and copy-paste:

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as fs from 'fs';
import { KissDockerCompose } from 'kiss-docker-compose'

export class KissdcStack extends cdk.Stack {
    constructor(scope: Construct, id: string, props?: cdk.StackProps) {
        super(scope, id, props);

        const dockerComposeFileAsString = fs.readFileSync('./docker-compose.yml', 'utf8');

        const kissDockerCompose = new KissDockerCompose(this, 'kiss-docker-compose', { dockerComposeFileAsString });

        // Exporting the value so you can find it easily
        new cdk.CfnOutput(this, 'Kiss-Docker-Compose-public-ip', {
            value: kissDockerCompose.ec2Instance?.instancePublicDnsName ?? '',
            exportName: 'Kiss-Docker-Compose-public-ip',
        });
    }
}

Deploy:

npx cdk deploy

After deployment finishes, you will see the export Kiss-Docker-Compose-public-ip, as below.

 ✅  KissdcStack

✨  Deployment time: 268.26s

Outputs:
KissdcStack.KissDockerComposepublicip = ec2-98-80-9-243.compute-1.amazonaws.com

To verify the deployment worked, curl that URL and verify you get a response from NGINX. For example:

curl ec2-98-80-9-243.compute-1.amazonaws.com

Finally, destroy the created infrastructure so you don’t rack up AWS charges:

npx cdk destroy

Evolving AWS Infrastructure

This is Part 1 of a series on Evolving AWS Infrastructure. Follow me on dev.to to help find the next posts. This particular post is a follow up to https://dev.to/gregoryledray/apply-kiss-to-infrastructure-3j6d which outlined this approach but didn’t provide the code needed to make it easy to use.

Blog