Scenario

• Certbot it's not currently installed in the web server so the certificate is generated somewhere else then copied to the web server
• The Route53 DNS method is currently used to manually renew the certificate
• The Route53 domain already existed so it doesn't need to be created
• Little to no maintenance after deployment of new infrastructure

Technologies used

A short description of the technoligies used.

1. Docker

   • Use a docker container with certbot and the dns-route53 plugin installed.
   • Below is the command used to locally test the container:

       docker run --rm -it --env AWS_ACCESS_KEY_ID=ACCESS_KEY --env AWS_SECRET_ACCESS_KEY=SECRET_KEY -v "/c/letsencrypt:/etc/letsencrypt" certbot/dns-route53 certonly --dns-route53 -d DOMAIN.COM -m EMAIL@TEST.COM --agree-tos --non-interactive --server https://acme-v02.api.letsencrypt.org/directory
   

2. AWS Batch and EFS

   • AWS Batch Jobs can be used to create on-demand Fargate instances to run the Docker container.
   • The problem is how to move the certificates created out of the container. The solution is to mount an EFS volume.
   • The creation/management of the ECS clusters, tasks, tasks definitions is transparent to the user, everything is done in AWS Batch.
   • In summary (in the AWS Batch console), first you need to create a "Compute Environment", then a "Job Queue" and finally a "Job Definition". After that, you can run Jobs selecting the proper Job Definition and Queue.
   • The video used to get a general idea on how AWS Batch works:
     • Batch can now use Fargate for a truly serverless experience

3. AWS Step Functions and Lambdas

   • Join all the steps using Step Functions to execute the lambdas and invoke the batch jobs.

4. AWS IAM, S3 and SSM Parameters

   • An S3 bucket is used to store the certificate copied from EFS.
   • AWS IAM, a user is created along its access and secret keys. The credentials are used in the certbot command execution to create/renew the certificate.
   • SSM Parameters, the parameters are used to store the access key and secret key from the user previously created. If these values are updated in the IAM User, the SSM parameters need to be updated too.

5. Terraform

   • Used to deploy the pre-requisites: IAM user, S3 bucket and SSM Parameters
   • Used to deploy and update the AWS Batch infrastructure

6. Serverless Framework

   • Used to the deploy and manage the the Step Functions and Lambda functions.

Implementation description

1. An EventBridge rule is scheduled to start the Step Function.

2. The step function process checks if the domain certificate needs to be renewed. It renews the certificate, copies it to S3, then copies it to the EC2 instance and restarts the httpd service using the SSM. Finally we get a list of the certificates.

Below the Step Function process is detailed.

1. CheckCertificate: A Lambda function is executed to check how many days are left before the certificate expires. This information is retrieved from the domain itself, meaning we're not checking the certificate stored in EFS.

2. ChoiceDaysLeft: A choice state is executed next. If the number of days is less or equal than 30 days then a renewal is executed, else only the certificate information is listed.
   
   

3. RenewCertificates: The AWS Batch Job (using Fargate) to run the renewal command is executed.

   

4. CopyToS3: All the files stored on EFS are copied to the S3 Bucket.

   

5. GetFromS3: The files inside the "live" folder are copied to the instance to the proper path.

6. The Job to run the "certbot list" command is executed.

   

Setting up the Infrastructure

NOTE: The code can be found in the GitHub repo

1. Follow the instructions to install Terraform

2. Deploy the pre-requisites folder.

3. Deploy the EFS infrastructure

   • Update the file prod.tfvars with the proper variables.

4. Deploy the AWS Batch infrastructure

   • Update the file prod.tfvars with the proper variables.
   • Update the variable "fileSystemId" in the job definition files "create_certs_job.json", "list_certs_job.json", "renew_certs_job.json"

5. Configure the IAM User and SSM Parameters

   • Create the access key and secret access key for the user "certbot_batch"
   • Copy the values to the SSM Parameters: /certbot_batch/access_key and /certbot_batch/secret_key.

6. Deploy the Serverless Framework infrastructure

   • Follow the instructions to install Serverless

7. If needed, execute the "Configurations needed only once" steps

Configurations needed only once (the first execution)

1. The Batch Job to create a new certificate is only run once at the beginning, so certbot creates the folder infrastructure needed in EFS. The subsequent executions will be of the renewal Job.
2. The current certificate in use in the instances was copied to EFS (after the folder structure is created).

Notes

• The approach could be modified to copy the certificate directly to the EC2 instance so the S3 bucket is not needed.
• The process could be improved by adding notifications when the process fails.

Links Reviewed

• Generate Let’s Encrypt Certificate with DNS Challenge and Namecheap
• Welcome to certbot-dns-route53’s documentation
• Certbot-Running with Docker
• Let’s Encrypt Wildcard Certificate Configuration with AWS Route 53