Jinwook Baek
Posted on July 3, 2020
This post was originally published on notion. Click here if you prefer to read in notion page which has better readability.
Introduction
This blog post illustrate development cycle using django app container. I assume readers are already somewhat familar with docker and docker-compose. Although I used django for app development, it is language-agnostic since the post is about containerized application deployment.
Walkthough is devided into three parts consisting three different environemt respectively. First part, describes the architecture of the app(api and async worker) and how they are deployed on local enviroment. Second part is how to deploy the docker containers on cloud using single ec2 instance with on staging environment. Third part, illustrate how to convert traditional ec2 deployment into ECS using fargate with github actions on prod environment.
You can skip to next part if you are already familiar with docker and AWS architectures.
Setup
First we need to prepare application which is development ready.
This application consist 3 docker containers
-
nginx- web server -
app- Api server -
worker- Async worker
app and worker share same base images, main difference is that app serves http request behind nginx web server using gunicorn and wsgi. Throughout this walkthough nginx and app container will share a common lifecycle.
worker runs asyncronously using redis queue as broker using Django-RQ.
Download source
You can download the app source code here.
github: kokospapa8/ecs-fargate-sample-app
Docker Containers
APP
Base app image
Since app and worker both use same base image and pip takes quite some time to build image, I have uploaded base image in the dockerhub repo.
# Creating image based on official python3 image
FROM python:3.6
# Your contacts, so people blame you afterwards
MAINTAINER Jinwook Baek <kokos.papa8@gmail.com>
# Sets dumping log messages directly to stream instead of buffering
ENV PYTHONUNBUFFERED 1
# Creating and putting configurations
RUN mkdir /config
ADD config/app /config/
# Installing all python dependencies
RUN pip install -r /config/requirements.txt
APP
FROM kokospapa8/django-sample:base
# Installing all python dependencies
RUN pip install -r /config/requirements.txt
# Open port 8000 to outside world
EXPOSE 8000
# When container starts, this script will be executed.
# Note that it is NOT executed during building
CMD ["sh", "/config/django_app.sh"]
# Creating and putting application inside container
# and setting it to working directory (meaning it is going to be default)
RUN mkdir /ecs-sample
WORKDIR /ecs-sample
ADD ecs-sample /ecs-sample/
Worker
FROM kokospapa8/django-sample:base
# Installing all python dependencies
RUN pip install -r /config/requirements.txt
# When container starts, this script will be executed.
# Note that it is NOT executed during building
CMD ["sh", "/config/django_worker.sh"]
# Creating and putting application inside container
# and setting it to working directory (meaning it is going to be default)
RUN mkdir /ecs-sample
WORKDIR /ecs-sample
ADD ecs-sample /ecs-sample/
Nginx config
# app_local.conf
server {
listen 80;
# all requests proxies to app
location / {
proxy_pass http://app:8000;
}
# domain localhost
server_name localhost;
}
Enviroment
As you can see from the source code you need to set some sensitive data into env
# settings/secrets.py
SECRET_KEY = get_env_variable("SECRET_KEY")
......
REDIS_HOST = get_env_variable("REDIS_HOST")
----
# export SECRET_KEY = '' #only SECRET_KEY is needed to be set for local dev
Docker compose
docker-compose-local.yml
Just for local environment, we are using sqlite and redis docker container to mimic cloud enviroment. We will be using RDS-mysql and ElasticCache-redis on staging and production environment.
# File structure version
version: '3'
services:
# Our django application
# Build from remote dockerfile
# Connect local app folder with image folder, so changes will be pushed to image instantly
# Open port 8000
app:
build:
context: .
dockerfile: config/app/Dockerfile_app
hostname: app
volumes:
- ./ecs-sample:/ecs-sample
expose:
- "8000"
environment:
- SECRET_KEY
- ENV=dev
- DJANGO_SETTINGS_MODULE=settings.local
- REDIS_HOST=redis
depends_on:
- redis
- worker
worker:
build:
context: .
dockerfile: config/app/Dockerfile_worker
hostname: worker
volumes:
- ./ecs-sample:/ecs-sample
environment:
- SECRET_KEY
- ENV=dev
- DJANGO_SETTINGS_MODULE=settings.local
- REDIS_HOST=redis
depends_on:
- redis
redis:
image: redis:5.0.5
expose:
- "6379"
restart: always
nginx:
image: nginx
hostname: nginx
ports:
- "80:80"
volumes:
- ./config/nginx/app_local.conf:/etc/nginx/conf.d/app_local.conf
depends_on:
- app
Let's run the containers
$ docker-compose -f docker-compose-local.yml up --build
Check result
Type in following url to see if you are getting correct response.
- http://localhost/api/healthcheck/
- http://localhost/api/v1/posts/
- http://localhost/admin/posts/
- http://localhost/django-rq/ - to check async worker
Moving on
Reference
How To Set Up Django with Postgres, Nginx, and Gunicorn on Ubuntu 16.04 | DigitalOcean
Posted on July 3, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
