CircleCI + AWS How to create CI/CD pipeline from scratch Part 2 - Setup and update servers
Ivan Iliukhin
Posted on July 25, 2020
After all previous steps, you've got the images that are stored inside the ECR and the script that automatically build
them. In this part of the tutorial we will:
- setup ECS clusters for development and production environments;
- add deployment commands to the CircleCI script.
Let's do it!
Initialize ECS cluster
Creating of cluster consists of three steps:
- Create an empty cluster with a VPC
- Define the task that will launch the selected container
- Add service that will launch and maintain the desired count of ec2 instances with the previously defined task
Create cluster
Let's start with the definition of cluster:
An Amazon ECS cluster is a logical grouping of tasks or services.
Roughly saying, clusters define the scope and set of rules for the launched tasks. To create one, follow the steps below:
- Select EC2 Linux + Networking you need two clusters one for development and one for master branches
-
Select 1
On Demand
t2.micro instance(or other types of ec2 instances), other configurations by default For the networking section, I recommend to use the default parameters too. It will create a new VPC with
security group allowing income traffic
to the 80 PORT.
Yes, that's all, you should create one for the development and one for the production.
Define task
Tasks are used to run Docker containers in Amazon ECS.
- Select EC2 launch type compatibility
- Choose the name for the task definition(or family, like sometimes it's called)
- For the as the task role choose the role with the
AmazonECSTaskExecutionRolePolicy
policy that we previously created - Select the memory limit, for example, 128 MB if your server is not going to handle a lot of complex requests
- And finally, add a container. On this tab we are interested in the following fields:
- Standard -> Image - initial image for the task should be copied from the ECR looks like 845992245040.dkr.ecr.us-west-2.amazonaws.com/simple_plug_server:master_latest
- Standard -> Port mappings - associate host 80 with the port which is using our application and will be defined next
- Advanced container configuration -> ENVIRONMENT -> Environment variables - define the variable with the name
PORT
and desired value for example - 4100. This value must be used in the port mapping as the container port
Great! You've created the first revision of the task definition. Of course, these tasks can be launched right inside the cluster, but we will use services to simplify updating tasks revisions. Let's add them.
Add service
An Amazon ECS service enables you to run and maintain a specified number of instances of a task definition simultaneously in an Amazon ECS cluster.
To create a service just click on the Create
button on the Services
and fill the form:
- Select EC2 in the
Launch type
selector, because we are deploying our tasks on EC2 instances - In the
Task Definition
select the task and revision that you are created earlier - In the
Cluster
select the cluster where you want to define a service. When you are creating service from a cluster this field will be already selected -
Service type
- REPLICA -
Number of tasks
- 1 because we do not care about scaling for now. - Other setting set by default
After repeating all steps for the development and production cluster, you've got two EC2 instances with running applications. They are available by Public DNS or IP.
Now let's go to the final part of this tutorial. 🚀
Add deployment scripts
With an official orb aws-ecs you can make this very easy. We already added all necessary environment variables in the second part of this tutorial so you should only modify the circleci config.
Result version of the .circleci/config.yml
version: 2.1
orbs:
aws-ecr: circleci/aws-ecr@6.9.1
aws-ecs: circleci/aws-ecs@1.2.0
jobs:
test:
docker:
- image: elixir:1.10
environment:
MIX_ENV: test
working_directory: ~/repo
steps:
- checkout
- run: mix local.hex --force
- run: mix local.rebar --force
- run: mix deps.get
- run: mix deps.compile
- run: mix test
- store_test_results:
path: _build/test/lib/simple_plug_server
workflows:
version: 2
test-build-deploy:
jobs:
- test
- aws-ecr/build-and-push-image:
repo: "simple_plug_server"
tag: "${CIRCLE_BRANCH}_${CIRCLE_SHA1},${CIRCLE_BRANCH}_latest"
requires:
- test
filters:
branches:
only:
- master
- development
- aws-ecs/deploy-service-update:
name: deploy-development
requires:
- aws-ecr/build-and-push-image
family: "simple-plug-server-development"
cluster-name: "SimplePlugServer-development"
service-name: "sps-dev-serv"
container-image-name-updates: "container=simple-plug-server-development,tag=${CIRCLE_BRANCH}_${CIRCLE_SHA1}"
filters:
branches:
only:
- development
- approve-deploy:
type: approval
requires:
- aws-ecr/build-and-push-image
filters:
branches:
only:
- master
- aws-ecs/deploy-service-update:
name: deploy-production
requires:
- approve-deploy
family: "simple-plug-server-production"
cluster-name: "SimplePlugServer-production"
service-name: "simple-plug-server-production"
container-image-name-updates: "container=simple-plug-server-production,tag=${CIRCLE_BRANCH}_${CIRCLE_SHA1}"
filters:
branches:
only:
- master
In this file was added three new jobs. Two aws-ecs/deploy-service-update
respond for the updating respective services in the clusters and approve-deploy
that's waiting for confirmation before the last step for the master branch. For different branches, flows will be a little different. It can be achieved by using parameter filters
in job definitions, where you can specify for which branches or git tags launch jobs.
aws-ecs/deploy-service-update job configuration
I would like to tell you about the parameters for the job aws-ecs/deploy-service-update
:
-
name - the name is used to make jobs in a workflow more human-readable. I am sure you would agree that's
deploy-production
looks much more clearer thanaws-ecs/deploy-service-update
. - requires - used to define the order of jobs execution, namely the previous job that must be finished successfully.
- family - there you should write the name of the task definition(Define task) that you used when you created the task
- cluster-name - it's pretty obvious - the name of the desired cluster where all magic happens
- service-name - the name of the service that's managing tasks inside the previously mentioned cluster
-
container-image-name-updates - updates the Docker image names and/or tag names of existing containers
that had been defined in the previous task definition
-
container - the name of the container that you used when you added the container to the task(circled in blue on the screenshot
Add container to a task and configure the port mapping
) -
tag - one of the tags that you are defined in the
aws-ecr/build-and-push-image
job, in this example it's a${CIRCLE_BRANCH}_${CIRCLE_SHA1}
-
container - the name of the container that you used when you added the container to the task(circled in blue on the screenshot
And that's all 🎉. When you push your branch with the new circleci config and start to work you will see something like that.
I hope that this tutorial was helpful and was not wasted your time. If you have any question and problems feel free to ask me about it in the comments. 👋
Posted on July 25, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.