Azure DevOps is a platform for running builds and deployments pipelines for your applications. It's recently rebranded to Azure DevOps, just to jump on the bandwagon of the buzzword. It's essentially every other build system, gives you a YAML declarative pipeline DSL, and a UI if you want to build your CICD pipelines by click and drag. It's no where as popular as some of the other big build servers like Jenkins or CircleCI, therefore it's lacking quite a bit in terms of documentations, tutorials and guides.

I recently helped someone deploying AWS infrastructure using Azure DevOps, weird I know. I was initially confused about the choice, but after checking out Azure DevOps's AWS plugin, I kinda understood why a small shop would choose it to be it's CICD tool. Azure DevOps follows Jenkins in that it only provides a few basic deployment modules (called tasks in Azure DevOps) out of the box, like bash scripts, npm builds and maven builds. The rest of its power comes from a rich library of plugins, such as the AWS plugin.

YAML DSL

The YAML DSL configures which jobs to run and on which kind of servers. It also exposes some pipeline configs like what to do when a job failed, do we continue or do we quit. Below is what I wrote for my project, minimal but gets the job done:

jobs:
- job: MyJob
  pool:
    vmImage: 'ubuntu-16.04'
  displayName: My First Job
  continueOnError: true
  workspace:
    clean: outputs
  steps:
......

steps: starts a list of modules to run.

AWS Credentials

AWS Credentials can be configured via the UI (Project Settings -> Service connections -> add new AWS Service Connection). A caveat is that Service Connections only get loaded during pipeline initiation, any new connections added after the pipeline has been created won't get loaded automatically. So you'll have to delete and re-create your pipeline to use new connections. I've explained this in a Github issue that talks about this caveat. It looks like quite a few people are running into this issue, so I'm just sharing lessons learned everywhere.

Cloudformation Module

The AWS plugin comes with several pretty useful modules, like CloudFormation Update/Create, Lambda Deploy and S3 Upload. Unfortunately I don't think they published the documentation for it, so I had to look at their source code to find the docs. I've only used CloudFormation Update/Create and another module called AWS CLI. The CloudFormation module greatly saved my time because I didn't have to handle the idempotence of multiple updates after the initial creation, the module knows to update instead of create if the CloudFormation stack is already created.

  steps:
  - task: CloudFormationCreateOrUpdateStack@1
    inputs:
      awsCredentials: 'aws_tokens'
      regionName: 'us-east-2'
      stackName: 'IAMRoleStack'
      templateFile: templates/iam_role.json
      capabilityIAM: 'true'
      capabilityNamedIAM: 'true'

Here is what I used to deploy my IAM role, awsCredentials, regionName are required for all AWS modules, and stackName and templateFile are required for all CloudFormation modules. The last two are only specific to this module.

AWS CLI Module(or not)

The AWS CLI module gave me false hopes, turned out that you still have to install AWS CLI yourself in order to use the module LOL. I first had to install setuptools and wheel, yeah, pip dependencies doesn't even come for free. I then have to install it in my user space, and it took me a while find where the CLI got installed to because it's not added to my PATH. So I just used the AWS Shell script module, which seems to do exact what the AWS CLI module does, except easier to write.

- script: |
      python -m pip install --upgrade pip==9.0.3 setuptools wheel
      pip install awscli --user
    displayName: 'Install tools'
  - task: AWSShellScript@1
    inputs:
      awsCredentials: 'aws_tokens'
      regionName: 'us-east-2'
      scriptType: 'inline'
      inlineScript: |
        eval $(/home/vsts/.local/bin/aws ecr get-login --no-include-email)