Building applications with pipelines

In larger organizations a lot of things are in place. Think of things like deployment pipelines for your applications. But how do you set those up using the services AWS provides? In this blog post I will guide you to how I do it for my pet projects.

I am a big fan of CloudFormation so the examples I will show are CloudFormation snippets.

The git repository

I am using AWS CodeCommit as a source repository. The reason for this choice is that it integrates with AWS CodePipeline. We also want the ability to test our infrastructure. So I usually use feature branches to do that. Here the first challange appears. We only need 1 repository and a pipeline to deploy our infrastructure. But we also need a pipeline that deploys our feature branch. In CloudFormation you can do this using conditions.

Parameters:
  FeatureGitBranch:
    Type: String
    Default: ""
  ProjectName:
    Type: String
    Default: ""   
Conditions:
  IsMainBranchPipeline: !Equals [!Ref FeatureGitBranch, ""]
  IsFeatureBranchPipeline: !Not [Condition: IsMainBranchPipeline]

By providing a FeatureGitBranch parameter we can now make some choices. For example we could only create the repository in the production pipeline.

Resources:
  PipelineRepo:
    Condition: IsMainBranchPipeline
    Type: AWS::CodeCommit::Repository
    Properties:
      RepositoryName: !Ref ProjectName
      RepositoryDescription: <repository-description>

The pipeline requirements

The pipeline uses a couple of resources. You can share these resources across your pipelines. So they are in a separate template and I deploy these resources once per region that I am using.

First you will need a S3 bucket to store the artifacts created by each stage.

ArtifactsBucket:
  Type: AWS::S3::Bucket
  DeletionPolicy: Retain
  UpdateReplacePolicy: Retain
  Properties:
    BucketName: !Sub codepipeline-artifacts-${AWS::AccountId}-${AWS::Region}
    VersioningConfiguration:
      Status: Enabled
    BucketEncryption:
      ServerSideEncryptionConfiguration:
        - ServerSideEncryptionByDefault:
            SSEAlgorithm: AES256

I am using a predictable bucket name so I can use it in my pipeline. Because bucket names need to have a globally unique name I also include the account id and region.

You will also need an IAM role. You will use this role in your pipeline to deploy your CloudFormation template.

CloudFormationServiceRole:
 Type: AWS::IAM::Role
 Properties:
   RoleName: !Sub cloudformation-execution-role-${AWS::Region}
   AssumeRolePolicyDocument:
     Version: "2012-10-17"
     Statement:
       - Action: sts:AssumeRole
         Effect: Allow
         Principal: { Service: cloudformation.amazonaws.com }
   ManagedPolicyArns:
     - arn:aws:iam::aws:policy/AdministratorAccess

Again I use a predictable name and I included the region. Since role names are unique in your account you can deploy this in every region. For the sake of simplicity I used the AdministratorAccess managed policy. You might want to limit this for your own scope.

The pipeline

The pipeline itself also needs a role. This role can pass the CloudFormation role and use the buckets created in the previous step.

CodePipelineExecutionRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Version: "2012-10-17"
      Statement:
        - Action: "sts:AssumeRole"
          Effect: Allow
          Principal:
            Service:
              - codepipeline.amazonaws.com
    Policies:
      - PolicyName: CodePipelineAccess
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action: "iam:PassRole"
              Resource: !Sub cloudformation-execution-role-${AWS::Region}
      - PolicyName: CodePipelineCodeAndS3Bucket
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Action:
                - s3:GetBucketAcl
                - s3:GetBucketLocation
              Effect: Allow
              Resource:
                - !Sub arn:${AWS::Partition}:s3:::codepipeline-artifacts-${AWS::AccountId}-eu-west-1
                - !Sub arn:${AWS::Partition}:s3:::codepipeline-artifacts-${AWS::AccountId}-eu-central-1
            - Action:
                - "s3:GetObject"
                - "s3:GetObjectVersion"
                - "s3:PutObject"
              Effect: Allow
              Resource:
                - !Sub arn:${AWS::Partition}:s3:::codepipeline-artifacts-${AWS::AccountId}-eu-west-1/*
                - !Sub arn:${AWS::Partition}:s3:::codepipeline-artifacts-${AWS::AccountId}-eu-central-1/*
      - PolicyName: CodeCommitAccess
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Action:
                - codecommit:GitPull
                - codecommit:GetBranch
                - codecommit:GetCommit
                - codecommit:UploadArchive
                - codecommit:GetUploadArchiveStatus
              Effect: Allow
              Resource:
                - !Sub arn:${AWS::Partition}:codecommit:${AWS::Region}:${AWS::AccountId}:${ProjectName}
                - !Sub arn:${AWS::Partition}:codecommit:${AWS::Region}:${AWS::AccountId}:${ProjectName}/*
      - PolicyName: CodePipelineCodeBuildAndCloudformationAccess
        PolicyDocument:
          Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action:
                - "codebuild:StartBuild"
                - "codebuild:BatchGetBuilds"
              Resource:
                - !GetAtt CodeBuildProjectBuildAndPackage.Arn
            - Effect: Allow
              Action:
                - "cloudformation:DescribeStacks"
              Resource: !Sub "arn:${AWS::Partition}:cloudformation:eu-*:${AWS::AccountId}:stack/*"
            - Effect: Allow
              Action:
                - "cloudformation:CreateStack"
                - "cloudformation:DeleteStack"
                - "cloudformation:UpdateStack"
                - "cloudformation:CreateChangeSet"
                - "cloudformation:ExecuteChangeSet"
                - "cloudformation:DeleteChangeSet"
                - "cloudformation:DescribeChangeSet"
                - "cloudformation:SetStackPolicy"
                - "cloudformation:SetStackPolicy"
                - "cloudformation:ValidateTemplate"
              Resource:
                - !Sub "arn:${AWS::Partition}:cloudformation:eu-*:${AWS::AccountId}:stack/${ProjectName}-*/*"

As you can see in the snippet the role is also allowed to start a CodeBuild project. And it can deploy a CloudFormation template.
You might have noticed that I used 2 regions in this role. This is because in this particular example I have a pipeline that uses 2 regions.

Because we use 2 regions we also need 2 artifact stores.

Pipeline:
  Type: AWS::CodePipeline::Pipeline
  Properties:
    ArtifactStores:
      - Region: eu-west-1
        ArtifactStore:
          Location: !Sub codepipeline-artifacts-${AWS::AccountId}-eu-west-1
          Type: S3
      - Region: eu-central-1
        ArtifactStore:
          Location: !Sub codepipeline-artifacts-${AWS::AccountId}-eu-central-1
          Type: S3
    RoleArn: !GetAtt CodePipelineExecutionRole.Arn
    RestartExecutionOnUpdate: true
    Stages:
      - Name: Source
        Actions:
          - Name: SourceCodeRepo
            ActionTypeId:
              Category: Source
              Owner: AWS
              Version: 1
              Provider: CodeCommit
            Configuration:
              PollForSourceChanges: False
              RepositoryName: !Ref ProjectName
              BranchName: !If [IsFeatureBranchPipeline, !Ref FeatureGitBranch, "main"]
            OutputArtifacts:
              - Name: SourceCodeAsZip
            RunOrder: 1

Depending on the IsFeatureBranchPipeline condition we select the correct branch for this pipeline. Depending on your project you might need a build step. I use CodeBuild to run unit tests and build and package any resources needed.

      - Name: BuildAndPackage
        Actions:
          - Name: CodeBuild
            ActionTypeId:
              Category: Build
              Owner: AWS
              Provider: CodeBuild
              Version: "1"
            Configuration:
              ProjectName: !Ref CodeBuildProjectBuildAndPackage
            InputArtifacts:
              - Name: SourceCodeAsZip
            OutputArtifacts:
              - Name: BuildArtifactAsZip

I did not include the CodeBuild project and IAM role snippet for CodeBuild. I will do a blog post on that in the future.

So lets go over to the fun part. The next stage is the actual deployment of the CloudFormation template. I am using 2 regions and I only want to deploy to these 2 regions for my production environment. I also have different parameters for my production environment. Again, I am using conditions to control this behaviour.

          - !If
            - IsMainBranchPipeline
            - Name: Production
              Actions:
                - Name: Ireland-CreateChangeSet
                  RunOrder: 1
                  Region: eu-west-1
                  InputArtifacts:
                    - Name: BuildArtifactAsZip
                  ActionTypeId:
                    Category: Deploy
                    Owner: AWS
                    Provider: CloudFormation
                    Version: "1"
                  Configuration:
                    ActionMode: CHANGE_SET_REPLACE
                    RoleArn: !Sub arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cloudformation-execution-role
                    StackName: !Sub ${ProjectName}-production
                    ChangeSetName: !Sub ${ProjectName}-production-ChangeSet
                    TemplatePath: BuildArtifactAsZip::packaged-template.yaml
                    Capabilities: CAPABILITY_NAMED_IAM
                    ParameterOverrides: |-
                      {
                        "EnvType": "production"
                      }
                - Name: Frankfurt-CreateChangeSet
                  RunOrder: 1
                  Region: eu-central-1
                  InputArtifacts:
                    - Name: BuildArtifactAsZip
                  ActionTypeId:
                    Category: Deploy
                    Owner: AWS
                    Provider: CloudFormation
                    Version: "1"
                  Configuration:
                    ActionMode: CHANGE_SET_REPLACE
                    RoleArn: !Sub arn:aws:iam::${AWS::AccountId}:role/cloudformation-execution-role
                    StackName: !Sub ${ProjectName}-production
                    ChangeSetName: !Sub ${ProjectName}-production-ChangeSet
                    TemplatePath: BuildArtifactAsZip::packaged-template.yaml
                    Capabilities: CAPABILITY_NAMED_IAM
                    ParameterOverrides: |
                      {
                        "EnvType": "production"
                      }
                - Name: Ireland-ExecuteChangeSet
                  RunOrder: 2
                  Region: eu-west-1
                  ActionTypeId:
                    Category: Deploy
                    Owner: AWS
                    Provider: CloudFormation
                    Version: "1"
                  Configuration:
                    ActionMode: CHANGE_SET_EXECUTE
                    RoleArn: !Sub arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cloudformation-execution-role
                    StackName: !Sub ${ProjectName}-production
                    ChangeSetName: !Sub ${ProjectName}-production-ChangeSet
                - Name: Frankfurt-ExecuteChangeSet
                  RunOrder: 2
                  Region: eu-central-1
                  ActionTypeId:
                    Category: Deploy
                    Owner: AWS
                    Provider: CloudFormation
                    Version: "1"
                  Configuration:
                    ActionMode: CHANGE_SET_EXECUTE
                    RoleArn: !Sub arn:aws:iam::${AWS::AccountId}:role/cloudformation-execution-role
                    StackName: !Sub ${ProjectName}-production
                    ChangeSetName: !Sub ${ProjectName}-production-ChangeSet
            - !Ref AWS::NoValue

In the 1 action I create a change set. In the second action I execute it. Please note the RoleArn used here. You could also specify a role from a different account. This allows you to use a separate account for different environments. For example you could host the pipeline in a deployment account. And have separate development, testing, acceptance and production accounts.

The feature branch

So we now have a working pipeline for the production environment. How about the feature branch? For this we use again the conditions.

          - !If
            - IsFeatureBranchPipeline
            - Name: FeatureDevelopment
              Actions:
                - Name: Ireland-CreateChangeSet
                  RunOrder: 1
                  Region: eu-west-1
                  InputArtifacts:
                    - Name: BuildArtifactAsZip
                  ActionTypeId:
                    Category: Deploy
                    Owner: AWS
                    Provider: CloudFormation
                    Version: "1"
                  Configuration:
                    ActionMode: CHANGE_SET_REPLACE
                    RoleArn: !Sub arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cloudformation-execution-role
                    StackName: !Sub ${ProjectName}-${FeatureName}
                    ChangeSetName: !Sub ${ProjectName}-feature-ChangeSet
                    TemplatePath: BuildArtifactAsZip::packaged-template.yaml
                    Capabilities: CAPABILITY_NAMED_IAM
                    ParameterOverrides: |
                      {
                        "EnvType": "development"
                      }
                - Name: Ireland-ExecuteChangeSet
                  RunOrder: 2
                  Region: eu-west-1
                  ActionTypeId:
                    Category: Deploy
                    Owner: AWS
                    Provider: CloudFormation
                    Version: "1"
                  Configuration:
                    ActionMode: CHANGE_SET_EXECUTE
                    RoleArn: !Sub arn:${AWS::Partition}:iam::${AWS::AccountId}:role/cloudformation-execution-role
                    StackName: !Sub ${ProjectName}-${FeatureName}
                    ChangeSetName: !Sub ${ProjectName}-feature-ChangeSet
            - !Ref AWS::NoValue

As you can see, the feature branch is only deployed to the eu-west-1 region. And it has set the EnvType parameter to development.

Pickup new commits

When you push your commits to the CodeCommit repository. You can use an event rule to trigger a new pipeline execution. To make this happen we need an IAM role and the rule itself.

CloudWatchEventRole:
   Type: AWS::IAM::Role
   Properties:
     AssumeRolePolicyDocument:
       Version: '2012-10-17'
       Statement:
       - Effect: Allow
         Principal: { Service: events.amazonaws.com }
         Action: sts:AssumeRole
     Policies:
       - PolicyName: root
         PolicyDocument:
           Version: '2012-10-17'
           Statement:
             - Action: codepipeline:StartPipelineExecution
               Resource: !Sub arn:${AWS::Partition}:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
               Effect: Allow

CloudWatchEventRule:
  Type: AWS::Events::Rule
  Properties:
    Description: Check CodeCommit Repo Changes
    EventPattern:
      detail-type:
        - "CodeCommit Repository State Change"
      source:
        - aws.codecommit
      resources:
        - !Sub arn:${AWS::Partition}:codecommit:${AWS::Region}:${AWS::AccountId}:${ProjectName}
      detail:
        event:
          anything-but:
            - referenceDeleted
        referenceType:
          - branch
        referenceName:
          - !If [IsFeatureBranchPipeline, !Ref FeatureGitBranch, "main"]
    Targets:
      - Id: codepipeline
        Arn: !Sub arn:${AWS::Partition}:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
        RoleArn: !Sub ${CloudWatchEventRole.Arn}

We are triggering on every event except when the branch removal.

Conclusion

You can store the template for the pipeline in your CodeCommit repository. When you create a feature branch you can deploy that template specifying the branch name. The pipeline can then deploy the project CloudFormation template. And with every commit that you push your feature environment will receive an update.

With this setup you have everything in a single repository. (Except for the shared S3 bucket and IAM role.) Having everything in a single repository makes it easier to maintain.

Blog