TheLazy.Dev
Portfolio GitHub

Blog

aws |cloud |devops |productivity

Introduction to AWS CDK

bismillahkani

bismillahkani

Posted on June 26, 2023

Introduction to AWS CDK

What is AWS CDK?

AWS CDK is an opensource software development framework to create and deploy the cloud infrastructure using your familiar languages like Python. AWS CDK make the infrastructure management fun and easy 🕺

Manual infrastructure Deployment

Let’s have a look into the process of deploying infrastructure manually. Typically we use the management console to create and deploy resources but sometimes we use CLI tools. For instance, to create a S3 bucket you log into the AWS account, click Create Bucket and fill the form. In addition to creation of resources, you have to manually create the bucket policy, ACL and IAM policy in order to integrate to other components in the application. This is fine to begin with and when the infrastructure requirements are minimal. When it comes to enterprise level infrastructure management where you have to manage multiple applications with multiple stacks that has many resources the manual process of deployment is not reliable nor consistent. It is error prone and maintenance of application infrastructure becomes nightmare.

Image description
From this example, we can understand that deploying and managing infrastructure manually for a fairly complex application is not a viable solution.

Infrastructure as Code (IaC)

Infrastructure as Code is the process of provisioning and managing the resources programmatically through machine readable definition files applying the same rigor as the application code development. For AWS the Infrastructure as Code is implemented using AWS CloudFormation. With AWS CloudFormation the application resources can be provisioned by writing the definition of resources in a cloud formation template either in YAML or JSON. For example, the following YAML template describes AWS S3 bucket to create.

AWSTemplateFormatVersion: 2010-09-09
Resources:
  S3Bucket:
    Type: 'AWS::S3::Bucket'
    Properties:
      AccessControl: PublicRead
      WebsiteConfiguration:
        IndexDocument: index.html
        ErrorDocument: error.html
    DeletionPolicy: Retain
  BucketPolicy:
    Type: 'AWS::S3::BucketPolicy'
    Properties:
      PolicyDocument:
        Id: MyPolicy
        Version: 2012-10-17
        Statement:
          - Sid: PublicReadForGetBucketObjects
            Effect: Allow
            Principal: '*'
            Action: 's3:GetObject'
            Resource: !Join 
              - ''
              - - 'arn:aws:s3:::'
                - !Ref S3Bucket
                - /*
      Bucket: !Ref S3Bucket
Enter fullscreen mode Exit fullscreen mode

Cloud formation YAML template

AWS CloudFormation takes this template and will create, update and delete resources on your AWS account according to the description in the template.

Benefits of IaC:

  1. Visibility - The template act as singe source of reference for the resources. Web console is not required.
  2. Stability - Version controlled and rollback to previous stable version incase new changes break the application.
  3. Scalability - Reusable and scales horizontally seamlessly.

How CDK works?

CDK is built on top of the AWS CloudFormation service and uses it as the engine for provisioning resources. AWS CDK uses the familiarity and expressive power of favorite programming languages for modelling your applications. The AWS CDK supports TypeScript, JavaScript, Python, Java, C#/.Net, and (in developer preview) Go. Developers can use their favorite programming languages to define reusable components called Constructs and compose several Constructs into Stacks and Apps.

Image description
Image credit: https://docs.aws.amazon.com/cdk/v2/guide/home.html

CDK Constructs

Constructs are basic building blocks of AWS CDK apps. Constructs are high level components that preconfigure cloud resources with proven defaults to make the infrastructure provisioning easier. AWS CDK provides a library of constructs that cover many AWS features and services. Developers can define the application infrastructure at high level using these constructs. Constructs are adjustable and composable and one can easily change the parameters and build custom constructs.

AWS Constructs library

The CDK constructs library includes all the resources that are available in AWS. For example, s3.Bucket class represents an AWS S3 bucket and dynamodb.Table represents an AWS DynamoDB table.

There are three level of constructs in this library,

  1. L1 Constructs are the low level constructs that directly represents the resources in AWS CloudFormation. It is also called CFN Resources.
  2. L2 Constructs are the next higher level constructs that are similar in function to L1 constructs but provide defaults, boiler plate and glue logic.
  3. L3 Constructs which are called as patterns are designed to accomplish common tasks in AWS For example, aws-apigateway.LambdaRestApi construct represents an API Gateway backed by AWS Lambda function. 
from aws_cdk import App, Stack
import aws_cdk.aws_s3 as s3
from constructs import Construct

class HelloCdkStack(Stack):

    def __init__(self, scope: Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        s3.Bucket(self, "MyFirstBucket", versioned=True)

app = App()
HelloCdkStack(app, "HelloCdkStack")
Enter fullscreen mode Exit fullscreen mode

CDK Stacks and Apps

The smallest unit of deployment in AWS CDK is called a stack. All of the resources defined within the scope of a stack is deployed as a single unit. You can define any number of stacks inside your AWS CDK app.

As an example, here we declare a stack class MyFirstStack that includes a single Amazon S3 bucket. However, this only declares a stack and the stack have to defined in some scope in order to be deployed.

class MyFirstStack(Stack):

    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)

        s3.Bucket(self, "MyFirstBucket")
Enter fullscreen mode Exit fullscreen mode 
app = App()

MyFirstStack(app, 'stack1')
MySecondStack(app, 'stack2')

app.synth()
Enter fullscreen mode Exit fullscreen mode

CDK Toolkit

The AWS CDK Toolkit known as cdk is the command line tool is the primary tool to provision and manage AWS CDK app. It executes the app, generates and deploys the AWS CloudFormation templates.

Setup

  1. AWS CLI - Make sure you have the latest version of the AWS CLI installed on your system.
  2. AWS Account and User - You’ll need access to an AWS account. Create IAM user with programmatic access and attach AdministratorAccess policy to the IAM user.
  3. Node.js - The AWS CDK uses Node.js (>= 10.13.0, except for versions 13.0.0 - 13.6.0).
  4. VSCode - IDE
  5. Python - 3.6 or later.
  6. AWS CDK Toolkit 
npm install -g aws-cdk
Enter fullscreen mode Exit fullscreen mode 
cdk --version
Enter fullscreen mode Exit fullscreen mode

CDK commands

Command Function
cdk ls Lists the stacks in the app
cdk synt Synthesizes and prints the CloudFormation template for the specified stack(s)
cdk bootstrap Deploys the CDK Toolkit staging stack; see Bootstrapping
cdk deploy Deploys the specified stack(s)
cdk destroy Destroys the specified stack(s)
cdk diff Compares the specified stack with the deployed stack or a local CloudFormation template
cdk init Creates a new CDK project in the current directory from a specified template

Your first CDK project

Create project directory 📁

mkdir cdk_workshop 
cd cdk_workshop
Enter fullscreen mode Exit fullscreen mode

Use cdk init to create a new Python CDK projects

cdk init sample-app --language python
Enter fullscreen mode Exit fullscreen mode

Activating the Virtualenv. The init script created bunch of code to get started and also created a virtual environment. Activate the virtual environment.

source .venv/bin/activate
Enter fullscreen mode Exit fullscreen mode

Install the Python modules.

pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

Project directory

Image description
The application entry point is app.py.

#!/usr/bin/env python3

import aws_cdk as cdk

from cdk_workshop.cdk_workshop_stack import CdkWorkshopStack

app = cdk.App()
CdkWorkshopStack(app, "cdk-workshop")

app.synth()
Enter fullscreen mode Exit fullscreen mode

The constructs are defined in cdk_workshop/cdk_workshop_stack.py

from constructs import Construct
from aws_cdk import (
    Duration,
    Stack,
    aws_sqs as sqs,
    aws_sns as sns,
    aws_sns_subscriptions as subs,
)

class CdkWorkshopStack(Stack):

    def __init__(self, scope: Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        queue = sqs.Queue(
            self, "CdkWorkshopQueue",
            visibility_timeout=Duration.seconds(300),
        )

        topic = sns.Topic(
            self, "CdkWorkshopTopic"
        )

        topic.add_subscription(subs.SqsSubscription(queue))
Enter fullscreen mode Exit fullscreen mode

Synthesize cloud formation template from the CDK app

cdk synth
Enter fullscreen mode Exit fullscreen mode

This will output the following template

Resources:
  CdkworkshopQueue18864164:
    Type: AWS::SQS::Queue
    Properties:
      VisibilityTimeout: 300
    Metadata:
      aws:cdk:path: cdkworkshop/CdkworkshopQueue/Resource
  CdkworkshopQueuePolicy78D5BF45:
    Type: AWS::SQS::QueuePolicy
    Properties:
      PolicyDocument:
        Statement:
          - Action: sqs:SendMessage
            Condition:
              ArnEquals:
                aws:SourceArn:
                  Ref: CdkworkshopTopic58CFDD3D
            Effect: Allow
            Principal:
              Service: sns.amazonaws.com
            Resource:
              Fn::GetAtt:
                - CdkworkshopQueue18864164
                - Arn
        Version: "2012-10-17"
      Queues:
        - Ref: CdkworkshopQueue18864164
    Metadata:
      aws:cdk:path: cdkworkshop/CdkworkshopQueue/Policy/Resource
  CdkworkshopQueuecdkworkshopCdkworkshopTopic7642CC2FCF70B637:
    Type: AWS::SNS::Subscription
    Properties:
      Protocol: sqs
      TopicArn:
        Ref: CdkworkshopTopic58CFDD3D
      Endpoint:
        Fn::GetAtt:
          - CdkworkshopQueue18864164
          - Arn
    Metadata:
      aws:cdk:path: cdkworkshop/CdkworkshopQueue/cdkworkshopCdkworkshopTopic7642CC2F/Resource
  CdkworkshopTopic58CFDD3D:
    Type: AWS::SNS::Topic
    Metadata:
      aws:cdk:path: cdkworkshop/CdkworkshopTopic/Resource
  CDKMetadata:
    Type: AWS::CDK::Metadata
    Properties:
      Modules: aws-cdk=1.18.0,jsii-runtime=Python/3.7.3
Enter fullscreen mode Exit fullscreen mode

The template contains the following resources:

  1. AWS::SQS::Queue - queue
  2. AWS::SNS::Topic - topic
  3. AWS::SNS::Subscription - the subscription between the queue and the topic
  4. AWS::SQS::QueuePolicy - the IAM policy which allows this topic to send messages to the queue

Bootstrapping the environment 💼

For the first time you deploy CDK app into an environment or region, a bootstrap stack has to be installed which includes the needed resources for toolkit operations.

cdk bootstrap
Enter fullscreen mode Exit fullscreen mode

Deploy 🚀

cdk deploy
Enter fullscreen mode Exit fullscreen mode

In the CloudFormation console, you will see that the stack has been created successfully.

Image description
Clean up

cdk destroy
Enter fullscreen mode Exit fullscreen mode

How CDK improve developers experience?

  1. Infrastructure and code in one place
  2. Easy to define IAM policy - minimal code and minimal permissions
  3. Refactoring code
  4. Snapshot test - Compare the changes to previously successful state
  5. Fine grained assertion tests - Test specific aspects of cloud formation templates
  6. Validation tests - Validate rules and checks

Conclusion

AWS CDK is a game changer in Infrastructure as Code. It is a complete developer friendly tool to provision and manage all your cloud infrastructure resources.

References

Author

Bismillah Kani is a passionate and multi-faceted AI Scientist and Cloud Architect with a passion for deep learning, machine learning, and artificial intelligence. As an AWS certified professional he is constantly striving for excellence and innovation in the field of AI and Cloud.

💖 💪 🙅 🚩
bismillahkani
bismillahkani

Posted on June 26, 2023

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Introduction to AWS CDK
aws Introduction to AWS CDK

June 26, 2023

No More Surprises: How the AWS Calculator Can Save You from Budget Overruns and Headaches
aws No More Surprises: How the AWS Calculator Can Save You from Budget Overruns and Headaches

May 4, 2023

The Amazon VPC Resource Map
aws The Amazon VPC Resource Map

February 28, 2023

Announcements from Werner Vogels Keynote at re:Invent 2022
aws Announcements from Werner Vogels Keynote at re:Invent 2022

December 1, 2022