I work at a consulting company where there are numerous developers and consultants that require a sandbox environment in AWS. Developers may be working in isolation or they may be collaborating with other developers on a shared project.

At present, everyone shares a single AWS account and it becomes challenging to govern. We routinely have to prune resources no longer in use when our monthly bill becomes out of control. This requires an admin to look through the resources in numerous regions and reach out to developers to figure out who owns a resource.

To help manage and govern our AWS usage, I'm going to use AWS Control Tower to allow admins and developers to provision isolated accounts at the developer or project scope and to provide a set of guardrails to enforce AWS best practices.

In the second part of this series, I'm going to describe how the Landing Zone is configured for my team and the customizations added to support some of our policies.

User Management

The first administrative task is to create User accounts in AWS Single Sign-On so that developers can access appropriate AWS accounts in the organization.

Control Tower configures an AWS Single Sign-On directory in the Master account. This directory can be configured to use a user identity store in Active Directory, an external identity provider, or through the built-in identity store in AWS SSO. For my team, we're going to use the built-in identity store.

The next step is to begin inviting the users into the Organization. The users will be added to the AWSAccountFactory group initially. This will give them permissions to use Account Factory to create new AWS accounts within the Organization. Upon create a new user, an invitation will be sent via email where the user can set their password and login.

A recommended practice is to create groups that model your organization's roles and responsibilities. Since groups in AWS SSO can't be members of other groups, assign users to groups managed by your organization and the AWS-managed groups.

I'm going to create a Developers group within AWS SSO and assign the developer users to that group.

There are a few of us on the team that will be administrators on Control Tower and the Organization. These users will have administrator access to the Master account and full access to AWS Control Tower. For them, I'm going to create an Admins group and assign their user accounts to that group.

In AWS SSO, the current list of groups:

For my organization-managed Admins group, I'm going to assign it the AWSAdministratorAccess Permission Set to the Master, Log archive, and Audit AWS accounts.

Permission Sets are backed by an IAM policy. Use Permission Sets to enforce permissions on a User or Group for an AWS Account.

Landing Zone Configuration

A common issue with using a single-shared AWS account is that the account becomes like the wild west. Resources are created but never terminated, team members come and go, and it is tedious to determine who created a resource when it comes time to prune resources when the AWS bill comes due.

Another issue with the single-shared AWS account is the blast radius if the account were ever to become compromised. When an account is compromised AWS disables the account until all issues are resolved. This could have a wide impact on other developers and projects.

To improve management and governance, each developer will have a dedicated AWS account. Additionally, there are shared projects which numerous developers work and collaborate. Each shared project will so have a dedicated AWS account.

Each developer AWS account and project AWS account will be added to the Landing Zone configured by Control Tower. The account will inherit the guardrails assigned to the account's OU. All API activities and resource configurations will also be logged.

Two Organizational Units (OUs) will be added as well. One for developers and one for projects. Using separate OUs will allow us to manage and govern the accounts with different guardrails and policies.

This diagram illustrates the layout of the OUs within the Landing Zone.

When creating OUs for Control Tower, it is best practice to create them through Control Tower and not through AWS Organizations. Using Control Tower to add OUs to the Landing Zone will ensure that they display within Control Tower and have guardrails applied to them. Control Tower will manage the lifecycle of an OU within AWS Organizations.

OUs within Control Tower should be parented to the Root OU and not nested in another OU. Nested OUs are not accessible to Control Tower as it only displays the top-level OUs.

After creating the developers and projects OUs within Control Tower, the Organizational Units page looks like this:

Examining the developers OU, it will show details on the OU including accounts assigned to the OU and guardrails.

At this time, the OU is ready to be used and new accounts can be assigned to the OU. There are further customizations required to enforce some custom policies which will be covered next.

Landing Zone Customizations

While AWS Control Tower provides a good base set of guardrails on member accounts, my team would like to add further policies. For instance for developer accounts, initially we would like to enforce these policies:

• Limit EC2 Instance Types to nano, micro, small, or medium instance types
• Limit RDS Instance Types to micro, small, or medium instance types

To enforce these policies, Service Control Policies (SCPs) can be configured within AWS Organizations and attached to an OU or an individual account.

Additionally, we would like to add a resource to each developer AWS account:

• Set up an initial budget of $25 per month and send notifications at configured thresholds

Resources can be added to AWS accounts using CloudFormation and by using StackSets to deploy to the member AWS accounts from the Master account.

Configuring Service Control Policies and CloudFormation stacks is straightforward, it can be difficult to do at scale. Managing SCPs at the OU level in AWS Organizations makes it easier to enforce policies, the same is not true for CloudFormation stack sets. After accounts are creating through the Account Factory, an admin will have to remember to set up the appropriate CloudFormation stacks on the new account. Also, if new resources and stacks need to be added to exist accounts, then an admin must manually configure the stacks on each AWS account. This can be very tedious and error prone.

AWS provides a custom solution named Customizations for AWS Control Tower.

From the Customizations for AWS Control Tower Deployment Guide:

The Customizations for AWS Control Tower solution combines AWS Control Tower and other highly-available, trusted AWS services to help customers more quickly set up a secure, multi-account AWS environment using AWS best practices. Before deploying this solution, customers need to have an AWS Control Tower landing zone deployed in their account. This solution enables customers to easily add customizations to their AWS Control Tower landing zone using an AWS CloudFormation template and service control policies (SCPs). You can deploy the custom template and policies to individual accounts and organizational units (OUs) within your organization. This solution integrates with AWS Control Tower lifecycle events to ensure that resource deployments stay in sync with the customer's landing zone. For example, when a new account is created using the AWS Control Tower account factory, the solution ensures that all resources attached to the account's OUs will be automatically deployed.

Information on Customizations for AWS Control Tower can be found here: https://aws.amazon.com/solutions/implementations/customizations-for-aws-control-tower/.

Customizations for AWS Control Tower configures the following deployment architecture in the Master account to deploy SCPs and CloudFormation Stack Sets on existing member accounts and new member accounts.

Deploying the Customizations for AWS Control Tower Solution

To configure Customizations for AWS Control Tower in the Master account use the CloudFormation template that's included. It can be found at: https://s3.amazonaws.com/solutions-reference/customizations-for-aws-control-tower/latest/custom-control-tower-initiation.template.

There are 2 parameters for the CloudFormation stack -- PipelineApprovalEmail and PipelineApprovalStage.

Set PipelineApprovalStage to Yes if you would like to approve any changes to SCPs or CloudFormation StackSets before deployment. An email will be sent to email address set in the PipelineApprovalEmail parameter to notify when there is a pending approval.

Setup Customization Configuration Package

The Customizations for AWS Control Tower will deploy SCPs and CloudFormation StackSets using policy files and templates contained in a Configuration Package. AWS CodePipeline in the Customizations for AWS Control Tower will create SCPs using the policy files and create CloudFormation StackSets from the templates and apply them to the appropriate accounts at the OU or account levels.

The developers guide for the Configuration Package and Customizations for AWS Control Tower can be found at: https://s3.amazonaws.com/solutions-reference/customizations-for-aws-control-tower/latest/customizations-for-aws-control-tower-developer-guide.pdf.

The Configuration Package is uploaded into an S3 bucket which will trigger the deployment process in AWS CodePipeline.

I created a GitHub repo with a sample Configuration Package that contains Service Control Policies for limiting instance types for EC2 and RDS and a CloudFormation template for configuring a Budget in each member account.

The manifest file in the Configuration File controls the deployment of SCPs and CloudFormation templates. In the manifest file, the list of Organization Policies (SCPs) and CloudFormation Resources is configured to include the OUs or individual member accounts to target. The configuration of the manifest file will be described in the following sections.

Configuring Service Control Policies

In our team's member accounts, we want to limit the size of the EC2 and RDS instances in order to save costs. This can be controlled via Service Control Policies (SCPs). SCPs are defined as IAM policies. The SCPs will be configured within AWS Organizations via the Customizations for AWS Control Tower solution.

Service Control Policies (SCPs) offer central control over the maximum available permissions for all accounts in your organization. SCPs help you to ensure your accounts stay within your organization’s access control guidelines.

The following is a sample SCP for limiting EC2 instance types to only nano, small, micro, or medium. In this policy there is an explicit deny to Run EC2 Instances that don't match the pattern of *.nano, *.small, *.micro, or *.medium.

custom-control-tower-configuration/policies/ec2-instance-types.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "LimitEC2InstanceType",
      "Effect": "Deny",
      "Action": "ec2:RunInstances",
      "Resource": "arn:aws:ec2:*:*:instance/*",
      "Condition": {
        "ForAnyValue:StringNotLike": {
          "ec2:InstanceType": [
            "*.nano",
            "*.small",
            "*.micro",
            "*.medium"
          ]
        }
      }
    }
  ]
}

The following is a sample SCP for limiting RDS instance types to only small, micro, or medium. In this policy there is an explicit deny to Create or Start RDS databases that don't match the pattern of *.small, *.micro, or *.medium.

custom-control-tower-configuration/policies/rds-instance-types.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "LimitRDSInstanceType",
      "Effect": "Deny",
      "Action": [
        "rds:CreateDBInstance",
        "rds:StartDBInstance"
      ],  
      "Resource": "arn:aws:rds:*:*:*",
      "Condition": {
        "ForAnyValue:StringNotLike": {
          "rds:DatabaseClass": [
            "db.*.micro",
            "db.*.small",
            "db.*.medium"
          ]
        }
      }
    }
  ]
}

With the SCPs defined in the two separate files, the next step is to configure them within the manifest so they are applied appropriately. It will apply the SCP to all current and future accounts within the developers OU.

This section of the manifest file will apply the SCPs to the appropriate member accounts in the landing zone.

custom-control-tower-configuration/manifest.yaml:

organization_policies:
  - name: limit-ec2-instance-types
    description: Limit EC2 instance type in member accounts
    policy_file: policies/ec2-instance-types.json
    apply_to_accounts_in_ou:
      - developers
  - name: limit-rds-instance-types
    description: Limit RDS instance type in member accounts
    policy_file: policies/rds-instance-types.json
    apply_to_accounts_in_ou:
      - developers

After deployment, we can see the SCP configured in AWS Organizations for the developers OU.

In any AWS account within the developers OU, the creation of EC2 or RDS instances outside of the policy will not be allowed.

Configuring CloudFormation Resources

In our team's member accounts, we want to create a budget for $25 per month (for example) and then notify once budget thresholds of 75/90/100/110% thresholds. This will help us govern costs so that we can stop unused resources before the bill becomes out of control.

To set up the Budget, a CloudFormation template is needed. A CloudFormation StackSet will be created within the Master account using the template and will create individual CloudFormation stacks in the appropriate member AWS accounts.

AWS CloudFormation StackSets extends the functionality of stacks by enabling you to create, update, or delete stacks across multiple accounts and regions with a single operation. Using an administrator account, you define and manage an AWS CloudFormation template, and use the template as the basis for provisioning stacks into selected target accounts across specified regions.

The CloudFormation template to define the Budget is shown here.

custom-control-tower-configuration/templates/budget.template

AWSTemplateFormatVersion: 2010-09-09
Description: Configures Budget Notifications for member accounts at 75/90/100/110% thresholds
Parameters:
  BudgetName:
    Description: Name to assign budget
    Type: String
  BudgetAmount:
    Description: Budget Limit for one month 
    Type: Number
    Default: 25
  NotificationEmailAddress:
    Description: Email Address to notify when 75/90/100/110% thresholds are reached
    Type: String
Resources: 
  AccountBudget:
    Type: AWS::Budgets::Budget
    Properties:
      Budget:
        BudgetName: !Ref BudgetName
        BudgetLimit:
          Amount: !Ref BudgetAmount
          Unit: USD
        TimeUnit: MONTHLY
        BudgetType: COST
      NotificationsWithSubscribers:
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 75
            ThresholdType: PERCENTAGE
          Subscribers:
            - SubscriptionType: EMAIL
              Address: !Ref NotificationEmailAddress
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 90
            ThresholdType: PERCENTAGE
          Subscribers:
            - SubscriptionType: EMAIL
              Address: !Ref NotificationEmailAddress
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 100
            ThresholdType: PERCENTAGE
          Subscribers:
            - SubscriptionType: EMAIL
              Address: !Ref NotificationEmailAddress
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 110
            ThresholdType: PERCENTAGE
          Subscribers:
            - SubscriptionType: EMAIL
              Address: !Ref NotificationEmailAddress

The next step is to configure the CloudFormation template within the manifest so the CloudFormation resources are created appropriately and targets the right accounts. The CloudFormation template to define a Budget will be configured to be created in all current and future AWS accounts in the developers OU.

This section of the manifest file will apply the CloudFormation resources to the appropriate member accounts in the landing zone.

custom-control-tower-configuration/manifest.yaml

cloudformation_resources:
  - name: budget-small
    template_file: templates/budget.template
    parameter_file: parameters/budget.small.json
    deploy_method: stack_set
    deploy_to_ou:
      - developers
    regions:
      - us-east-1

The parameters file will define the parameter values to be used when creating the CloudFormation StackSets. In this case, the values for a small budget will be used. You will notice that BudgetAmount is $25

custom-control-tower-configuration/parameters/budget.small.json

[
  {
    "ParameterKey": "BudgetName",
    "ParameterValue": "budget-small"
  },
  {
    "ParameterKey": "BudgetAmount",
    "ParameterValue": "25"
  },
  {
    "ParameterKey": "NotificationEmailAddress",
    "ParameterValue": "name@email.com"
  }
]

After deployment, we can see the CloudFormation StackSet created in the Master account in the landing zone with a few instances in the member accounts:

If we look in a member account, we can see the instance of the CloudFormation stack that was created from the StackSet in the Customizations for AWS Control Tower solution in the Master account.

Conclusion

This post shows how to configure the Landing Zone created by AWS Control Tower and also how to deploy custom policies and resources to AWS accounts in the Landing Zone using the Customizations for AWS Control Tower.

Stay tuned for additional posts in this series on AWS Control Tower and governance.