TheLazy.Dev
Portfolio GitHub

Blog

terraform |opa |aiven |redis

Enforce fine-grained policy control across your data infrastructure with Open Policy Agent and Terraform

dewanahmed

Dewan Ahmed

Posted on July 5, 2023

Enforce fine-grained policy control across your data infrastructure with Open Policy Agent and Terraform

In this tutorial, you'll use Open Policy Agent (OPA) to enforce fine-grained policy control across development and production environments for Terraform deployments. This tutorial assumes that you are already using Terraform.

The challenge

Rapu started at Crab Inc. as a Junior DevOps Engineer. He is shadowing a senior engineer on the team to learn how the team deploys PostgreSQL, Redis, Kafka, and other data-related services across development and production environments.

The development team is based in Montreal, Canada and they should only create cloud resources in the Google Cloud Montreal region. However, to ensure high availability for the company's North American customers, the production environment supports multiple AWS cloud regions in the US East location. Previously, there were no guardrails in place and Rapu deployed to cloud regions where he wasn't supposed to deploy.

These are the specific regions for Prod and Dev.

  • Prod: "aws-us-east1"
  • Dev: "google-northamerica-northeast1"

Your goal is to help Rapu enforce these policies so that resources don't get created in the wrong cloud or region.

Prerequisites

The concept of the tutorial is agnostic to the Terraform provider you choose. For the sake of a demo, I'll choose Aiven Terraform Provider. Aiven provides highly-available and scalable data infrastructure based on open-source technologies. For this tutorial, you'll create a free Aiven account and an Aiven authentication token.

Install OPA and Terraform.

Install jq (optional).

The Story

In this section, you'll help Rapu create Terraform files that describe an Aiven for Redis® resource. This can be any cloud resource for your use case and Aiven for Redis is used as an example. Besides the Aiven for Redis resource in the services.tf file, create a provider.tf file for the provider and version details. You'll also create a variables.tf to define the required variables for Aiven Terraform Provider.

Our protagonist Rapu will learn how to decouple and enforce policies using some common tools. When decoupling policies using Open Policy Agent, the structure is pretty consistent no matter the tool or service.

There is a tool/service, in this case we will be using Terraform. This tool will generate some data that will be used as Input for our decision. The input file will be sent to OPA to be compared against the Policy(written in Rego) and any additional Data.

As an added bonus Rapu will learn how to write Unit tests for his policies, which is part of clean code and best practicies.

Here's a high-level overview of the system:

High-level overview

Here's a detailed version of the same system:

Detailed view

In this tutorial, you run terraform and opa commands manually and from your local machine. The above diagram shows how these tools can be used in an automated way. For example, a CI/CD pipeline can deploy cloud resources using Terraform if the OPA policy allows. A deny from OPA might result in a Slack or email notification to the developer.

Set up Terraform files

In an empty directory, create these three files:

provider.tf file:

terraform {
  required_providers {
    aiven = {
      source  = "aiven/aiven"
      version = "~> 4.1.0"
    }
  }
}

provider "aiven" {
  api_token = var.aiven_api_token
}
Enter fullscreen mode Exit fullscreen mode

variables.tf file:

variable "aiven_api_token" {
  description = "Aiven console API token"
  type        = string
}

variable "project_name" {
  description = "Aiven console project name"
  type        = string
}
Enter fullscreen mode Exit fullscreen mode

service.tf file:

resource "aiven_redis" "redis-demo" {
    project = var.project_name # Find your Aiven project name from top-left of Aiven console
    plan = "hobbyist" # For this exercise, the hobbyist plan will do
    service_name = "redis-demo" # Choose any service name
    cloud_name = "google-northamerica-northeast1" # Choose any cloud region from https://docs.aiven.io/docs/platform/reference/list_of_clouds
}
Enter fullscreen mode Exit fullscreen mode

To set the values for the environment variables like aiven_api_token or project_name, you can either use TF_VAR_name, use variables on the command line, or use a variable definition file.

Prepare the Terraform manifest for OPA

Before executing your Terraform manifest with OPA, it's important to add two environment variables so that Terraform client knows about two variables that Aiven Terraform Provider requires. Please replace the placeholder values with the actual values for your Aiven API token and Aiven project name.


export TF_VAR_aiven_api_token=YOUR_AIVEN_API_TOKEN

export TF_VAT_project_name=YOUR_AIVEN_PROJECT_NAME
Enter fullscreen mode Exit fullscreen mode

Now initialize this directory with the terraform init command and ask Terraform to calculate what changes it will make and store the output in plan.binary.

terraform init
terraform plan --out tfplan.binary
Enter fullscreen mode Exit fullscreen mode

Use the command terraform show to convert the Terraform plan into JSON so that OPA can read the plan.

terraform show -json tfplan.binary > tfplan.json
Enter fullscreen mode Exit fullscreen mode

For improved readability you can pipe the output through jq before saving the file.

terraform show -json tfplan.binary | jq > tfplan.json
Enter fullscreen mode Exit fullscreen mode

Here is a sample output of tfplan.json:


{
    "format_version": "1.1",
    "terraform_version": "1.2.8",
    "variables": {
        "aiven_api_token": {
            "value": "3Xi1J+E0G3vo0fwr2vWMLl0XgHwRyA6pCX8C6rQQZVhoyFfz9WAMreaGZPAI+jRUGWgtslQKQtIZTICCDZlZkQn3sRYHGBcAxgXqoiT3l9cYVbVvyPNSVPGHrSvBhSCXIYgWX3AXkOG/kQiJ1r0CZn0Y0gK/pRyiti6dImIzyEsZWja9FZk+mV/M/6BAZMKpa/EokkKUj4puMpUX4B3//slU9yUdicr2wCe/uyx53K64rU/OWZYCbqfTI6QcsjZc1wd8/a+0aLsv651qZwxmgTAenmj0JC5tXWD+Dx89NaiZcUdGxhyg58ZYfYh6U3YDm5S/ovDcvq9m/ffMKbb2Sut2vVELPO1l6AA70U1besBR0dE="
        },
        "project_name": {
            "value": "devrel-dewan"
        }
    },
    "planned_values": {
        "root_module": {
            "resources": [
                {
                    "address": "aiven_redis.redis-demo",
                    "mode": "managed",
                    "type": "aiven_redis",
                    "name": "redis-demo",
                    "provider_name": "registry.terraform.io/aiven/aiven",
                    "schema_version": 0,
                    "values": {
                        "additional_disk_space": null,
                        "cloud_name": "google-northamerica-northeast1",
                        "disk_space": null,
                        "maintenance_window_dow": null,
                        "maintenance_window_time": null,
                        "plan": "hobbyist",
                        "project": "devrel-dewan",
                        "project_vpc_id": null,
                        "redis_user_config": [],
                        "service_integrations": [],
                        "service_name": "redis-demo",
                        "service_type": "redis",
                        "static_ips": null,
                        "tag": [],
                        "termination_protection": null,
                        "timeouts": null
                    },
                    "sensitive_values": {
                        "components": [],
                        "redis": [],
                        "redis_user_config": [],
                        "service_integrations": [],
                        "tag": []
                    }
                }
            ]
        }
    },
    "resource_changes": [
        {
            "address": "aiven_redis.redis-demo",
            "mode": "managed",
            "type": "aiven_redis",
            "name": "redis-demo",
            "provider_name": "registry.terraform.io/aiven/aiven",
            "change": {
                "actions": [
                    "create"
                ],
                "before": null,
                "after": {
                    "additional_disk_space": null,
                    "cloud_name": "google-northamerica-northeast1",
                    "disk_space": null,
                    "maintenance_window_dow": null,
                    "maintenance_window_time": null,
                    "plan": "hobbyist",
                    "project": "devrel-dewan",
                    "project_vpc_id": null,
                    "redis_user_config": [],
                    "service_integrations": [],
                    "service_name": "redis-demo",
                    "service_type": "redis",
                    "static_ips": null,
                    "tag": [],
                    "termination_protection": null,
                    "timeouts": null
                },
                "after_unknown": {
                    "components": true,
                    "disk_space_cap": true,
                    "disk_space_default": true,
                    "disk_space_step": true,
                    "disk_space_used": true,
                    "id": true,
                    "redis": true,
                    "redis_user_config": [],
                    "service_host": true,
                    "service_integrations": [],
                    "service_password": true,
                    "service_port": true,
                    "service_uri": true,
                    "service_username": true,
                    "state": true,
                    "tag": []
                },
                "before_sensitive": false,
                "after_sensitive": {
                    "components": [],
                    "redis": [],
                    "redis_user_config": [],
                    "service_integrations": [],
                    "service_password": true,
                    "service_uri": true,
                    "tag": []
                }
            }
        }
    ],
    "configuration": {
        "provider_config": {
            "aiven": {
                "name": "aiven",
                "full_name": "registry.terraform.io/aiven/aiven",
                "version_constraint": "~> 4.1.0",
                "expressions": {
                    "api_token": {
                        "references": [
                            "var.aiven_api_token"
                        ]
                    }
                }
            }
        },
        "root_module": {
            "resources": [
                {
                    "address": "aiven_redis.redis-demo",
                    "mode": "managed",
                    "type": "aiven_redis",
                    "name": "redis-demo",
                    "provider_config_key": "aiven",
                    "expressions": {
                        "cloud_name": {
                            "constant_value": "google-northamerica-northeast1"
                        },
                        "plan": {
                            "constant_value": "hobbyist"
                        },
                        "project": {
                            "constant_value": "devrel-dewan"
                        },
                        "service_name": {
                            "constant_value": "redis-demo"
                        }
                    },
                    "schema_version": 0
                }
            ],
            "variables": {
                "aiven_api_token": {
                    "description": "Aiven console API token"
                },
                "project_name": {
                    "description": "Aiven console project name"
                }
            }
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Note The api token in the above example is invalid and is shown as an example only.

Write OPA policies in Rego

OPA policies are written in Rego. The following Rego checks if Rapu can deploy to a development environment or a production environment based on the type of Terraform resource and the cloud region they choose.

In the same folder, create a sub-folder called policy and create a file within called terraform.rego. Add the following code to that file:

terraform.rego file:

package terraform.analysis

import input as tfplan
import future.keywords

dev_env_cloud_prefix := "google-northamerica-northeast1"

prod_env_cloud_prefix := "aws-us-east"

resource_types := {"aiven_kafka", "aiven_pg", "aiven_opensearch", "aiven_redis"}

default allow_dev_deployment := false

default allow_prod_deployment := false

allow_dev_deployment if {
    some resource in tfplan.planned_values.root_module.resources
    resource.type in resource_types
    startswith(resource.values.cloud_name, dev_env_cloud_prefix)
}

allow_prod_deployment if {
    some resource in tfplan.planned_values.root_module.resources
    resource.type in resource_types
    startswith(resource.values.cloud_name, prod_env_cloud_prefix)
}
Enter fullscreen mode Exit fullscreen mode

Let's analyze this file. Crab Inc. uses PostgreSQL for their relational database, Redis for caching, OpenSearch for search and analytics, and Apache Kafka as the central message bus. Therefore, the resource_types field limits the use to these four resources. Crab Inc. allows developers to deploy in the Montreal, Canada region only. The dev_env_cloud_prefix field takes care of that requirement. Similarly, production deployments are allowed at any one of Aiven's supported AWS cloud regions in the US East coast which the prod_env_cloud_prefix field takes care of.

Execute the following command from the main directory to find out if OPA allows the Terraform deployment to go through:

./opa exec --decision terraform/analysis/allow_prod_deployment --bundle policy/ tfplan.json
Enter fullscreen mode Exit fullscreen mode

With the current Terraform service definition, the output of the above command is:

{
  "result": [
    {
      "path": "tfplan.json",
      "result": false
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

If you have jq installed on your machine, you can find the exact result with:

./opa exec --decision terraform/analysis/allow_prod_deployment --bundle policy/ tfplan.json | jq '.result[0].result'
Enter fullscreen mode Exit fullscreen mode

The opa exec command is taking in the tfplan.json as an input and validating this against the policy we defined in the allow_prod_deployment section under policy/terraform.rego file. terraform/analysis is denoting the package name in that Rego.

Let's make a change in the services.tf file and change the cloud_name field to aws-us-east1. Now if you repeat the previous steps to create the tfplan.json and run opa exec command, the output should be true.

hint:

terraform plan --out tfplan.binary
terraform show -json tfplan.binary | jq > tfplan.json
opa exec --decision terraform/analysis/allow_prod_deployment --bundle ./policy tfplan.json | jq '.result[0].result'
Enter fullscreen mode Exit fullscreen mode

Create data block in Rego

Rapu has done a great job implementing his first policy. However, typically data isn't hard coded in the policy. So let's rewrite the current policy and create some news ones.

Create the following data.json file in the policy folder this should be right next to our rego file:

Filename data.json:

{
    "team": "devrel",
    "app": "crab_cast",
    "dev": {
        "cloud": "google-northamerica-northeast1"
    },
    "prod": {
        "cloud": "aws-us-east1"
    }
}
Enter fullscreen mode Exit fullscreen mode

Now that we have a data file, we can go back and update our Rego policy.

file name terraform.rego:

package terraform.analysis

import input as tfplan
import future.keywords

# notice we removed the hard coded variables

resource_types := {"aiven_kafka", "aiven_pg", "aiven_opensearch", "aiven_redis"}

default allow_dev_deployment := false

default allow_prod_deployment := false

allow_dev_deployment if {
    some resource in tfplan.planned_values.root_module.resources
    resource.type in resource_types
    startswith(resource.values.cloud_name, data.dev.cloud)  # referencing the new data block
}

allow_prod_deployment if {
    some resource in tfplan.planned_values.root_module.resources
    resource.type in resource_types
    startswith(resource.values.cloud_name, data.prod.cloud)  # referencing the new data block
}
Enter fullscreen mode Exit fullscreen mode

Now we can rerun the policy check. Remember we added the data file to our policy folder so OPA should be aware of the new data.

terraform plan --out tfplan.binary
terraform show -json tfplan.binary | jq > tfplan.json
opa exec --decision terraform/analysis/allow_prod_deployment --bundle ./policy tfplan.json
Enter fullscreen mode Exit fullscreen mode

Unit testing in Rego

Now that we have a few policies in place, we are going to add unit tests to ensure our policies are good before we enforce them in production.

Create a rego file for our tests.

file policy/test_terraform.rego:

package terraform.test_analysis

import data.terraform.analysis

test_allow_dev_deployment {
    analysis.allow_dev_deployment with input as {"planned_values": {"root_module": {"resources": [{
        "address": "aiven_redis.redis-demo",
        "mode": "managed",
        "type": "aiven_redis",
        "name": "redis-demo",
        "provider_name": "registry.terraform.io/aiven/aiven",
        "schema_version": 1,
        "values": {
            "cloud_name": "google-northamerica-northeast1",
            "plan": "hobbyist",
            "project": "devrel-dewan",
            "service_name": "redis-demo",
            "service_type": "redis",
        },
    }]}}}
}

test_not_allow_prod_deployment {
    not analysis.allow_prod_deployment with input as {"planned_values": {"root_module": {"resources": [{
        "address": "aiven_redis.redis-demo",
        "mode": "managed",
        "type": "aiven_redis",
        "name": "redis-demo",
        "provider_name": "registry.terraform.io/aiven/aiven",
        "schema_version": 1,
        "values": {
            "cloud_name": "google-northamerica-northeast1",
            "plan": "hobbyist",
            "project": "devrel-dewan",
            "service_name": "redis-demo",
            "service_type": "redis",
        },
    }]}}}
}
Enter fullscreen mode Exit fullscreen mode

With our testing file in place let's run the tests. In this command we are calling the OPA binary, with the subcommand test on the target folder policy.

opa test policy
Enter fullscreen mode Exit fullscreen mode

Some homework for you

Now that you have learned about writing and testing OPA policies for your data infrastructure in place, please write the following two policies and two unit tests:

Policies to add

  1. project must start with team name
  2. service_name must include app name

Unit tests to add

  1. test that Aiven project name must contain team name
  2. test that Aiven service name must contain app name

If you need a hint or two, take a look at the solutions.

Great job helping Rapu succeed

Let's look at all the things Rapu has accomplished on his first project at Crab, Inc.

  • Created an Aiven Terraform file
  • Converted the Terraform plan into binary
  • Converted the binary output into JSON
  • Created a Rego policy to verify the resource configuration
  • Tested our Rego policy on our local CLI
  • Cleaned up our Rego policy by moving hard coded data to a data file
  • Wrote addition policies to very more specifics of the resources
  • Added unit tests for each of our policies

Thanks for spending the time learning with us today. Here are some additional resources to help you learn about the tools in this tutorial.

Link Description
Aiven Docs The Aiven docs will help you get unblocked using any Aiven resources
Terraform Docs The Terraform docs are a great reference to get started with Terraform
Rego Playground The Rego playground is available to interactively test your rego policies
OPA Docs The OPA docs are extensively written to answer any questions you may have

If you have any question on this tutorial, please check out the FAQ page, ask on Aiven community forum, or raise an issue.

💖 💪 🙅 🚩
dewanahmed
Dewan Ahmed

Posted on July 5, 2023

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Enforce fine-grained policy control across your data infrastructure with Open Policy Agent and Terraform
terraform Enforce fine-grained policy control across your data infrastructure with Open Policy Agent and Terraform

July 5, 2023