In this guide, we explore the essential elements of terraform destroy, unraveling why this command is a fundamental part of the Terraform workflow. Additionally, we will cover best practices and considerations to ensure the effective and safe execution of terraform destroy within your infrastructure management processes.

‍What is terraform destroy

Among the suite of Terraform commands, terraform destroy holds a crucial role in infrastructure management. This command is specifically used to remove the infrastructure that has been provisioned using Terraform Infrastructure-as-Code (IaC) configuration. 

When terraform destroy is executed, Terraform reviews the state file to identify and systematically remove managed infrastructure from your cloud environment (AWS, Azure, GCP, etc.). 

While each of the commands in the Terraform workflow (terraform init->terraform plan->terraform apply), contributes to creating a new infrastructure, terraform destroy command is explicitly used to delete all (or some targeted) infrastructure defined in your Terraform IaC. 

Unlike terraform apply, which brings resources up to date with the desired state, terraform destroy command does the opposite by ensuring that all (or some) of the resources managed by Terraform are deleted.

Before we dive in deeper, for additional context, here is a short description of Terraform commands and their functions:

• terraform init: Used to initialize a working directory containing Terraform config files and to download providers and modules.
• terraform validate: Validates the Terraform config in that particular directory to ensure they are syntactically valid and internally consistent.
• terraform plan: Creates an execution plan. Using this command prompts Terraform to perform a refresh and determine the actions necessary to achieve the desired state in specified config files.
• terraform apply: Used to apply the changes required to reach the desired configuration state. By default, the apply command scans the current directory for the config and applies the changes appropriately.

How does terraform destroy Command Works?

To demonstrate how the command works, let us take an example to help us understand terraform destroy more clearly. 

For that, we’ll spin up two EC2 instances, review the entire Terraform workflow, and assess the aftermath of terraform destroy.

#main.tf

provider "aws" {

  region = "us-west-1"

}

variable "instances" {

  default = {

    "env0-instance-az1 = "us-west-1b",

    "env0-instance-az2" = "us-west-1c"

  }

}

resource "aws_instance" "tf_instances" {

  for_each = var.instances

  ami           = "ami-0ce2cb35386fc22e9"

  instance_type = "t2.micro"

  availability_zone = each.value

  tags = {

    Name = each.key

  }

}

Terraform Workflow to Create Infrastructure

By running the Terraform workflow (terraform init-> terraform plan-> terraform apply), we were able to create env0-instance-az1 and env0-instance-az2 instances. They’re deployed in their respective availability zones.

Let us inspect the state file (terraform.tfstate) now, to check the metadata of our provisioned infrastructure (env0-instance-az1 and env0-instance-az2). We can see the information of both our instances in the state file.

Executing terraform destroy Command

After running terraform destroy, we can observe that our state refreshes and Terraform outputs the resources that will be destroyed from our cloud environment.

Like terraform apply, we are prompted to confirm these changes that will be made to our infrastructure. Confirm with a ‘yes’ to destroy the resources.

After the resources are deleted, Terraform updates our state (terraform.tfstate) file to reflect that these resources no longer exist.

When, Why and How to use terraform destroy?

Here are some scenarios when and why you might use terraform destroy:

• Cost Management: If you're running resources that are no longer needed or are underutilized, using terraform destroy can delete those resources and help reduce costs by ensuring you're not paying for what you don't use.
• Security Purposes: In cases where threat actors might have compromised your infrastructure or if you need to ensure that all resources are in a known secure state, destroying and re-provisioning can be an effective strategy.
• Resource Reallocation: In scenarios where infrastructure needs to be re-architected or reallocated for different projects or priorities, destroying the current setup might be necessary before reallocating those resources.

Using terraform destroy Options

Let us look at how we would use terraform destroy command with various options.

1. Terraform destroy -auto-approve

The -auto-approve flag always skips the interactive approval for confirming the changes for your infrastructure. In the case of terraform destroy, it bypasses the approval step, and Terraform proceeds to destroy your infrastructure.

In automated environments such as CI/CD pipelines, where human interaction is not possible, the -auto-approve flag ensures that the destroy operation can proceed without manual intervention.

2. Terraform destroy -target 

The terraform destroy -target=resource_type.resource_name option should be used with caution and in specific scenarios where you need to remove an individual resource or a set of resources, without impacting the entire infrastructure managed by Terraform.

3. Terraform destroy -refresh=false

The -refresh option in the terraform destroy command can be used to control whether Terraform should refresh the state before performing the terraform destroy.

For instance, in a large infrastructure environment, you can use -refresh=false option when the refresh process is time-consuming since Terraform checks the status of each resource in the cloud provider. 

This flag skips the state file refresh and speeds up the destroy process.

4. Terraform destroy -lock and -lock-timeout

The terraform destroy -lock flag is used to control Terraform's state-locking mechanism during the destroy operation.

Using the -lock=true can be beneficial when you are working in a team environment where multiple operations in the same state might occur simultaneously. 

This ensures that no other operations interfere with the destroy process, maintaining state consistency and preventing potential conflicts.

The -lock-timeout flag can be used to extend the waiting period for a state lock. For example, terraform destroy -lock-timeout=20s locks the state file for 20 seconds.

‍Best Practices

The destroy command is irreversible, and running it carelessly might cause important infrastructure to disappear forever from your environment. 

Before discussing best practices, it is important to note that terraform destroy can be run locally and in automated CI/CD pipelines. The terraform destroy command is typically used locally for dev or test environments to swiftly destroy resources, while in CI/CD pipelines, it's recommended for staging and production to ensure controlled, team-reviewed resource destruction.

Let us look at the best practices that must be followed when dealing with terraform destroy.

1. Regular Backup of State Files and Sensitive Data

Regularly backing up state files (terraform.tfstate), including the automatically generated terraform.tfstate.backup, is essential.

Terraform creates the .backup file before it writes to the main terraform.tfstate file, serving as an immediate previous version of your state. This can be invaluable in case of accidental corruption or deletion of the terraform.tfstate file.

When planning backups, ensure both the current state file and the .backup file are included.

For enhanced disaster recovery, consider replicating state files in remote backends (like S3) to other regions. Additionally, sensitive data or secrets (such as cloud provider API keys, SSH keys, database credentials, etc.) managed by Terraform should also be securely backed up and encrypted to prevent unauthorized access.

2. Using Workspaces for Environment Isolation

Manage separate state files for different environments (such as development, staging, and production) under the same configuration using Terraform workspaces. 

Utilizing workspaces for environment isolation is a best practice that can significantly reduce the risk of impacting the wrong environment when performing destructive operations like terraform destroy. 

Before executing the destroy command, always confirm you are in the correct workspace to ensure that only the intended environment is affected. 

3. Implement a Review Process for Infrastructure

Implementing a comprehensive review process by incorporating a gated check-in process in CI/CD pipelines adds an additional layer of security and accountability.

This process should include:

• Code Review: Changes to Terraform configurations, especially those leading to resource destruction, should be reviewed by multiple team members.

• Impact Analysis: Automatically run terraform plan -destroy to show what resources will be affected without making any changes. Document and review infrastructure changes.

• Environment-specific gates: Enforce stricter reviews and require additional checks or tests that mimic the production environment for the staging environment. For production, implement the highest level of scrutiny by senior infrastructure engineers and double-check automated tests.

• Audit Trails: Ensure that all actions and approvals related to the Terraform changes are logged and accessible for audit purposes.

What Happens when terraform destroy Fails?

When terraform destroy fails, it can lead to several issues:

1. Partial Resource Deletion

This can occur if Terraform encounters an error while attempting to destroy one or more resources. The error could be due to permissions issues, dependencies between resources that Terraform is not aware of, or external factors such as network timeouts.

‍Mitigation:

• Resolve dependencies: Manually review and ensure all dependencies are correctly reflected in your Terraform configurations. Use -target flag to destroy specific resources.
• Check Permissions: Verify that you have sufficient RBAC permissions to delete Terraform resources, in case you're using Terraform Cloud or other platforms.
• Network Timeouts: In case of network timeouts, re-running the terraform destroy command with a stable network may resolve the problem.

‍2. Inconsistent State

The Terraform state file may become inconsistent with the actual infrastructure if manual changes were made outside Terraform, or if previous Terraform operations were interrupted.

‍Mitigation:

• Refresh State: Run terraform refresh to update the local state file with the actual state of resources in the cloud.
• Manual State Edit: If refreshing the state does not resolve the inconsistencies, manually edit the state file using the terraform state rm command to remove the problematic resources from the state file.

3.State Locking Issues

Terraform state locking prevents multiple simultaneous operations from corrupting the state. If a previous operation didn't complete properly, the state may remain locked, blocking further operations.

‍Mitigation:‍

• Investigate Lock: Use the -lock-timeout option to wait for a lock to be released before or use terraform force-unlock with the lock ID, ensuring no other operations are currently running.
• Manual Unlock: As a last resort, manually unlock the state through the backend (like S3, Azure Blob Storage, etc.) if the automatic unlock fails.

env0: Scheduling, TTL, Destroy Protection and more

‍Time to Live (TTL) is one of the functionalities env0's offers, which uses destroy to streamline environment lifecycle management. 

Configuring TTL provides users with the ability to easily set predefined timers, dictating the lifespan of their environments, which helps optimize resource allocation and avoid cost sprawl. 

Additionally, env0 also allows you to set Organizational- and Project-level TTL. These can be used to provide helpful guardrails, standardize IaC usage, and grant autonomy to teams new to Infrastructure-as-Code, without the risk of things going south.

Moreover, env0’s Scheduling functionality aids a layer of automation, triggering of infrastructure deployments and destructions on a predefined schedule, defined by cron expressions. 

To avoid accidental deletion, env0 also comes with a Destroy Protection option. When enabled, it restricts functionalities such as the ‘_Destroy’_ button, ‘Time Left’ indication, and the ‘TTL panel’ to be applied to the protected environment.

Lastly, the Skip State Refresh feature in env0 allows users to bypass state mismatches during the destruction phase of an environment. 

This is a last-resort option equivalent to the Terraform command, terraform plan -refresh=false, ensuring that the environment can still be destroyed even when state mismatches occur, such as inaccessible secrets or data sources.

Frequently Asked Questions

Q. What is the difference between terraform destroy and state rm?

To answer this question, here is a short comparison table:

Q. Is Terraform destroy reversible?

No. Once executed, terraform destroy is not reversible. It permanently deletes resources from the cloud.

Q. How can I prevent accidental execution of terraform destroy?

It's recommended to use Terraform state locking. Another alternative is to use the prevent_destroy lifecycle meta-argument. When prevent_destroy is set to true, Terraform will generate an error if the destroy operation tries to destroy the resources, acting as a safeguard against accidental deletion.

Q. Does terraform destroy delete manually created resources?

No. The terraform destroy command just deletes the resources that are under Terraform’s management and control.

Q. Can I preview what terraform destroy will do before executing it?

Yes, you can execute terraform plan -destroy for a preview of the actions Terraform will take, allowing you to verify that only the intended resources are targeted for deletion.