Learn how to configure OIDC provider with EKS and how to create IRSA. Source code is available in github.

Overview

In this blog, you will learn how to configure EKS cluster, OpenID connect (OIDC), IAM roles, and Kubernetes Service accounts using OpenTofu/Terraform. I will mainly use this configuration for granting permissions to read, write and list objects in my S3 bucket, but you can use this configuration to create your own IAM roles and policies for any Amazon service.

We will basically follow these five steps to achieve this:

1. Create an IAM OpenID Connect(OIDC) provider for your cluster.
2. Create IAM policy with permissions to write objects to your S3 bucket.
3. Create IAM role to access this policy.
4. Attach the IAM role and policy.
5. Use a Service Account to securely grant your pods AWS access via an IAM role.

OIDC provider configuration

OpenID Connect (OIDC) is used to securely connect your Kubernetes Service Account with an AWS IAM role. This allows your pods to assume the IAM role and access AWS services like S3 without needing AWS credentials directly inside the pods.

I created this OIDC provider using OpenTofu configuration file for my EKS cluster:

data "tls_certificate" "eks" {
  url = aws_eks_cluster.[your_cluster_name].identity[0].oidc[0].issuer
}

resource "aws_iam_openid_connect_provider" "eks" {
  client_id_list  = ["sts.amazonaws.com"]
  thumbprint_list = [data.tls_certificate.eks.certificates[0].sha1_fingerprint]
  url             = aws_eks_cluster.[your_cluster_name].identity[0].oidc[0].issuer
}

• Replace [your_cluster_name] with your actual cluster name.

Create IAM policy

The IAM policy determines what your pods can do on AWS resources (e.g., access S3) when they assume the associated IAM role.

I created the necessary IAM policy using OpenTofu to access my S3 bucket:

resource "aws_iam_policy" "eks_s3_access_policy" {
  name        = "eks-s3-access-policy"
  policy      = <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::[your_bucket_name]"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::[your_bucket_name]/*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:PutObjectAcl",
            "Resource": "arn:aws:s3:::[your_bucket_name]/*"
        }

            ]
}
EOF
}

• Replace [your_bucket_name] with your actual bucket name.

Create an IAM role

The IAM role enables your Kubernetes pods to securely access AWS services by assuming the role through the ServiceAccount, inheriting the permissions specified in the attached IAM policy which we created before this step.

Creating an IAM role:

resource "aws_iam_role" "eks_s3_access" {
  name = "eks-s3-access"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "${aws_iam_openid_connect_provider.eks.arn}"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${replace(aws_iam_openid_connect_provider.eks.url, "https://", "")}:sub": "system:serviceaccount:default:s3-access-sa"
        }
      }
    }
  ]
}
EOF
}

AWS IAM supports federated identities using OIDC. This feature allows us to authenticate AWS API calls with supported identity providers and receive a valid OIDC JSON web token (JWT). We are passing this token to the AWS STS AssumeRoleWithWebIdentity API operation and receive IAM temporary role credentials. Such credentials can be used to communicate with services likes Amazon S3 and DynamoDB.

Attach the policy to the IAM role

I have created a OpenTofu configuration file to create this policy and role attachment for the ServiceAccount to use:

resource "aws_iam_role_policy_attachment" "eks_s3_access_policy_attachment" {
  role       = aws_iam_role.eks_s3_access.name
  policy_arn = aws_iam_policy.eks_s3_access_policy.arn
}

Finally, we will create a ServiceAccount in our cluster which serves as the Kubernetes identity that allows your pods to securely assume the AWS IAM role. This enables the pods to access AWS resources like S3 with the permissions defined in the IAM policy, without requiring AWS credentials to be manually managed or exposed within the cluster.

Create ServiceAccount

The Service Account in this setup is crucial for securely allowing Kubernetes pods to assume an AWS IAM role, enabling them to access AWS resources, such as S3, without needing to embed AWS credentials directly in the pods.

Here is YAML file I used to create the ServiceAccount in my cluster:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: s3-access-sa
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::[AWS account ID]:role/eks-s3-access

Replace the [AWS account ID] with your actual account ID.

The eks.amazonaws.com/role-arn: arn:aws:iam::[AWS account ID]:role/eks-s3-access is used to map your Kubernetes Service Account to the AWS IAM role.

Now, you have got the necessary permissions to give access to a Workload running inside the cluster to resources outside of the cluster such as the S3 bucket.

Conclusion

In this article, we explored how to securely grant a Kubernetes pod access to AWS resources using IAM roles. This approach is particularly useful for scenarios where workloads running inside the cluster need to interact with external AWS services.

Based on my experience, getting it up and running wasn't a quick task. However, I hope this guide helps others save time and avoid the same challenges.