This post looks at how a native K8s service account can be used along with an IAM role as a strategy for managing AWS credentials for applications running in EKS.

This use case arises when an application running in an EKS pod wants to use the AWS SDK to call some AWS service. Requests must be signed with an AWS credential. The SDK will use the default credential provider chain to identify a suitable credential to use. In this scenario, we want to leverage the AWS_WEB_IDENTITY_TOKEN_FILE credential. The following paragraphs look at the steps required to surface that credential, so we can connect to and read from an S3 bucket.

We will take advantage of EKS's built-in support for using AWS IAM user and roles as entities for authenticating against a cluster. The first step is to create an IAM OIDC Identity provider using the OpenID Connect provider URL, which is automatically provided during cluster creation.

Create IAM OIDC provider for the EKS cluster

The OpenID Connect provider URL is available under the EKS dashboard under tabs: Configuration - Details
The provider URL can be copied and used to create a new Identity Provider in IAM.

• Provider Type - choose OpenID Connect
• Provider URL - paste the OIDC URL from your cluster
• Audience - sts.amazonaws.com If a provider already exists matching the name of your clusters' provider URL, then there is no need to continue with this step.

Create IAM Role for the EKS Service Account

In this step, we create an IAM policy which specifies the permissions our container will need in order to connect to and read from an S3 bucket. Once the policy is created, we require a new role against which the policy will be attached. The following snippet provides a sample policy file which grants permissions to read from a specific S3 bucket.

IAM Policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::my-s3-bucket/*"
      ]
    }
  ]
}

Create a new IAM role. Attach the previously described IAM policy. Edit the trust policy and add the following statement;

IAM Role for Service Account

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::<aws account>:oidc-provider/oidc.eks.<region>.amazonaws.com/id/<oidc provider ID>"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.<region>.amazonaws.com/id/<oidc provider ID>:sub": "system:serviceaccount:<cluster namespace>:<service account name>"
                }
            }
        }
    ]
}

• OIDC provider ID = OIDC provider URL for your cluster
• Region = AWS region where your cluster is deployed
• Cluster namespace = Namespace where your EKS service account lives
• Service account name = Associate the Role with a service account in our EKS cluster

Detailed instructions for creation of both IAM Policy and Role are available at EKS Documentation

Associate IAM Role with K8s Service Account

Add the following annotation to your k8s service account definition to associate the newly created IAM Role

apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<aws account>:role/eks-s3-role
    eks.amazonaws.com/sts-regional-endpoints: "true"
  name: my-serviceaccount
  namespace: my-cluster-namespace

The service account is also annotated to use the AWS Security Token Service Regional endpoint rather than the global endpoint to reduce latency, build in redundancy and increase session token validity.

The service account is associated with pods running in your namespace through the _serviceAccountName _directive in your deployment definition.

spec:
      serviceAccountName: my-serviceaccount
      containers:
      ....

The next time EKS attempts to schedule a pod with this definition, it will trigger a mutating webhook, which is responsible for writing the following environment variables to our pod

• AWS_WEB_IDENTITY_TOKEN_FILE
• AWS_ROLE_ARN

We can verify the environment variables exist by running the command:
kubectl exec -n <namespace> <pod-name>-- env | grep AWS


Testing S3 Access

Once we have verified the presence of the AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN environment variables, our pod is ready to connect to and read from S3. The following Typescript snippet illustrates a simple example which fetches the names of all the S3 buckets in our account. It uses the AWS SDK for JavaScript V3

import {

  ListBucketsCommand,

  ListBucketsCommandOutput,

  S3Client,

} from '@aws-sdk/client-s3';

export const s3API = (): Client=> {

  return {

    listBuckets: async () => {

      let bucketNames: string[] = [];

      const s3Client = new S3Client(

          {region: "eu-west-1"}

      );

      let data: ListBucketsCommandOutput;

      try {

        data = await s3Client.send(new ListBucketsCommand({}));

        data.Buckets.forEach((bucket) => {

          bucketNames.push(bucket.Name);

        });

      } catch (err) {

        console.log('Error', err);

      }

      return bucketNames;

    },

Summary

By using IAM Roles with k8s native service accounts, we obviate the need to provide extended permissions to the EKS node IAM Role. This allows us to follow the principle of least privilege. We can scope IAM permissions for each service account, ensuring containers only have access to those privileges needed to complete its task.

References:

• Creating an IAM OIDC provider https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html

• EKS Workshop IAM Roles for Service Accounts https://www.eksworkshop.com/beginner/110_irsa/oidc-provider/

• Create IAM Role for Service Account https://docs.aws.amazon.com/eks/latest/userguide/create-service-account-iam-policy-and-role.html

• Associate IAM role to Service Account https://docs.aws.amazon.com/eks/latest/userguide/specify-service-account-role.html

• AWS SDK for JavaScript v3 https://github.com/aws/aws-sdk-js-v3