What is AWS Roles Anywhere?

AWS Roles Anywhere enables you to use AWS Policies and AWS Roles for workloads such as servers, containers and applications that are running outside AWS without having to create long-term credentials.

AWS Roles Anywhere uses SSL certificates to manage workload access through IAM Roles and handing over short-living session keys.

How does AWS Roles Anywhere work?

A workload needs to have an SSL certificate (and a private key) that is signed by a Certificate Authority configured in AWS Roles Anywhere as a Trust Anchor. It is possible to configure external CAs or to use an AWS Private CA.

AWS Roles Anywhere Profiles map which IAM Roles are possible to be assumed by the workload. The workload is then given a short-living session keys which grants access to AWS services according to the role policies.

This is more secure than if user credentials or other long-term credentials are used, as they usually have much more access than a single role. It's also possible to set boundaries for the roles added to the Profile which override the role's policies.

What use-cases does AWS Roles Anywhere have?

These roles can be used in any kind of workload running outside AWS that needs access to the resources in the cloud. For example,

• On-premises databases servers could connect to S3 to fetch data.

• If you use Kubernetes in a private cloud, you can create a role that gives the application container running in the cluster permissions to specific AWS resources only.

• If you already have an existing CA and PKI (Public-Key Infrastructure) system, another use case is to use AWS Roles Anywhere to use the existing certificates to grant access to AWS services.

Let's dive in deeper...

Some infrastructure is needed first

First, a Certificate Authority (CA) is needed. For this example, we will create an AWS Private CA and then create an AWS Roles Anywhere trust anchor for it.
After that, we will create an IAM Role with S3 read-only access and a Roles Anywhere Profile that links the IAM role to the trust anchor and CA.

We will not dive deep into these and concentrate more on the workload side. Terraform code used to create CA, trust anchor, profile, role and an S3 bucket for this example can be found in https://github.com/jpohjolainen/aws_roles_anywhere

Once run, Terraform will output the ARNs of the newly created CA, trust anchor, role and profile, and also a S3 bucket name.

These will be needed later on for creating a certificate for signing in to AWS and when revoking a certificate.

Speaking of certificates, let's create a new certificate request and get a certificate from the CA.

Note: Only the first private CA will be free for 30 days. If you create one and then delete it and create another, you will need to pay for the CA immediately. A private CA costs 300€ ($400) a month.

Note 2: It is also possible to create your own CA with an OpenSSL command and configure it using external CA. Here is how to do that: https://aws.amazon.com/blogs/security/iam-roles-anywhere-with-an-external-certificate-authority/

Creating a certificate and private key for our application

Create Certificate Signing Request (CSR) and a new Private Key.

Create CSR app-cert.csr and private key app-private-key. Change the Subject to your liking. C=Country, ST=State, OU=Organization Unit, O=Organization and CN=Common Name (name of app, or hostname/domain)

openssl req -new -newkey rsa:2048 \
 -out "app-cert.csr" \
 -keyout "app-private.key" \
 -subj "/C=DE/ST=Berlin/OU=DevOps/CN=app1"

The previous command prompted for password for the private key, but we need to remove it as this is for an application

openssl rsa -in "app-private.key" -out "app-private-nopass.key"

Request a certificate from the CA based on the CSR.

aws acm-pca issue-certificate \
 --certificate-authority "arn:aws:acm-pca:eu-west-1:xxxxxx:certificate-authority/zzzzzzzzz" \
 --csr "fileb://app-cert.csr" \
 --signing-algorithm "SHA256WITHRSA" \
 --validity Value=365,Type="DAYS"

The --csr option really has fileb:// instead of file://. It is used to read files in binary format in the AWS CLI.

Note: The following signing algorithms are available: SHA256WITHECDSA, SHA384WITHECDSA, SHA512WITHECDSA, SHA256WITHRSA, SHA384WITHRSA and SHA512WITHRSA.
The specified signing algorithm family (RSA or ECDSA) must match the algorithm family of the CA's secret key.

The validity type can be YEARS, MONTHS, DAYS and the number as the value, END_DATE with value of YYYYMMDDHHMMSS or ABSOLUTE with a unix timestamp as the value.

Get the CertificateARN from the reply. It is used in next section.

Download the issued certificate.

Use the AWS CLI to download the certificate from the private CA

aws acm-pca get-certificate \
 --certificate-arn "arn:aws:acm-pca:eu-west-1:xxxxxx:certificate-authority/zzzzzzzzz/certificate/af7d3bf5c562a7d91f9310da8ae6ea8d" \
 --certificate-authority-arn "arn:aws:acm-pca:eu-west-1:xxxxxx:certificate-authority/zzzzzzzzz" \
 --output json \
 |jq -r '.Certificate, .CertificateChain' > app-cert.pem

Note: This uses jq to get the certificate in JSON format. It's possible to use --output text and direct that to a file, but then you need to edit the file to move the second -----BEGIN CERTIFICATE----- to its own line.

Show the certificate

openssl x509 -in app-cert.pem -noout -text

Creating a container with an application

Once we have the certificate and the private key created, we need to have a configuration for AWS.
$HOME/.aws/config is used by AWS SDK and CLI for getting access to AWS by signing in with the certificate and assume the role.

AWS provides a tool to help with the sign-in called AWS Signing Helper. In this example, it is downloaded inside the Dockerfile when building the image.
https://docs.aws.amazon.com/rolesanywhere/latest/userguide/credential-helper.html

.aws/config

The helper tool can then be used in the file $HOME/.aws/config to login to the AWS when SDK or CLI is used. Here we need the ARNs that the Terraform code above returns. Save this to a file called aws-config. The Dockerfile expects this name:

[default]
    credential_process = /usr/local/bin/aws_signing_helper credential-process --certificate /app/app-cert.pem --private-key /app/app-private-nopass.key --trust-anchor-arn arn:aws:rolesanywhere:eu-west-1:xxxxxx:trust-anchor/yyyyyyyy --profile-arn arn:aws:rolesanywhere:eu-west-1:xxxxxx:profile/ccccccc --role-arn arn:aws:iam::xxxxxx:role/RolesAnywhere

Note: Change the --trust-anchor-arn, --profile-arn and --role-arn to values gotten from Terraform code.

Small application

Here is a small Python code to print S3 buckets. Save this to a file gets3buckets.py

import boto3
import time

def hello_s3():
    s3_resource = boto3.resource("s3")
    print("Hello, Amazon S3! Let's list your buckets:")
    for bucket in s3_resource.buckets.all():
        print(f"\t{bucket.name}")


if __name__ == "__main__":
    while True:
        hello_s3()
        time.sleep(5)

Copy the certificate, private key, aws-config and the above python code into a directory.

Dockerfile

Create a Dockerfile in the same directory as the certificates and the other files previously created:

FROM debian:stable-slim

ARG homedir=/app

RUN DEBIAN_FRONTEND=noninteractive apt-get update \
    && apt-get upgrade \
    && apt-get install --no-install-recommends -y \
        awscli \
        curl \
        python3-boto3 \
    && rm -rf /var/lib/apt/lists/*

# Download AWS Signing Helper
RUN cd /usr/local/bin \
    && curl -LO https://rolesanywhere.amazonaws.com/releases/1.1.1/X86_64/Linux/aws_signing_helper \
    && chmod 0755 aws_signing_helper

# Create user to run the app
RUN adduser --system --home "$homedir" --no-create-home --shell /bin/false userapp
RUN mkdir "$homedir" && chown userapp "$homedir"

# After this everything is run under the user
USER userapp

COPY --chown=userapp --chmod=0600 ./app-cert.pem "$homedir"
# This should never be copied inside the image. It should be mounted from outside
COPY --chown=userapp --chmod=0600 ./app-private-nopass.key "$homedir"


RUN mkdir "$homedir"/.aws \
    && chmod 0700 "$homedir"/.aws

# Copy the aws-config 
COPY --chown=userapp --chmod=0644 ./aws-config "$homedir"/.aws/config

COPY --chown=userapp --chmod=0755 gets3buckets.py "$homedir"

WORKDIR "$homedir"

CMD python3 /app/gets3buckets.py

Note: The private key should never be baked into the container except when testing locally. The private key should be in a secret store like AWS Secrets Manager and then mounted from there when using inside Kubernetes.

Build the container

docker build -t s3rolesanywhere:test . 

When running the container, it will use the certificate and the private key with the aws_signing_helper tool to request AWS secrets and session keys, and assume the role. It will then print the buckets from S3 every 5 seconds.

docker run -ti --rm s3rolesanywhere:test 

The output should be something like this:

Hello, Amazon S3! Let's list your buckets:
    private-ca-crl-xxxxxxxx
    ...

Revoking certificate

Get certificate serial number

To revoke a certificate, you need to have its Serial number. You can get it from the certificate with openssl command

openssl x509 -in app-cert.pem -noout -serial 

Revoke certificate in AWS Private CA

Then, you can revoke the certficate in AWS Private CA with the following command

aws acm-pca revoke-certificate \
 --certificate-authority-arn <ARN of CA> \
 --certificate-serial <Serial of the cert to revoke> \
 --revocation-reason "<Reason for revoking>"

Note: Only these are valid values for --revocation-reason: AFFILIATION_CHANGED, CESSATION_OF_OPERATION, A_A_COMPROMISE, PRIVILEGE_WITHDRAWN, SUPERSEDED, UNSPECIFIED, KEY_COMPROMISE, CERTIFICATE_AUTHORITY_COMPROMISE

It may take some time for the CRL file to appear.

Download AWS Private CA CRL file

You can then download the CRL from the S3 bucket configured for CRL in the AWS Private CA.

List CRL files in the Private CA S3 Bucket (private_ca_s3_bucket is the output from Terraform)

aws s3 ls s3://<private_ca_s3_bucket>/crl

Get the CRL file from the S3 bucket

aws s3 cp s3://<private_ca_s3_bucket>/crl/xxxxxxxxx.crl .

This file is in DER format, and it needs to be in PEM format for AWS Roles Anywhere to accept.

openssl crl -inform DER -in xxxxxxxx.crl -outform PEM -out privateca.crl

Import CRL to AWS Roles Anywhere

A CRL file in PEM format needs to be then uploaded/imported to the AWS Roles Anywhere for it to revoke the access to the certificate

aws rolesanywhere import-crl \
 --crl-data privateca.crl \
 --name <give some name> \
 --trust-anchor-arn <ARN of the Turst Anchor> --enabled

Note: You can only import 2 CRLs. Even with same names it doesn't overwrite them, so you need to manually remove them. Check aws rolesanywhere list-crls and aws rolesanywhere delete-crl AWS CLI commands.

After some time, the access with that certificate should not work anymore. The error will be something like

botocore.exceptions.CredentialRetrievalError: Error when 
retrieving credentials from custom-process: 
2024/02/16 11:11:34 AccessDeniedException: Certificate revoked

Conclusion

It is a pretty neat way of giving AWS access to applications and servers outside of AWS. It is more secure than creating normal, long-term user credentials and using them for applications. Compared to normal user credentials, if the session is hijacked in transit, then it is only valid for a short while instead of potentially being live for even months or years. And if the private key is compromised, then it can only access the services configured with policies to the role.

But revoking certificates is not all that straightforward. The CA keeps up a list called Certificate Revocation List (CRL). This needs to be imported to AWS Roles Anywhere seperately. AWS Roles Anywhere doesn’t have any automated update system of the CRL, even with AWS Private CA that can push the CRL to S3. It can only be imported through AWS CLI or API, but for example Terraform does not yet have that capability.

I feel that unless a company is already heavily invested in certificates and PKI systems, the AWS Roles Anywhere brings complications not needed on top of all other user management, especially as Terraform doesn't support import-crl now, and playing with AWS Private CA or OpenSSL requires more manual work than I think is necessary.