The Curse of The Hour

Session management in AWS is complicated, especially when authenticating with IAM roles. A common way to obtain AWS credentials is to assume an IAM role and be given a set of temporary session keys that are only good for a certain period of time. The maximum session duration is a setting on the IAM role itself, and it is one hour by default. So if users don't specify a value for the DurationSeconds parameter, their security credentials are valid for only one hour.

A typical boto3 request to assume IAM role looks like:

response = client.assume_role(
    RoleArn='string',
    RoleSessionName='string',
    Policy='string',
    DurationSeconds=123,
    ExternalId='string',
    SerialNumber='string',
    TokenCode='string'
)

where DurationSeconds is the duration of the role session. It can go up to the maximum session duration setting for the role. So if your IAM role is only setup to go up to an hour, you wouldn't be able to extend the duration of your sessions unless you update the settings on the IAM role itself.

Long-Lasting Credentials

So an alternative must be introduced to extend IAM role sessions. This is when I found RefreshableCredentials , a botocore class acting like a container for credentials needed to authenticate requests. Moreover, it can automatically refresh the credentials! This is exactly what I need. But it's usage is poorly documented. Looking at its source code:

def __init__(self, access_key, secret_key, token, expiry_time, refresh_using, method, time_fetcher=_local_now):

The __init__ function takes several arguments, half of which I don't recognize. But there's a class method that can be used to initialize an object:

@classmethod
def create_from_metadata(cls, metadata, refresh_using, method):
    instance = cls(
         access_key=metadata['access_key'],
         secret_key=metadata['secret_key'],
         token=metadata['token'],
         expiry_time=cls._expiry_datetime(metadata['expiry_time']),
         method=method,
         refresh_using=refresh_using)
    return instance

where metadata is a dictionary containing information abound the current session, ie. access_key, secret_key, token, and expiry_time, all are things we can get from boto3's STS client's assume_role() request.

To construct the metadata response, we make a simple boto3 API call:

import boto3
sts_client = boto3.client("sts", region_name=aws_region)
params = {
    "RoleArn": self.role_name,
    "RoleSessionName": self.session_name,
    "DurationSeconds": 3600,
}
response = sts_client.assume_role(**params).get("Credentials")
metadata = {
    "access_key": response.get("AccessKeyId"),
    "secret_key": response.get("SecretAccessKey"),
    "token": response.get("SessionToken"),
    "expiry_time": response.get("Expiration").isoformat(),
}

refresh_using is a callable that returns a set of new credentials, taking the format of metadata. Remember that in Python, functions are first-class citizens. You can assign them to variables, store them in data structures, pass them as arguments to other functions, and even return them as values from other functions. So I just need a function that generates and returns metadata.

def _refresh(self):
    " Refresh tokens by calling assume_role again "
    params = {
        "RoleArn": self.role_name,
        "RoleSessionName": self.session_name,
        "DurationSeconds": 3600,
    }

    response = self.sts_client.assume_role(**params).get("Credentials")
    credentials = {
        "access_key": response.get("AccessKeyId"),
        "secret_key": response.get("SecretAccessKey"),
        "token": response.get("SessionToken"),
        "expiry_time": response.get("Expiration").isoformat(),
    }
    return credentials

Now we're ready to create a RefreshableCredentials object:

from botocore.credentials import RefreshableCredentials
session_credentials = RefreshableCredentials.create_from_metadata(
    metadata=self._refresh(),
    refresh_using=self._refresh,
    method="sts-assume-role",
)

and we can use the credentials to generate a IAM role session that lasts for as long as we need:

from boto3 import Session
from botocore.session import get_session
session = get_session()
session._credentials = session_credentials
session.set_config_variable("region", aws_region)
autorefresh_session = Session(botocore_session=session)

And of course we can generate a boto client within that session, ie.:

db_client = autorefresh_session.client("rds", region_name='us-east-1')