Credentials substitution at runtime in Python

Introduction

Often we hear that credentials should not be hardcoded in the code or Version Control (like git). You can leverage the dynamic substitution of such credentials using Environment variables tied to the code as a variable.

Pre-requisite

Python 3.8 or up
PyYaml - To read YAML configuration file

Steps

Create a yaml file as the following

# credentials.yml

aws:
  access_key_id: "${AWS_ACCESS_KEY_ID}"
  secret_access_key: "${AWS_SECRET_ACCESS_KEY}"

We will read this yaml file and get the python dictionary object

# main.py
from typing import Dict
import yaml

def read_credentials_file() -> Dict:
  try:
    with open("credentials.yml", mode="r", encoding="utf-8") as cred_file:
      return yaml.safe_load(cred_file)
  except yaml.YAMLError as yaml_err:
    print(f"Unable to read the file. Error: {yaml_err}")
    raise

We will use the string.Template from the standard library to substitute the value from the environment variables

# main.py
import string
import os

def substitute_creds(creds: Dict) -> None:
  creds["aws"]["access_key_id"] = string.Template(
    creds["aws"]["access_key_id"]
  ).substitute(
    {"AWS_ACCESS_KEY_ID": os.getenv("AWS_ACCESS_KEY_ID")}
  )

  creds["aws"]["secret_access_key"] = string.Template(
    creds["aws"]["secret_access_key"]
  ).substitute(
    {"AWS_SECRET_ACCESS_KEY": os.getenv("AWS_SECRET_ACCESS_KEY")}
  )

Add the following two variables as part of the environment. They can be added in the .env file as well

$ export AWS_ACCESS_KEY_ID="abcd1234"
$ export AWS_SECRET_ACCESS_KEY="secret123"

Complete Program

# main.py
from typing import Dict
import string
import yaml
import os

def read_credentials_file() -> Dict:
  try:
    with open("credentials.yml", mode="r", encoding="utf-8") as cred_file:
      return yaml.safe_load(cred_file)
  except yaml.YAMLError as yaml_err:
    print(f"Unable to read the file. Error: {yaml_err}")
    raise

def substitute_creds(creds: Dict) -> None:
  creds["aws"]["access_key_id"] = string.Template(
    creds["aws"]["access_key_id"]
  ).substitute(
    {"AWS_ACCESS_KEY_ID": os.getenv("AWS_ACCESS_KEY_ID")}
  )

  creds["aws"]["secret_access_key"] = string.Template(
    creds["aws"]["secret_access_key"]
  ).substitute(
    {"AWS_SECRET_ACCESS_KEY": os.getenv("AWS_SECRET_ACCESS_KEY")}
  )

if __name__ == "__main__":
  credentials = read_credentials_file()
  substitute_creds(credentials)

  # WARNING: Never print or log credentials or any sensitive information
  print(f"Credentials: {credentials}")

The above code when run should produce the following output

$ python3 main.py
Credentials: {"aws": {"access_key_id": "abcd1234", "secret_access_key": "secret123"}}

Blog

Credentials substitution at runtime in Python

Syed Mohammad Ibrahim

Introduction

Pre-requisite

Steps

Complete Program

Join Our Newsletter. No Spam, Only the good stuff.

Related