How to Authenticate using Keys, BasicAuth, OAuth2 in Python

Not all APIs are as well documented as Twilio. This guide should help you work with APIs which are secured using Keys, BasicAuth, or OAuth.

We will be working with the following APIS

You can find the source code here

Insecure APIs
Reading values from.env files
APIs with Keys
APIs with Basic Auth
API Wrappers
The Session Object
APIs secured via OAuth2
Using the GitHub API (OAuth2)
Using the Genius API (OAuth2)

Some familiarity with the requests library is expected. If you need a refresher, you can refer to my previous article.

Insecure APIs

The Cat Facts API does not require any authentication and is fairly straightforward to work with. Let's make a request to the following endpoint

https://cat-fact.herokuapp.com/facts

The above API returns random Cat Facts

import requests
api_endpoint = "https://cat-fact.herokuapp.com/facts"
response = requests.get(
    api_endpoint
)
for idx, item in enumerate(response.json()):
    print(f"{idx+1}. {item['text']}")

Reading from .env files

Before moving on to the next sections, let's look at how to read variables from a .env file. It's highly recommended to store your credentials in a .env file to avoid them being exposed to others.

We will need to install the python-dotenv library.

pip install python-dotenv

Assume have a .env file with some random API Token

API_TOKEN = "SOME API TOKEN"

Let's try reading the API Token in Python.

from dotenv import load_dotenv
import os 

load_dotenv()
API_TOKEN = os.environ.get("API_TOKEN")

The get function accepts a variable name stored in the .env file as an argument.

APIs with Keys

This is the most common form of authentication when consuming APIs. The API Key/Token is passed in as a header while making the request. We will be working with the Cat as a Service (CAAS) API. You can get a key here

from dotenv import load_dotenv
import os 
import requests

api_endpoint = "https://api.thecatapi.com/v1/breeds"

load_dotenv()
CAT_API_KEY = os.environ.get("CAT_API_KEY")

headers = {
    "x-api-key" : CAT_API_KEY
}
response = requests.get(
    api_endpoint,
    headers = headers
)

for idx, item in enumerate(response.json()):
    print(f"{idx+1}. {item['name']} : {item['description']}")

We created a dictionary called headers to store the API Key. The key in the dictionary is "x-api-key". However, this can differ based on the API you are working with. Some APIs require the key to be named "Authorization", "authorization", "token". It is best to refer to your API's documentation's authentication section.

Bearer Authentication is pretty common and it requires the word "Bearer " (note the space) to be at the beginning of the API Token/Key.

headers = {
    "authorization": f"Bearer {access_token}"
}

We will be using bearer authentication in an example in a later section.

APIs with Basic Auth

An API secured using Basic Auth requires a username and password. Usually, the username is the Client ID and the password is the Client Secret of the API. In some cases, the username can be left blank. This should be mentioned in the API documentation.

The Twilio API is secured using Basic Auth. You can sign up on the Twilio website and get access to the API credentials.

from requests.auth import HTTPBasicAuth
from dotenv import load_dotenv
import os 
import requests

load_dotenv()
TWILIO_ACCOUNT_SID = os.environ.get("TWILIO_ACCOUNT_SID")
TWILIO_ACCOUNT_TOKEN = os.environ.get("TWILIO_ACCOUNT_TOKEN")

api_endpoint = f'https://api.twilio.com/2010-04-01/Accounts/{TWILIO_ACCOUNT_SID}/Calls.json?PageSize=5'

auth = HTTPBasicAuth(TWILIO_ACCOUNT_SID, TWILIO_ACCOUNT_TOKEN)

response = requests.get(api_endpoint , auth = auth)

for idx, item in enumerate(response.json()['calls']):
    print(f"{idx+1}. {item['duration']}")

We create an instance of HTTPBasicAuth. It takes in the username and password respectively as arguments. This instance is passed as an argument when making the request. In the case of twilio, the username is your account sid and the password is your account token. As mentioned before, it can be different for different APIs. If the API you are using, uses Basic Auth to secure its endpoints, refer to the docs for the username and password.

API Wrappers

With respect to Python, API wrappers are essentially libraries/packages which can be installed using pip. These libraries help communicate with APIs in a syntactically cleaner way. Under the hood, the libraries still make use of requests and headers to make requests. However, the wrappers make your code look cleaner.

The Twilio API we discussed earlier has a wrapper. It can be installed using pip

pip install twilio

Let's try to do the same thing we did in the previous section with Twilio

from twilio.rest import Client
from dotenv import load_dotenv
import os 


load_dotenv()
TWILIO_ACCOUNT_SID = os.environ.get("TWILIO_ACCOUNT_SID")
TWILIO_ACCOUNT_TOKEN = os.environ.get("TWILIO_ACCOUNT_TOKEN")
client = Client(TWILIO_ACCOUNT_SID , TWILIO_ACCOUNT_TOKEN)

calls = client.calls.list(limit=5)

for idx, record in enumerate(calls):
    print(f"{idx}. {record.duration}")

As you can see, the code is a few lines shorter and looks much cleaner.

Unfortunately, not all APIs have a wrapper. However, a lot of them do. Before a consumer, an API directly, try searching for a wrapper around it. This will make it significantly easier to work with the API.

The Session Object

Instead of passing the API Key or HTTPBasicAuth Instance every time you make a request to a secured API endpoint, you can create a session object. You'll have to authenticate once and can make requests without needing to pass the key or the auth instance.

We will work with the GitHub API which is secured using BasicAuth. The username will be your GitHub username and the password is your personal access token. You can get one by following this tutorial.

from requests.auth import HTTPBasicAuth
from dotenv import load_dotenv
import os 
import requests

load_dotenv()
GITHUB_API_TOKEN = os.environ.get("GITHUB_API_TOKEN")

base_api_endpoint = "https://api.github.com/user"

auth = HTTPBasicAuth("rahulbanerjee26", GITHUB_API_TOKEN)

session = requests.Session()
session.auth = auth

response = session.get(base_api_endpoint + '/repos')

for idx, item in enumerate(response.json()):
    print(f"{idx+1}. {item['name']}")

response = session.get(base_api_endpoint + '/emails')
for idx, item in enumerate(response.json()):
    print(f"{idx+1}. {item['email']}")

After I set the session's auth value to the HTTPBasicAuth instance, I can simply make requests without passing the authentication each time. Our requests are still being authenticated, but the Session object takes care of it.

APIs secured via OAuth2

Using OAuth2 web flow to authenticate is usually used in Flask/Django apps when you need a "Sign Up using Google", "Sign Up using Facebook" option. However, some APIs need OAuth2 for all their endpoints. The GitHub API supports OAuth2 authentication as well. We will also be talking about the Genius API. Although it supports key-based authentication, its endpoint requires OAuth2, it is possible to get a token and authenticate yourself by passing the key in the headers object. However, we will be using the OAuth web flow to authenticate ourselves.

I won't be going too much into detail on how OAuth2 works since that is beyond the scope of this article. Below is a high-level overview. If it doesn't make sense, skip over to the Github or Genius API section and it should make more sense.

We will have to create a client app on the API's website
The client app will have a client ID and Client Secret
We will have to make a request to the API's authentication endpoint. The client ID and client Secret will be passed as query parameters.
The authentication endpoint will ask for permission and will have to be authorized

Once authorized, it will return a code
This code will have to be given to another endpoint which will exchange it for an access token.
This access token can now be used as a key and be passed as a header object when making requests to the endpoint.

Let's take a look at a couple of examples.

Using the GitHub API (OAuth2)

As mentioned above, OAuth2 is mostly used with Flask/Django APPs. When working with OAuth2, you will need a web app URL and a URL to redirect the user to once they authorize/give permission. Since we do not have a web app, we do not have any URL. However we can use HTTPBin. Whenever we need an URL, we can use the following URL

https://httpbin.org/anything

First, you will have to create a GitHub App. When asked for the web app URL or the redirect URL, use the above-discussed URL. Once you have created the app, store the Client ID and Client Secret in the .env file.

import requests
import os
from dotenv import load_dotenv
from urllib.parse import urlencode
import webbrowser

load_dotenv()
CLIENT_ID = os.environ.get("GITHUB_CLIENT_ID")
CLIENT_SECRET = os.environ.get("GITHUB_CLIENT_SECRET")
REDIRECT_URI = "https://httpbin.org/anything"

params = {
    "client_id": CLIENT_ID,
    "redirect_uri": REDIRECT_URI,
    "scope": "user"
}

endpoint = "https://github.com/login/oauth/authorize"
endpoint = endpoint + '?' + urlencode(params)
webbrowser.open(endpoint)

code = input("Enter the Code: ")

The first few lines are importing libraries and loading the credentials from the .env file. The parameter dictionary contains the client ID, the redirect URL which is the HTTPBin URL we discussed earlier and the scope. The value of the scope determines the endpoints you can access and the HTTP Verb Actions you can do.

We create an URL along with the query parameter and open it in a browser using the webbrowser library. The Pythons script waits for use the input our code,

Once you click authorize, you should be redirected to the HTTPBin URL and a JSON Object should be displayed. Look at the value for the key "code". This value will be exchanged for an API Token. Input the code.

print("Got Code")

params = {
    "client_id": CLIENT_ID,
    "client_secret": CLIENT_SECRET,
    "redirect_uri": REDIRECT_URI,
    "code": code,
}
endpoint = "https://github.com/login/oauth/access_token"
response = requests.post(endpoint, params=params, headers = {"Accept": "application/json"}).json()
access_token = response['access_token']
print("Got Access Token")

Once, we get the code, we make another request to an endpoint to get an access token. This time we pass the code along with the client secret as parameters. After GitHub validates the credentials along with the code, it will return an access token. This access token can be used as an API Key.

session = requests.session()
session.headers = {"Authorization": f"token {access_token}"}

base_api_endpoint = "https://api.github.com/user"

response = session.get(base_api_endpoint)
print(response)

response = session.get(base_api_endpoint + '/repos')
print(response)

response = session.get(base_api_endpoint + '/emails')
print(response)

Using the Genius API (OAuth2)

Let's take a look at another example. I'll skip the part where we import the libraries and load the credentials.

parameters = {
    'client_id': GENIUS_CLIENT_ID,
    'redirect_uri': 'https://httpbin.org/anything',
    'response_type': 'code',
    'scope': 'me'
}
endpoint = "https://api.genius.com/oauth/authorize"
endpoint = endpoint + '?' + urlencode(parameters)
webbrowser.open(endpoint)
code = input("Enter the Code: ")

The "response_type" is mandatory for some APIs, the value should always be "code"

After we authorize, we will see a JSON object similar to the one we saw when working with the GitHub API. Input the Code.

print(code)
parameters = {
    "code": code,
    "client_id": GENIUS_CLIENT_ID,
    "client_secret": GENIUS_CLIENT_SECRET,
    "redirect_uri": 'https://httpbin.org/anything',
    "response_type": "code",
    "grant_type": "authorization_code"
}

response = requests.post("https://api.genius.com/oauth/token", params = parameters).json()
print(response)
access_token = response["access_token"]

"grant_type" is also required by some APIs. The value is always "authorization_code". After our code is validated, we get an access token. This token can be used as an API Key.

session = requests.session()
session.headers = {"authorization": f"Bearer {access_token}"}

base_api_endpoint = "https://api.genius.com/account"

response = session.get(base_api_endpoint)
print(response)

Conclusion

I hope this article serves as a good guide to work with APIs in Python. Before consuming an API directly, always look for a wrapper. The 5 mins you spend looking for a wrapper might save you hours of headache.

Connect with me on LinkedIn, Twitter