Integrate Gitlab with Google Cloud workload identity federation

In Gitlab 14.7, connecting to AWS, GCP and vault, and other cloud services is now possible by introducing the CI_JOB_JWT_V2 environment variable. I'll use this environment variable to impersonate a service account via workload identity federation.

Workload identity federation

Workload identity federation allows you to impersonate an existing service account on Google Cloud. Everyday use cases for workload identity federation include:

Enabling a background application or continuous integration/continuous delivery (CI/CD) pipeline that runs outside of Google Cloud to access Google Cloud resources and APIs..
Enabling users of a web application that runs outside of Google Cloud to access data stored in a Google Cloud service, such as Cloud Storage or BigQuery.

To use workload identity federation, you configure Google Cloud to trust an external identity provider such as Amazon Web Services (AWS), Azure Active Directory (AD), an OIDC-compatible identity provider, or a SAML 2.0-compatible identity provider Preview. Applications can then use credentials issued by the external identity provider to impersonate a service account by following these steps:

Setup the workload identity provider.
Obtain a credential from the trusted identity provider.
Exchange the credential for a token from the Security Token Service.
Use the token from the Security Token Service to impersonate a service account and obtain a short-lived Google access token.

Setup the workload identity provider

Setting up the proper Identity Pool

gcloud iam workload-identity-pools create POOL_ID \
    --location="global" \
    --description="DESCRIPTION" \
    --display-name="DISPLAY_NAME"

Adding an OpenID Connect Provider to the pool (more specifically our Gitlab instance).

gcloud iam workload-identity-pools providers create-oidc PROVIDER_ID \
    --location="global" \
    --workload-identity-pool="POOL_ID" \
    --issuer-uri="ISSUER" \
    --allowed-audiences="AUDIENCE" \
    --attribute-mapping="MAPPINGS" \
    --attribute-condition="CONDITIONS""

Assigning service accounts that can be impersonated by these identities and the conditions:

gcloud iam service-accounts add-iam-policy-binding SERVICE_ACCOUNT_EMAIL \
    --role=roles/iam.workloadIdentityUser \
    --member="MEMBER_EXPRESSION"

Remark: the member expression is not clear within the workload identity federation console, but you can find more details if you navigate to the connected service account and click on permissions.

Conditions

Conditions make it especially useful, because this allows you to incorperate fine grained permissions in your pipeline. You could scope to certain branches, branches and even limit the users who can trigger the build.

For mapping we could do something like this:

export MAPPINGS="attribute.project_path=assertion.project_path"

For conditions we could then add the following:

export CONDITIONS="attribute.custom_path==\"glenn.bostoen/workload-federation-poc\""

Obtaining a credential from the trusted identity provider

It's quite easy on Gitlab to retrieve your credentials, it's injected in the environment variable CI_JOB_JWT_V2. The specification of the token can be found here, it looks as follows:

{
  "jti": "c82eeb0c-5c6f-4a33-abf5-4c474b92b558",
  "iss": "https://gitlab.example.com",
  "aud": "https://gitlab.example.com",
  "iat": 1585710286,
  "nbf": 1585798372,
  "exp": 1585713886,
  "sub": "project_path:mygroup/myproject:ref_type:branch:ref:main",
  "namespace_id": "1",
  "namespace_path": "mygroup",
  "project_id": "22",
  "project_path": "mygroup/myproject",
  "user_id": "42",
  "user_login": "myuser",
  "user_email": "myuser@example.com",
  "pipeline_id": "1212",
  "pipeline_source": "web",
  "job_id": "1212",
  "ref": "auto-deploy-2020-04-01",
  "ref_type": "branch",
  "ref_protected": "true",
  "environment": "production",
  "environment_protected": "true"
}

You could use this environment variable directly or output it to a temporary file:

echo ${CI_JOB_JWT_V2} > .ci_job_jwt_file

Exchange the credential for a token from STS

The following shell scripts allow you to exchange your JWT token for e federated token.

#!/bin/sh -x

PAYLOAD=$(cat <<EOF
{
"audience": "//iam.googleapis.com/projects/${PROJECT_NUMBER}/locations/global/workloadIdentityPools/${POOL_ID}/providers/${PROVIDER_ID}",
"grantType": "urn:ietf:params:oauth:grant-type:token-exchange",
"requestedTokenType": "urn:ietf:params:oauth:token-type:access_token",
"scope": "https://www.googleapis.com/auth/cloud-platform",
"subjectTokenType": "urn:ietf:params:oauth:token-type:jwt",
"subjectToken": "${CI_JOB_JWT_V2}"
}
EOF
)

FEDERATED_TOKEN=$(
    curl -X POST "https://sts.googleapis.com/v1/token" \
        --header "Accept: application/json" \
        --header "Content-Type: application/json" \
        --data "${PAYLOAD}" |
        jq -r '.access_token'
)

Impersonate service account

You can now use this federated token to impersonate a service account by getting an access token for this service account.

ACCESS_TOKEN=$(
    curl -X POST "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/${SERVICE_ACCOUNT_EMAIL}:generateAccessToken" \
        --header "Accept: application/json" \
        --header "Content-Type: application/json" \
        --header "Authorization: Bearer ${FEDERATED_TOKEN}" \
        --data '{"scope": ["https://www.googleapis.com/auth/cloud-platform"]}' |
        jq -r '.accessToken'
)
echo "${ACCESS_TOKEN}"

Integration on Gitlab

You could use the scripts defined above and integrate these directly or you could use the gcloud CLI to easily impersonate a service account. So before we had the setup with permanent keys on Gitlab:

gcp-auth:
    image: google/cloud-sdk:slim
    before_script:
        - gcloud auth activate-service-account --key-file ${GOOGLE_APPLICATION_CREDENTIALS}
        - gcloud config set project ${GOOGLE_PROJECT}

This can now be replaced by the following script:

gcp-auth:
  image: google/cloud-sdk:slim
  script:
    - echo ${CI_JOB_JWT_V2} > .ci_job_jwt_file
    - gcloud iam workload-identity-pools create-cred-config "${GCP_WORKLOAD_IDENTITY_PROVIDER}"
      --service-account="${GCP_SERVICE_ACCOUNT}"
      --output-file=.gcp_temp_cred.json
      --credential-source-file=.ci_job_jwt_file
    - gcloud auth login --cred-file=`pwd`/.gcp_temp_cred.json
    - gcloud config set project ${GOOGLE_PROJECT}

We just need to specify the workload identity provider we want to use and the service account we want to impersonate.

Summary

If you have already isolated the authentication step to Google Cloud in your Gitlab templates, it should be quite straightforward to switch to this new mechanism. It gives you more granularity on who or what has access to your Cloud provider and is more secure in general by not having a permanent key on your instance. It does require some extra work on your Google Cloud project, but this can also be tackled by having some Terraform project templating.

Blog