Intermediate GitHub CI Workflow Walk Through
Liu Yongliang
Posted on February 15, 2022
Motivation
There are many interesting use cases out there for GitHub Actions. One of the main use cases is to create custom continuous integration (CI) workflows that are hosted and executed in your GitHub repository. I should add that that it's FREE for public repositories, with some usage limits/policies :)
With that introduction out of the way, here I am going to walk through a slightly more complex workflow that I am using in the open-source project MarkBind
Requirements
The reason why I would like to explain the workflow in detail here is that I think some requirements are fairly typical and it could serve as an example of a common CI script. Of course, the main reference that I would recommend is the official documentation by GitHub Also, this goes without saying that what I presented here is not the only way to do things.
So, let's talk about what I want the workflow to do:
- Run tests
- Run it whenever someone makes a PR against the master branch
- Run it in all major OSes
- To ensure that it ain't just working on my machine
- Also run it when a PR is merged, or when someone commits directly to the master branch
- Sometimes senior devs might directly commit a quick fix to the master branch and this should still trigger the tests
- Deploy developer guide
- Run it whenever a PR is merged, or when someone commits directly to the master branch
- This is similar to the test runs but it is only triggered when a PR is merged (not when it is created)
- The rationale for updating the developer guide per PR merge is so that our developer guide is always up-to-date with the latest development of the project, which could be slightly ahead of the released version
- Run it whenever a PR is merged, or when someone commits directly to the master branch
- Deploy user guide
- Run it whenever we release a new version
- Some context: when we release a new version we will push a tag of the master branch to GitHub. Hint: this is how we are going to trigger this step
- Run it whenever we release a new version
Code
Now we know the what, let me share the how.
First, here's a brief summary of what we need to know about workflows:
- A workflow can have multiple jobs that can run sequentially or in parallel (this is the default)
- Each job can have multiple steps that run sequentially.
- Both jobs and steps can be configured to run under certain conditions.
Test
So to achieve just what we need for the test, we can do something like this:
name: CI
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
test:
strategy:
matrix:
platform: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '12'
- run: npm i -g npm@8.3.1
- run: npm run setup
- run: npm run test
Some explanations of the syntax used:
-
on.push.branches
says that the workflow is only triggered when pushing to master, which is what happens when a PR is merged -
on.pull_request
says that the workflow will run when someone sends over a PR, which is nice to ensure that the changes don't break the build - the use of strategy and matrix is pretty much boilerplate code that is used to specify that the job named
test
will run on all three OSes.- This will run the tests in Ubuntu, macOS, and Windows, in parallel. It will by default fail-fast to stop the other two test runs if one of them failed unexpectedly.
Dev Guide
To achieve just what we need for the developer guide update, we can do something like the following:
name: CI
on:
push:
branches:
- master
jobs:
deploy-docs:
# disabled on forks
if: github.repository == 'MarkBind/markbind'
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '12'
- run: npm i -g npm@8.3.1
- run: npm run setup
- name: Deploy DG on any commit to master, to markbind.org/devdocs
run: >-
npm run build:web &&
npm run build:dg &&
npm run deploy:dg
Now we reach the fun part... how to include the above with the test job so that it only runs when all tests have passed?
Here's my approach:
# code for test job omitted
deploy-docs:
needs: test
# disabled on forks
if: github.event_name == 'push' && github.repository == 'MarkBind/markbind'
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '12'
- run: npm i -g npm@8.3.1
- run: npm run setup
- name: Deploy DG on any commit to master, to markbind.org/devdocs
run: >-
npm run build:web &&
npm run build:dg &&
npm run deploy:dg
There is quite a bit of stuff here, so here's a summary:
- I have defined another job named
deploy-docs
- I specified it to only run if the previous job
test
is done and successful by doingneeds: test
- I added a check to ensure that this job, unlike the test, will not run for pending PRs.
if: github.event_name == 'push' && github.repository == 'MarkBind/markbind'
- it first checks if it is a
push
event (and not PR) - it then checks if the repository is the root repository
- this is added to ensure that forks of the repo do not execute this job because they have no permission/access to publish the developer/user guides.
The rest is just set up and deploy commands that you may ignore.
User Guide
Lastly, let's deal with the user guide which only needs to run per release.
To achieve just what we need for the user guide update, we can do something like the following:
name: CI
on:
push:
branches:
- master
tags:
- 'v[0-9]+.[0-9]+.[0-9]+'
jobs:
deploy-docs:
# disabled on forks
if: github.repository == 'MarkBind/markbind'
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '12'
- run: npm i -g npm@8.3.1
- run: npm run setup
- name: Deploy UG on release, to markbind.org
if: github.ref_type == 'tag'
run: >-
npm run build:ug &&
npm run deploy:ug
To integrate it into the entire workflow, the following changes are required (full script at the end):
name: CI
on:
push:
branches:
- master
tags:
- 'v[0-9]+.[0-9]+.[0-9]+'
pull_request:
branches:
- master
# some code in between
- name: Deploy UG on release, to markbind.org
if: github.ref_type == 'tag'
run: >-
npm run build:ug &&
npm run deploy:ug
- The addition of
on.push.tags
ensures that when a new tag on the master branch is pushed to GitHub, as part of making a new release, will trigger the workflow.- This runs the
test
job and the developer guide deployment step as well.- It could easily be turned off such that only the user guide step is run.
- the
v[0-9]+.[0-9]+.[0-9]+
is a glob pattern to match semantic versioning tags.
- This runs the
- The
if: github.ref_type == 'tag'
in the user guide step will ensure that if this is just a PR merge or a push event to master, the step will be skipped.- The details of the
github
object that I am accessing here is available here
- The details of the
Full Script
Putting everything together:
name: CI
on:
push:
branches:
- master
tags:
- 'v[0-9]+.[0-9]+.[0-9]+'
pull_request:
branches:
- master
jobs:
test:
strategy:
matrix:
platform: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '12'
- run: npm i -g npm@8.3.1
- run: npm run setup
- run: npm run test
deploy-docs:
needs: test
# disabled on forks
if: github.event_name == 'push' && github.repository == 'MarkBind/markbind'
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '12'
- run: npm i -g npm@8.3.1
- run: npm run setup
- name: Deploy DG on any commit to master, to markbind.org/devdocs
run: >-
npm run build:web &&
npm run build:dg &&
npm run deploy:dg
- name: Deploy UG on release, to markbind.org
if: github.ref_type == 'tag'
run: >-
npm run build:ug &&
npm run deploy:ug
Conclusion
Now that CI script is written and workflow is automated, robots can finally take my job.
Posted on February 15, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.