Introduction

For over a decade now, I have been involved in building distributed scalable applications. I have built monolith systems with PHP and ASP.NET, worked on the Frontend and Backend teams on different projects, and transitioned into the microservices world primarily developing with JavaScript and Node.js. In all these years of work, testing and shipping code has always been an experience I wished could be improved. This led me to quickly embrace continuous integration and continuous deployment/delivery (CI/CD) platforms even before it became cool.

Ever since I moved from the stone age of pushing code with FTP servers and using specialized deployment workflows of different hosting platforms, CircleCI has always been my CI/CD platform of choice. My automated testing and deployment workflow involved connecting my repositories on GitHub to CircleCI to build CI/CD pipelines. So imagine my curiosity when GitHub Actions was launched offering a CI/CD server built right into the code hosting platform itself.

Should I move all my pipelines to GitHub? Will my experience on GitHub Actions be better than what I have enjoyed with CircleCI or would I just ruin everything all for the sake of being "trendy"? Would I need to learn a whole new syntax and write all my CI/CD pipelines all over again?

These and many more were the questions swimming through my mind. So, I decided to test out GitHub Actions and compare the experience with CircleCI and also weigh the cost if I decide to make GitHub Actions my CI/CD platform.

Full disclosure: I am affiliated with CircleCI as a guest author on their blog. However, the opinions expressed in this post are entirely my own.

CircleCI and How It Works

Overview

CircleCI is a continuous integration and continuous deployment/delivery platform that enables you to develop and run build pipelines. You can choose to run your pipelines on CircleCI's cloud or your private infrastructure.

Setting up a CI/CD pipeline on CircleCI involves three (3) basic steps:

• Connect your repository to CircleCI
• Write your pipeline configuration script
• Run your pipeline on CircleCI's servers

Pricing and Plans

As of the time of this writing, CircleCI has 3 pricing plans for its cloud service offerings; Free, Performance, and Scale.

Personally, I haven't had a need for a paid plan as the Free plan offers a lot of features I need for my small to mid-level projects. The Performance plan is the only plan with a fixed price of 30 bucks. It offers a high level of concurrently running jobs (up to 80) and allows you to run builds on macOS. The Performance plan also includes Docker layer caching with leads to faster builds and improves the performance of your pipelines.

For requirements that exceed what is obtainable from the Free and Performance plans, you have to ability to contact CircleCI's sales team to get a custom plan.

UI

CircleCI's user interface has gone through a major overhaul over the past few years and it's in a pretty good state at the moment.

The bigger and bolder font and predominantly white background is a major improvement over what I used to work with about 3 years ago. The flow from setting up a new project up to the point where you see the first build is also very easy to follow especially for first-time users of the platform. Your project type is also auto-detected and a sample configuration script is suggested for you to use as a baseline.

On the builds page shown below, it's very clear to see what has passed and what has failed. The color-coding is just right, with appropriate badge names and icons to indicate the status of the build.

One improvement I would like to see though is quick links to common projects settings like Environment variables from the builds page.

Also, the Project Settings link on the build page is too far to the right which makes it difficult to locate (page resolution currently makes it appear as though it's below the project title). It would be nice to have this button closer to the project title. I find myself using this button a lot so it will be helpful to make it easier to locate.

Also, I think the action buttons for re-running and canceling a build need to be more prominent.

I didn't notice the buttons for a long time using the service and for the important role they perform, I believe they deserve more highlight than they are currently getting. Making them colored or look more clickable should help in bringing more attention to them.

One other neat feature of CircleCI's UI is the build page, which I find convenient to use in viewing the steps in the pipeline job.

The text is clear and legible, and the arrangement of components and colors ensure that the extra details hanging around the page do not take away focus from the main item, the build steps.

The far-right icon on each step's tab was a bit confusing for me because I assumed it's for downloading an artifact generated for that step, but instead it leads to a webpage that displays the activities in the selected step in plain text. I wouldn't mind a download button for artifacts generated per step, but you can easily find all your artifacts on the ARTIFACTS tab.

The UI also has an Insights page where you can view data on how your builds are performing over time. This can help you make more informed decisions on improving the efficiency of your pipelines.

In summary, I would score the UI 8/10. I believe a good number of improvements can still be made but it already does excellently well in giving a satisfying user experience.

GitHub Actions and How It Works

Overview

GitHub Actions is a platform for automating developer workflows. Unlike CircleCI, GitHub actions features are not limited to continuous integration and deployment workflows. Building and running CI/CD pipelines is just one of the workflows that GitHub Actions (GHA for short) can automate. Other workflows that can be automated include labeling GitHub issues, creating release notes upon deployment, assigning issues to repository collaborators, etc.

You can begin setting up a workflow in a selected repository on GitHub by going to the Actions tab. There, GitHub can set you up with a bunch of templates of common workflows or help you build one from scratch.

Pricing and Plans

GitHub Actions pricing is bundled together with other features that GitHub offers with paid plans starting at $4 (yeah, just 4 bucks).

Actions are only free for public repositories thus, no free plan covers a private repository you would like to use.

The pricing tiers are distinguished by the total number of minutes your builds can run within a month. GitHub Actions allows a workflow to run for a total of 58 minutes, this includes queuing and execution time so make sure you factor this into the plan you choose.

There is no statement on concurrency on the pricing plans. This is because GitHub Actions only allows a maximum of two workflows to run concurrently.

The paid plans appear cheaper than that of CircleCI as long as you're comfortable with the tradeoffs on build time and concurrency limits.

UI

One convenience that is immediately obvious with using GitHub Actions is the fact that your repository is automatically connected to actions. There is no extra step of integrating it with a third party for CI/CD.

So I decided to use GitHub Actions for some of my projects and here are a couple of things I have observed about the user interface.

First, it was pretty easy to set up with my Nodejs-based project as I just picked a continuous integration template for Node.js and I got a workflow configuration script right away. The special directory for GitHub Actions was also automatically set up and in a few minutes, I had my build running.

However, I wasn't a big fan of the workflow details page, the all-dark scheme wasn't just working for me including the grey step labels and icons. I could see the icons colored red when things fail but I don't think the grey icons are a good indicator that a step was successful.

The overly simplistic UI also gave me the impression of GitHub Actions not being a feature-rich service yet, especially for CI/CD.

Environment variables are also set in the Secrets section under the Settings tab. I would prefer that you don't have to leave the Actions tab to set your environment variables and that every control related to GitHub Actions should be located under the Actions tab.

Overall, I'll score the UI a 6/10. It's simple so you can easily locate stuff, but it tries too hard to fit into the main GitHub UI flow which might not give the best user experience for a CI/CD tool.

Comparing CircleCI and GitHub Actions

Use Cases

As mentioned earlier GitHub Actions is more than CI/CD; CI/CD is just one of the processes you can automate with GitHub Actions. This enables you to do more with the product, including automatically generating Changelogs when you release a build to production. You can also automatically version your releases and assign issues submitted by users of your software to team members working on the product.

CircleCI on the other hand is a CI/CD-first platform specialized for building and running CI/CD pipelines. It comes with a feature-rich toolset to cover a wide range of software engineers' DevOps requirements.

If you're looking to build CI/CD pipelines for your GitHub repos and also automate a set of manual tasks you perform on GitHub, GHA is a good choice. However, if you're looking for a dedicated DevOps platform to build, manage, and run your CI/CD pipelines, CircleCI is your best bet.

Setup and Configuration

Setup on both platforms is very similar. However, CircleCI requires an extra step of linking up your remote repository provider (GitHub included) to the platform. Linking your repositories is a one-time step on CircleCI. Once you're done with this step, you have access to all your repositories and you can start building CI/CD pipelines for any one of them.

GitHub repositories already come with an Actions tab where you can start setting up your build pipelines right away.

Both services allow the configuration of pipelines using a YAML file where you can declare the environment where your build is to run and the steps involved in the build. A good number of the configuration properties are also similar, with some differences.

One major difference is in how environments are provisioned. CircleCI uses the concept of Executors to define the environment in which your build will run. An executor defines the underlying technology or environment in which to run a job. For example, the docker executor allows you to pull in a Docker image to use for your build as shown below:

version: 2.1
jobs:
  build:
    working_directory: ~/repo
    docker:
      - image: circleci/node:10.16.3

The docker executor allows you to run both CircleCI-hosted images (and oh my, there are a lot of them for different OS and installed software) and private images. Anything that works in Docker works in CircleCI.

You can also define executors to use any version of Linux, macOS, or Windows machine you may prefer for your build.

Another benefit you get with using Docker on CircleCI is caching of your Docker containers. This feature is not available on GitHub Actions at the moment.

GitHub Actions on the other hand uses the runs-on directive to provision a build environment. You can use this directive to specify Linux, Windows, and macOS platforms supported by GitHub Actions (find the list here). An example of provisioning an environment is shown below:

jobs:
  build:
    runs-on: ubuntu-latest

You can also use a self-hosted runner as the machine your build runs on in GitHub Actions. A container directive can also be used to specify a Docker container to be used but more on that later.

Integrations and Extendability

GitHub Actions' best feature is that it is built right into GitHub, and its biggest flaw is that it is built right into GitHub. Being a GitHub-only CI/CD platform means that you cannot pull code from other remote code hosting services like Bitbucket.

During the days when you had to have a paid plan to get private repositories on GitHub, most developers (myself included) turned to services like Bitbucket and Gitlab to host private source codes for free. Code hosted on these platforms cannot take advantage of GitHub Actions.

CircleCI on the other hand supports both GitHub and Bitbucket repositories at the moment. I would love to see them support Gitlab also. However, being open to supporting other code hosting platforms gives them an edge here.

In terms of functional extendability, CircleCI and GitHub Actions both support reusable configurations. CircleCI has Orbs while GitHub Actions allows you to create self-contained actions that can be used within your configuration scripts. These two allow you to package repetitive or commonly used tasks into reusable modules that can be easily plugged into your main configuration script.

Below is an example of a CircleCI Heroku Orb used to deploy an application to Heroku with a single command:

version: 2.1
orbs:
  heroku: circleci/heroku@0.0.10
workflows:
  heroku_deploy:
    jobs:
      - heroku/deploy-via-git

And below is a GitHub Actions reusable Docker action use to print a message (config truncated):

steps:
      - name: Hello world action step
        id: hello
        uses: actions/hello-world-docker-action@v1
        with:
          who-to-greet: 'Mona the Octocat'

Both platforms allow you to supply parameters to the reusable configuration modules. However, GitHub limits the number of steps that can be parameterized in its custom actions.

Docker Support

Both platforms support the use of Docker images, but in my opinion, CircleCI does it better than GitHub Actions. In order to use Docker images on GitHub Actions, you will need to run the image within a machine that supports it (only Linux is supported at the moment) or build the image from the ground up. This is why reusable Docker actions are created to make this step less tedious.

Here is a sample configuration using a Node.js Docker image in GitHub Actions

jobs:
  container:
    runs-on: ubuntu-latest
    container: node:10.16-jessie

CircleCI on the other hand providers the docker executor that allows you to run your Docker image directly. Below is a CircleCI configuration using a Node.js container:

jobs:
  build:
    working_directory: ~/repo
    docker:
      - image: circleci/node:10.16.3

CircleCI also allows you to run Docker images not just in Linux environments but macOS and Windows.

From following GitHub issues and Stackoverflow threads, Docker support is still a bit buggy on GitHub Actions. Maybe due to the fact that the product is still fairly new compared to CircleCI.

CircleCI has perfected its Docker support over the years to make it (almost) the de-facto environment for running builds.

Comparison Summary

In order to cover as many criteria as possible to compare the two services (this article won't end if I were to write on them all), I have put together a summary table to declare my choice between the two based on different criteria.

Also, this helps to have a quick glance in order to make a decision:

Conclusion

So there you have it, my experience with CircleCI and GitHub Actions with a focus on CI/CD features and capabilities. Based on this comparison, I will be sticking with CircleCI for my CI/CD operations while I keep a tab on GitHub Actions to see the improvements that come in the future.

Please note that this is my personal opinion regarding the two platforms so feel free to come to your own conclusions. However, I hope the information I have provided in this piece will help you in making your decision regarding which platform to use for your day-to-day CI/CD operations.

Happy Coding :)