Setting up a Self-Hosted Runner on GitHub Actions

GitHub Actions is a powerful tool for automating workflows directly in GitHub. While GitHub-hosted runners are convenient, self-hosted runners offer much more flexibility by giving you full control over the hardware, operating system, and installed software. This guide will walk you through setting up a self-hosted runner on GitHub, giving you the ability to optimize your environment to meet specific project requirements.

Why Companies are Switching to GitHub/GitLab for CI/CD Pipelines

With the recent closure of AWS CodeCommit, many companies are transitioning to platforms like GitHub and GitLab for code storage and management. Solution architects are adapting CI/CD pipelines to align with this shift, leveraging GitHub and GitLab’s built-in CI/CD tools—GitHub Actions and GitLab CI/CD—rather than relying on standalone systems. This consolidation of code storage and CI/CD pipelines simplifies integration, enhances collaboration, and optimizes workflow visibility by keeping everything in one platform.

GitHub Actions has become especially popular, providing companies with a robust CI/CD tool that automates code testing, deployment, and integration. However, GitHub Actions has limitations, particularly for companies with large-scale or complex workflows.

Overcoming GitHub Actions Limitations with Self-Hosted Runners

While GitHub Actions is powerful, it has certain restrictions, especially when relying solely on GitHub-hosted runners. These runners have runtime and billing limitations that could lead to higher costs or performance impacts if workflows exceed the provided limits. For example, GitHub Actions offers free tier limits (especially for public repositories) but imposes runtime limits, data transfer restrictions, and concurrency caps on GitHub-hosted runners. For extensive workflows, upgrading to a paid GitHub plan may be necessary.

An alternative to relying solely on GitHub-hosted runners is self-hosted runners. Self-hosted runners provide several benefits, helping to overcome GitHub Actions’ limitations:

No Runtime Limits: Self-hosted runners do not have a predefined runtime cap, which makes them ideal for handling longer or more complex jobs.
Enhanced Resource Control: They allow you to choose hardware that meets the specific resource demands of your workflows, providing greater flexibility in CPU, memory, and storage.
Cost Efficiency: Instead of upgrading to higher GitHub subscription tiers to increase runner usage, self-hosted runners can operate on existing infrastructure, reducing the overall costs.
Flexibility in Software: With self-hosted runners, you can install and run custom software that may not be available in GitHub-hosted environments.

For companies looking to maximize GitHub Actions workflows without incurring high costs, self-hosted runners offer a flexible, efficient solution. You can read more about GitHub Actions’ billing limitations in the GitHub Actions Billing Documentation.

Levels of Self-Hosted Runners:

GitHub provides three levels of self-hosted runners for different scopes:

Repository-Level: Dedicated to a single repository.
Organization-Level: Available to multiple repositories within an organization.
Enterprise-Level: Can be assigned to repositories across various organizations within an enterprise.

Step-by-Step Guide to Setting Up a Self-Hosted Runner

Step 1: Create a New Self-Hosted Runner

To begin, navigate to the appropriate settings page depending on the level at which you want to set up the runner:

Repository-Level: Go to Repository Settings → Actions → Runners → New self-hosted runner.

Organization-Level: Go to Organization Settings → Actions → Runners → New runner.

GitHub will display a setup page to guide you through creating a new self-hosted runner.

Step 2: Choose Your Machine and Download the Runner

After Clicking on “New self-hosted runner”, This page will appear at both the Repository and Organization Level. On the setup page, select the machine type and architecture for your runner:

Machine: Choose from Linux, Windows, or macOS.
Architecture: Select your desired architecture, such as x64 or ARM.

After making your selections, GitHub provides a download script that you can use to install the runner application on your VM or server.

NOTE: In ./Config there is one token generated which only works for a few minutes, if you have waited for a long time then need to again create a runner (STEP1).

Step 3: Configure the Runner on Your Machine

Once the runner is downloaded, follow these steps to configure it:

Run the Configuration Script: Execute ./config.sh on your machine. During this step, you’ll generate a token from GitHub that’s required to authenticate the runner. Note that the token is only valid for a short period—if you wait too long, you’ll need to generate a new one.
Set Runner Options:
- Runner Group: Specify a group if needed (ensure the group exists before specifying it).
- Runner Name: Provide a custom name for your runner.
- Labels: Labels are used to identify the runner in your workflow YAML files. By default, labels include self-hosted, the OS type (e.g., Linux), and architecture (e.g., x64). You can add custom labels as needed.
- Working Directory: Specify the directory where the runner will download workflow files and operate.

Step 4: Start the Runner

To start the runner, use the following command: ./run.sh

This command starts the runner in interactive mode, allowing it to listen for and process incoming jobs. Make sure your workflow YAML files include labels that match the runner configuration so GitHub can correctly assign jobs to your self-hosted runner.

Step 5: Set Up the Runner as a Service

To set up the self-hosted runner as a service on Linux (using systemd), GitHub provides an svc.sh script, which makes it easy to install, start, check, stop, and uninstall the runner service.

Install the Service: Stop the runner if it’s currently running, then install it as a service by running:

sudo ./svc.sh install

To install it as a specific user, specify the username:

sudo ./svc.sh install USERNAME

Start the Service:

sudo ./svc.sh start

Check Service Status: To check if the runner service is active and listening, run:

sudo ./svc.sh status

Stop and Uninstall the Service:

If needed, stop the service with:

sudo ./svc.sh stop

To completely remove the service, run:

sudo ./svc.sh uninstall

Setting up the GitHub self-hosted runner as a service ensures it will automatically restart if the machine reboots, making it ideal for production environments.

Best Practices for Managing Self-Hosted Runners

Security: Self-hosted runners have access to your local network and systems, so ensure they’re secured and access is limited.
Regular Updates: Keep your runner application updated to avoid compatibility issues and gain the latest features and security patches.
Monitoring and Logging: Use monitoring tools and GitHub’s built-in logs to track runner activity, resource usage, and any issues that arise.
Scaling: For high-demand environments, consider adding multiple runners or using runners with autoscaling capabilities in cloud environments.

References and Further Reading:

For more detailed information, here are some helpful resources:

Blog