In modern software development, CI/CD (Continuous Integration/Continuous Deployment) pipelines and building reusable components are crucial for maintaining high productivity and rapid deployment cycles. Adopting monorepos has become increasingly popular because they allow for better organization and management of code, particularly in large projects. By housing multiple packages within a single repository, teams can share code more efficiently and maintain consistency across different projects.

However, as monorepos grow, so does the build time. In some cases, build times can become relatively long. This is exactly what happened in our company. We created a monorepo to hold most of our React components and various utility functions. Unfortunately, the build time for this monorepo took about 12 minutes. This long build time hampered productivity and increased costs, especially when builds failed or defects needed fixing, requiring additional time for corrections.

Caching the Build Using Turbo Repo

To address this issue, we implemented Turbo Repo. Turbo Repo works by caching previous builds and only rebuilding the parts of the codebase that have changed. This looked promising, as our build time was significantly reduced. By using Turbo Repo, we could optimize our build process and make it more efficient.

Company Rules and Regulations

Using Turbo Repo worked well locally, but we faced challenges when integrating it into our CI/CD pipeline using GitHub Actions. According to Turbo Repo's documentation, a repository needs to be connected to Vercel for caching to work properly. However, this posed a problem because connecting to Vercel could potentially expose our code to a third party, which goes against our company's privacy regulations.

To comply with company policies, we found an alternative method: uploading the cache created by Turbo Repo to GitHub Artifact. This way, we could still use the benefits of Turbo Repo without exposing our code.

How It Works

Here's how we implemented this method:

• Pipeline Trigger: Every time a pipeline is triggered, it uploads the current cache created by Turbo Repo to GitHub Artifact.
• Downloading Previous Cache: When a new pipeline is triggered, it downloads the previous cache from GitHub Artifact.
• Cache Utilization: Turbo Repo uses the downloaded cache to determine which parts of the codebase need rebuilding, significantly reducing build times. This method not only ensured that we complied with company regulations but also reduced our build time from 12 minutes to just 3 minutes.

name: CI

on:
  pull_request:
    branches:
      - main
      - feature/*
      - feat/*

jobs:
  ci:
    runs-on: ubuntu-latest
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

    strategy:
      matrix:
        # run both checks in parallel to speed up the CI
        run: ['pnpm run check-all', 'pnpm run build']

    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - uses: pnpm/action-setup@v2
        name: Install pnpm
        id: pnpm-install
        with:
          version: 8
          run_install: false

      - name: Install Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: 'pnpm'

      - name: Install dependencies
        run: pnpm install
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Download Existing Artifact from Target
        uses: dawidd6/action-download-artifact@v2
        continue-on-error: true
        with:
          workflow: prerelease.yml
          branch: ${{github.event.pull_request.base.ref}}
          if_no_artifact_found: ignore

      - name: Check Turbo cache existence
        id: turbo_cache
        uses: andstor/file-existence-action@v2
        with:
          files: ".turbo"

      - name: Download Existing Artifact from last workflow
        if: steps.turbo_cache.outputs.files_exists != 'true'
        uses: dawidd6/action-download-artifact@v2
        continue-on-error: true
        with:
          workflow: prerelease.yml
          if_no_artifact_found: ignore

      - name: ${{ matrix.run }}
        run: ${{ matrix.run }}
        continue-on-error: true

      - name: Upload .turbo
        uses: actions/upload-artifact@v2
        with:
          name: .turbo
          path: .turbo

Conclusion

By optimizing our build process using Turbo Repo and leveraging GitHub Artifact for caching, we achieved a significant reduction in build times while adhering to company policies. This improvement not only speeds up our development cycle but also boosts overall productivity and efficiency within the team. Developers can now iterate and test changes much faster, leading to quicker deployment of new features and enhancements.