The Practical Developer

GitHub Actions In A Monorepo: Caching, Path Filters, And Secret Boundaries That Actually Work

May 10, 2024 by The Practical Developer

A naive monorepo CI runs all jobs on every PR, takes 25 minutes, and burns money. The version that works has path-filtered jobs, cross-job caching, and reusable workflows. Here is the working setup that runs in 4 minutes for a typical PR.

ci-cd

devops

github-actions

Code on a laptop screen, the right setting for the careful work of CI/CD configuration

The team has a monorepo with three apps: web, api, mobile. A CI pipeline runs everything on every PR: install, lint, typecheck, test, build, deploy preview. Total time: 25 minutes. A PR that touches only api/ waits the same 25 minutes as one that touches everything.

This is the default monorepo CI pattern, and it’s wrong. PRs should run only the jobs affected by their changes. Caching should be aggressive. Long-running jobs should run in parallel where possible. Secrets should be scoped so a compromised PR cannot exfiltrate prod credentials.

This post is the working setup: path filters, cache configuration, reusable workflows, and the secret-boundary pattern that prevents lateral compromise. Total CI time on a typical single-app PR drops to ~4 minutes.

Path filters: only run what changed

Use path filters to skip jobs entirely:

# .github/workflows/api.yml
name: api
on:
  pull_request:
    paths:
      - 'api/**'
      - 'shared/**'
      - '.github/workflows/api.yml'

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm test --workspace=api

A PR that touches only web/ doesn’t trigger this workflow at all. The CI dashboard shows what was actually relevant.

For more nuanced filtering, dorny/paths-filter computes a per-job filter:

jobs:
  changes:
    runs-on: ubuntu-latest
    outputs:
      api: ${{ steps.filter.outputs.api }}
      web: ${{ steps.filter.outputs.web }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            api: 'api/**'
            web: 'web/**'

  api-test:
    needs: changes
    if: needs.changes.outputs.api == 'true'
    runs-on: ubuntu-latest
    steps: ...

  web-test:
    needs: changes
    if: needs.changes.outputs.web == 'true'
    runs-on: ubuntu-latest
    steps: ...

One workflow, multiple conditional jobs. Cleaner than separate workflows for some teams.

Caching: the highest-leverage CI improvement

actions/setup-node@v4 with cache: 'npm' (or pnpm, yarn) caches dependencies automatically. For other caches, the explicit pattern:

- name: Cache build artifacts
  uses: actions/cache@v4
  with:
    path: |
      .next/cache
      node_modules/.cache
    key: ${{ runner.os }}-build-${{ hashFiles('**/package-lock.json') }}-${{ github.sha }}
    restore-keys: |
      ${{ runner.os }}-build-${{ hashFiles('**/package-lock.json') }}-
      ${{ runner.os }}-build-

key is the exact match. restore-keys are progressively more lenient fallbacks; first match wins. Result: even on a fresh PR, you get most of yesterday’s cache.

For Docker layer caching, use docker/build-push-action with a registry-based cache:

- uses: docker/build-push-action@v5
  with:
    context: ./api
    push: true
    tags: ghcr.io/.../api:${{ github.sha }}
    cache-from: type=registry,ref=ghcr.io/.../api:buildcache
    cache-to: type=registry,ref=ghcr.io/.../api:buildcache,mode=max

For monorepo TypeScript projects, the tsc --build incremental cache plus a workflow cache turns a 90s typecheck into 5 seconds.

Matrix builds: parallel where possible

Tests across multiple Node versions, multiple OSes, multiple databases:

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        node: ['18', '20', '22']
        os: [ubuntu-latest, macos-latest]
    runs-on: ${{ matrix.os }}
    steps: ...

fail-fast: false continues other matrix jobs after one fails. Useful for “we want to know which Node version broke it.”

For tests that take a long time, shard across runners:

jobs:
  test:
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
    runs-on: ubuntu-latest
    steps:
      - run: npm test -- --shard=${{ matrix.shard }}/4

Vitest, Jest, and Playwright all support sharding. A 12-minute test suite becomes 3 minutes on 4-way parallel.

Reusable workflows

Don’t copy-paste the same setup across multiple workflows. Factor out:

# .github/workflows/_node-setup.yml
name: node-setup
on:
  workflow_call:
    inputs:
      node-version: { required: true, type: string }

jobs:
  setup:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ inputs.node-version }}
          cache: 'npm'
      - run: npm ci
# .github/workflows/api.yml
jobs:
  setup:
    uses: ./.github/workflows/_node-setup.yml
    with: { node-version: '20' }

  test:
    needs: setup
    runs-on: ubuntu-latest
    steps:
      - run: npm test --workspace=api

Reusable workflows make CI changes one-place-to-edit. Major maintenance win.

Secret boundaries

A pull request from a fork can run workflows. Without care, those workflows can read your secrets and exfiltrate them. The default behavior is reasonable:

  • pull_request events from forks have no access to repository secrets.
  • pull_request_target events do have access, and run on the target branch’s code, not the fork’s. Use it carefully.

Rule: any workflow that needs production secrets should run on push to main, not on pull_request. Preview environments use scoped credentials, not prod ones:

on:
  push:
    branches: [main]    # only on main; forks can't push to main

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production       # GitHub Environments adds approval gates
    steps:
      - uses: actions/checkout@v4
      - run: ./deploy.sh
        env:
          AWS_KEY: ${{ secrets.PROD_AWS_KEY }}

The environment: production line gates this on a manual approval (configurable in repo settings). Deploys cannot proceed without an approver.

For preview environments on PRs, use a scoped IAM role (or separate AWS account) that can only manage preview resources:

- run: ./deploy-preview.sh
  env:
    AWS_KEY: ${{ secrets.PREVIEW_AWS_KEY }}  # different role, smaller scope

A compromised PR can drop preview environments; it can’t touch production.

Concurrency: cancel old runs

When a PR gets a new push, cancel the old workflow run. Saves money and surfaces the latest result.

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

Same group → previous run is cancelled. Makes the dashboard cleaner and CI cheaper.

Self-hosted runners: when worth it

GitHub-hosted runners are billable per minute. For a heavy CI workload, self-hosted runners on EKS (via actions-runner-controller) can be 5-10x cheaper.

The trade-off: you operate the runners. They need scaling, security patching, monitoring. For small teams, GitHub-hosted is fine. For 10+ engineers running CI all day, self-hosted often pays back.

Caveat: never self-host runners that handle PRs from forks. The fork’s code runs on your hardware; security implications are real.

Pre-merge vs post-merge

Default GitHub setup runs CI before merge. For very large monorepos, “merge queue” (in beta as of 2024) runs CI after a merge candidate is selected, sequentially, so two PRs that pass individually but break together are caught.

For most teams under 50 engineers, pre-merge CI plus required status checks is enough. Merge queues are for very high-velocity repos where the rate of “passes individually, breaks combined” is significant.

Observability for CI itself

A few queries worth running periodically:

  • Median PR time-to-merge from CI start. Should be under 10 minutes.
  • % of CI failures that are flaky (pass on retry). Should be under 5%.
  • Most-expensive jobs by total minutes consumed.

The third one is where you find optimization opportunities. Often a single test suite or build step dominates the bill; speeding it up by 30% is the win.

The takeaway

A monorepo CI that runs everything on every PR is paying for nothing. Path filters, aggressive caching, matrix sharding, reusable workflows, and proper secret boundaries take a 25-minute pipeline to 4 minutes. The work is one or two days of focused tuning. The payoff is faster PR cycles and lower bills, every day, for years.

Don’t write CI configs once and forget them. Re-tune quarterly as the codebase grows and patterns emerge. The compounding wins are real.

A note from Yojji

The kind of CI-engineering discipline that takes a slow, expensive pipeline and turns it into a fast, cheap one (path filters, caching, matrix sharding, secret boundaries) is the kind of long-haul DevOps work Yojji’s teams put into the products they ship for clients.

Yojji is an international custom software development company founded in 2016, with teams across Europe, the US, and the UK. They specialize in the JavaScript ecosystem (React, Node.js, TypeScript), cloud platforms (AWS, Azure, GCP), and full-cycle product engineering, including the CI/CD work that decides whether your team ships fast or waits in the build queue.

Keep Reading