Running Lighthouse CI on Every Pull Request

Performance regressions that ship to production are almost always regressions nobody measured at review time. Running Lighthouse CI on every pull request closes that gap with a deterministic, merge-blocking gate: each PR and each subsequent commit triggers an audit on standardized throttling, an assertion step compares the median against the budget, and the result posts back into the PR thread as both a status check and an inline comment. This guide is part of the Lighthouse CI Configuration & Storage reference; it assumes you already have a committed lighthouserc.json and focuses narrowly on wiring the per-PR gate and surfacing its result to reviewers.

Unlike a local audit, which inherits the developer's hardware, browser extensions, and background load, CI execution gives every PR the same Chrome version, the same simulated network, and the same CPU model. That reproducibility is the whole point: a metric delta in the PR comment is a real delta, not noise from someone's laptop. The gate you build here is the enforcement point for whatever ceilings you set upstream — a number that was aspirational in a spreadsheet becomes a wall a merge cannot pass through.

PR-Gate Flow

The gate runs three stages in one lhci autorun invocation. Collection builds the app, serves it, and runs Lighthouse three times keeping the median. Assertion compares that median to the budget and sets the process exit code. Reporting posts the status check and a comment. A non-zero exit on a required check blocks the merge. Because all three stages live in a single command, there is no partial state to reconcile: either the run finishes green and the PR is mergeable, or it finishes red and it is not.

Diagnostic Steps

Before trusting the gate, confirm the same run is deterministic and the server binds before collection starts. Two checks catch most setup problems.

Run the audit locally against the built preview and confirm the assertion summary prints:

npm run build && npx lhci autorun --collect.url=http://localhost:3000/

Expected tail — a summary table followed by the exit status:

Saved LHR to .lighthouseci/lhr-1718000000000.json
Checking assertions against 1 URL...
All results processed!
Done running Lighthouse!

Confirm Chrome can launch headless on the runner and the config is valid:

npx lhci healthcheck --fatal

Expected output: Healthcheck passed!. A failure here is almost always a missing --no-sandbox flag or a config path typo, not a real performance problem. If healthcheck passes but collection still hangs, the runner is usually starved for shared memory — pass --collect.chromeFlags="--no-sandbox --disable-dev-shm-usage" so Chrome writes temp files to disk instead of a tiny /dev/shm partition.

Implementation

The workflow runs on every pull_request event. It caches dependencies, builds, then runs Lighthouse CI and posts the result to the PR. The LHCI_GITHUB_APP_TOKEN (or GITHUB_TOKEN) lets lhci post the status check and comment back to the thread.

name: Lighthouse CI
on:
  pull_request:
    branches: [main]

jobs:
  lighthouse:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"
      - run: npm ci
      - run: npm run build
      - name: Wait for preview server
        run: npx serve -s dist -l 3000 & npx wait-on http://localhost:3000
      - name: Run Lighthouse CI
        run: npx lhci autorun
        env:
          LHCI_GITHUB_APP_TOKEN: ${{ secrets.LHCI_GITHUB_APP_TOKEN }}
          LHCI_BUILD_CONTEXT__CURRENT_HASH: ${{ github.event.pull_request.head.sha }}
      - name: Upload reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: lighthouse-reports
          path: .lighthouseci/

Starting the preview server in the background and blocking on wait-on removes the most common flake — Lighthouse collecting before the server is listening, which surfaces as ERR_CONNECTION_REFUSED. Caching via setup-node's built-in cache: npm cuts redundant installs without managing a separate cache key. The if: always() on the upload step matters: when the assertion fails you most want the reports, so the artifact must upload even after a non-zero exit. Setting LHCI_BUILD_CONTEXT__CURRENT_HASH to the PR head SHA (not the synthetic merge commit GitHub creates) keeps the comment and the stored run pointing at the commit the author actually pushed.

CI Gating Assertion

This is the exact assertion block the gate evaluates. It lives under ci.assert.assertions in lighthouserc.json. The error level forces a non-zero exit on breach; warn posts a comment but lets the pipeline pass. A budget.json referenced from assertMatrix or assertions caps transfer sizes.

{
  "ci": {
    "collect": { "numberOfRuns": 3, "settings": { "throttlingMethod": "simulate" } },
    "assert": {
      "assertions": {
        "categories:performance": ["error", { "minScore": 0.9 }],
        "first-contentful-paint": ["error", { "maxNumericValue": 1800 }],
        "largest-contentful-paint": ["error", { "maxNumericValue": 2500 }],
        "cumulative-layout-shift": ["error", { "maxNumericValue": 0.1 }],
        "total-blocking-time": ["error", { "maxNumericValue": 200 }],
        "resource-summary:script:size": ["warn", { "maxNumericValue": 250000 }]
      }
    },
    "upload": { "target": "temporary-public-storage" }
  }
}

These ceilings are for a high-end mobile device on a simulated 4G profile (1.6 Mbps down, 150 ms RTT) at P75 of three CI runs. The 2500 ms LCP and 0.1 CLS values are the "good" field thresholds; total-blocking-time is a lab proxy for interaction latency, so it sits at 200 ms rather than a field INP number. Use error only for thresholds that have held across two weeks of baselines, and keep newer metrics at warn — note the script-size cap above stays at warn — so the gate earns reviewer trust before it starts blocking merges. The distinction between the two levels is the entire behavioural contract of the gate.

To scale the same assertion across multiple viewports and routes in parallel, move to GitHub Actions Performance Matrices. If a metric flaps across runs even on identical code, it is not ready for error yet — tighten the environment first, as covered in Reducing Lighthouse CI Variance in Staging.

Median of Three and Variance

numberOfRuns: 3 is not decoration. A single Lighthouse run on a shared CI runner can swing 200-400 ms on LCP purely from CPU contention with neighbouring jobs. Collecting three runs and asserting against the median discards a single unlucky outlier: one slow run cannot fail the gate on its own, because the median is the middle of the sorted three, not the maximum. The diagram below shows the case that trips people up — run 2 exceeded the 2500 ms ceiling, yet the PR passed because the median run did not.

Three runs trade about two extra minutes of CI time for a gate that does not cry wolf. If your runner is still noisy after that, the fix is upstream of numberOfRuns: pin CPU throttling so every run models the same device, as detailed in Calibrating CPU Throttling for CI Runners. Bumping to five runs helps only marginally once the environment itself is stable, and it doubles the wall-clock cost of every PR.

Reading the Report Comment

The inline comment is what reviewers actually read, so its shape matters. Each audited URL gets a row with its performance score and each asserted metric's value, and a link to the full hosted report. When you upload to temporary-public-storage, that link expires after a few days — fine for PR review, not for trend history. For a durable record, point upload.target at a self-hosted server so every PR run is queryable months later. A breach comment names the offending audit and shows expected-versus-found, which lets a reviewer decide in seconds whether the regression is intended (a new hero image) or accidental (an unshaken dependency).

Keep the comment single, not stacked. With the GitHub App runner, set the comment to update in place so each new commit overwrites the previous verdict rather than appending a fresh block to the thread. A PR with fourteen commits should show one Lighthouse comment reflecting the latest head, not fourteen stale ones.

Verification

After the workflow runs, confirm three things. The PR shows a Lighthouse CI status check — green on pass, red on breach. An inline comment lists each audited URL with its scores and a link to the full report. And in branch protection, the lighthouse-ci (or Lighthouse CI) check is marked Required, which is what actually makes a breach unmergeable.

A passing run ends with a zero exit and this assertion summary in the job log:

Checking assertions against 1 URL, 3 runs...
All results processed!
Done running Lighthouse! Assertions passed.

A breach prints the failing assertion and exits non-zero:

largest-contentful-paint failure for maxNumericValue assertion
      expected: <=2500
      found: 3120
1 result(s) failed

If the check shows green but a regression still merged, the check is not marked Required in branch protection — fix that first, because the assertion was working correctly. The PR gate catches regressions a human introduces; it does not catch slow drift from dependency bumps between PRs, so pair it with Scheduling Nightly Lighthouse Runs once the per-PR gate is trusted.

Frequently Asked Questions