Lighthouse CI & WebPageTest Integration

A performance gate is only as trustworthy as the determinism of the runs behind it. Bolting an audit onto a pipeline as an advisory comment changes nothing; turning that audit into a required status check that exits non-zero on a real regression changes everything. This reference treats Lighthouse CI and WebPageTest as two halves of one enforcement contract: Lighthouse CI supplies fast, reproducible lab metrics that gate every pull request, and WebPageTest supplies the deep network-waterfall and filmstrip diagnostics that explain why a metric moved. Wired together behind a single assertion layer, they convert subjective "the site feels slow" debates into quantified thresholds that block suboptimal code before it merges. The thresholds themselves come from Defining Web Performance Budgets, and the statistical method for keeping them honest lives in Threshold Calibration & Baseline Management; this page is where those numbers become an executable gate.

The integration splits into six engineering concerns: how synthetic runs are collected deterministically, what metrics and thresholds define a breach, how the lab-versus-field division of labour is drawn, how the gate is wired into CI/CD, how regressions are caught in production through observability, and what happens when the gate itself misfires. This page is the top-level spec for all of them. It links down to the detailed guides for each surface — the collection and storage layer in Lighthouse CI Configuration & Storage, field telemetry in Custom Performance Beacons & RUM, parallel coverage in GitHub Actions Performance Matrices, isolated network agents in WebPageTest Private Instance Setup, scheduled drift detection in Continuous Performance Monitoring, and tool selection in Comparing Performance Testing Tools.

Architecture Overview

A pull request fans out to two synthetic engines in parallel. Lighthouse CI collects a median of repeated runs and emits a JSON report; a WebPageTest agent runs the same URLs through a controlled connection profile and returns its own metrics. Both result sets feed a single assertion step that compares values against the budget. A pass releases the merge through a status check; a breach exits non-zero. Either way, artifacts upload to a storage backend that powers the trend dashboard and the regression baseline. The rule that keeps this honest is separation of concerns: the engines measure, a single assertion layer decides, and branch protection enforces. No engine is ever allowed to render its own verdict, because two engines with two opinions is how a gate quietly stops meaning anything.

The key design rule is that Lighthouse CI and WebPageTest never disagree on whether the gate passed — assertions live in one place. They differ only in depth: Lighthouse CI is the fast verdict on every commit, WebPageTest is the slower forensic engine you reach for when a verdict needs explaining or when you need real network control that simulated throttling cannot reproduce. A concrete way to picture the split: Lighthouse CI answers "did this PR make the page slower?" in under two minutes, while WebPageTest answers "which byte on which connection made it slower?" in five to ten. You gate on the first and diagnose with the second, and both read the same budget so their answers can never contradict each other.

Metric Selection & Threshold Matrix

Gate on a small set of primary metrics and keep the rest as diagnostic signals. Primary metrics block the merge; diagnostic signals annotate the PR but never fail it. Map every lab threshold back to a field percentile — set the lab assertion 10 to 15 percent tighter than your P75 field value to absorb the lab-to-field gap, and always pin the device class and connection profile the number applies to. A threshold with no device and no percentile attached is not a budget, it is a wish; "LCP under 2.5 s" means nothing until it reads "LCP P75 under 2500 ms on mid-range mobile over Fast 3G."

INP is a field-only metric, so the lab gate uses TBT as its proxy and the real INP ceiling — 200 ms at P75 for high-end mobile on 4G — is enforced through field telemetry. The methodology for deriving these numbers from your own traffic lives in Percentile-Based Threshold Tuning; this matrix is a representative starting point, not a copy-paste constant. The gap between the lab ceiling and the field P75 is deliberate. Lab runs on a warmed CDN cache, a single deterministic CPU throttle, and no real packet loss, so lab numbers run optimistic against the messy tail of real devices. Setting the lab gate roughly 12 percent tighter than the field P75 buys back that optimism, so a PR that passes the lab still has headroom when it meets a real mid-range Android on a congested cell.

A word on why the primary set is deliberately short. Every metric you promote to a blocking gate is a metric that can block a merge on its own noise, so each one has to earn its place by being both user-meaningful and stable enough to assert. LCP, CLS, and a script-byte budget clear that bar: they map directly to what a user feels, and on a median of three runs they are stable to within a few percent. TBT earns its place only as the lab stand-in for INP, which cannot be measured in a lab at all. Everything else — Speed Index, Time to Interactive, total transfer size — is genuinely useful for diagnosis but too correlated or too jittery to gate on, so it rides along as a non-blocking annotation on the PR. The discipline is to resist the urge to gate on ten metrics; a gate that fails for ten different reasons is a gate no one can reason about, and reasoning about the gate is the whole point. When teams complain that the budget is "always red," the cause is almost never a genuinely slow site and almost always a primary set that grew past what the runner can measure stably.

The Lab and Field Division of Labour

Lab and field are not competing sources of truth; they answer different questions and neither can substitute for the other. Lab tools (Lighthouse CI, WebPageTest) run a scripted device on a controlled connection, so they are deterministic enough to gate a merge but blind to the diversity of real hardware. Field tools (RUM beacons) capture what actual users on actual devices experienced, so they are the ground truth for percentile targets but far too noisy and delayed to block a single pull request. The whole integration exists to let each do what it is good at and to keep them anchored to one another.

The load-bearing insight is direction of trust. The field sets the targets — you derive the P75 ceiling from real traffic — and the lab enforces a tightened copy of those targets fast enough to catch a regression before merge. A change in the field percentile should propagate into the lab budget within a calibration cycle; a change in the lab gate should never silently drift away from the field it is meant to protect. When the two disagree, the field is authoritative about what users feel and the lab is authoritative about what this specific PR changed.

Asset and Tool Configuration

Both engines read from version-controlled config so runs are reproducible across machines. Lighthouse CI is configured through lighthouserc.json; the canonical annotated version of that file, including collection determinism and storage targets, is the subject of Lighthouse CI Configuration & Storage. The collect and assert blocks that matter for the gate:

{
  "ci": {
    "collect": {
      "url": ["https://staging.example.com/", "https://staging.example.com/checkout"],
      "numberOfRuns": 3,
      "settings": {
        "preset": "desktop",
        "throttlingMethod": "simulate"
      }
    },
    "assert": {
      "assertions": {
        "categories:performance": ["error", { "minScore": 0.9 }],
        "largest-contentful-paint": ["error", { "maxNumericValue": 2000 }],
        "cumulative-layout-shift": ["error", { "maxNumericValue": 0.1 }],
        "total-blocking-time": ["error", { "maxNumericValue": 150 }],
        "resource-summary:script:size": ["error", { "maxNumericValue": 200000 }]
      }
    }
  }
}

The maxNumericValue numbers above are the desktop-on-cable column of the threshold matrix: a 2000 ms LCP ceiling and a 150 ms TBT ceiling at P75, both tightened from a field P75 nearer 2200 ms and 175 ms. If you also gate mobile, run a second config with the mobile preset and the mid-range column, because a single set of thresholds cannot honestly cover a laptop on fibre and a mid-range Android on Fast 3G at the same time — the separate-config pattern is worked in Mobile vs Desktop Budget Divergence.

WebPageTest runs from a script that submits the same URLs to an agent and polls for the result, then maps the returned metrics back to the same budget. Provisioning the dedicated agent is covered in WebPageTest Private Instance Setup; the comparison harness reads the WebPageTest JSON and exits non-zero on a breach:

// wpt-gate.js — submit URLs to a WebPageTest agent and assert the result
const WebPageTest = require("webpagetest");
const wpt = new WebPageTest(process.env.WPT_SERVER, process.env.WPT_API_KEY);

const BUDGET = { lcp: 2000, cls: 0.1, tbt: 150, bytesJs: 200000 };
const URLS = ["https://staging.example.com/", "https://staging.example.com/checkout"];

function run(url) {
  return new Promise((resolve, reject) => {
    wpt.runTest(
      url,
      { connectivity: "Cable", runs: 3, location: "ci-agent:Chrome", pollResults: 5, timeout: 300 },
      (err, result) => (err ? reject(err) : resolve(result.data.median.firstView))
    );
  });
}

(async () => {
  let failed = false;
  for (const url of URLS) {
    const m = await run(url);
    const checks = {
      lcp: m["chromeUserTiming.LargestContentfulPaint"],
      cls: m["chromeUserTiming.CumulativeLayoutShift"],
      tbt: m.TotalBlockingTime,
      bytesJs: m.breakdown.js.bytes,
    };
    for (const [k, v] of Object.entries(checks)) {
      if (v > BUDGET[k]) {
        console.error(`FAIL ${url} ${k}=${v} exceeds budget ${BUDGET[k]}`);
        failed = true;
      } else {
        console.log(`PASS ${url} ${k}=${v} within ${BUDGET[k]}`);
      }
    }
  }
  process.exit(failed ? 1 : 0);
})();

The two budgets — assertions in lighthouserc.json and BUDGET in wpt-gate.js — are the same numbers expressed twice. Keep them in one shared JSON file imported by both if you run both engines as required checks. A single source of truth for the budget is the difference between a gate you trust and a gate that drifts: when the field P75 moves and you retighten the ceiling in only one of the two files, the engines start rendering different verdicts on the same PR and the whole contract unravels.

{
  "lcp": 2000,
  "cls": 0.1,
  "tbt": 150,
  "bytesJs": 200000,
  "meta": { "device": "desktop", "connection": "cable", "percentile": "P75" }
}

Store that budget.json in the repo, import it from wpt-gate.js, and generate the lighthouserc.json assertions from it in a small build step so neither file can be edited alone. The meta block is not decoration — it records the device, connection, and percentile the numbers belong to so a future reader never has to guess what "2000" measured.

CI/CD Gating Integration

The gate lives in a GitHub Actions workflow on pull_request. Lighthouse CI runs lhci autorun, which collects, asserts, and uploads in one command; a non-zero exit blocks the merge once the check is required in branch protection. The per-PR comment and status-check pattern is detailed in Running Lighthouse CI on Every Pull Request, and fanning the same run across viewports and routes is covered in GitHub Actions Performance Matrices.

name: Performance Gating
on:
  pull_request:
    branches: [main]

jobs:
  lighthouse-ci:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    concurrency:
      group: lhci-${{ github.ref }}
      cancel-in-progress: true
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: "20"
          cache: "npm"
      - run: npm ci
      - run: npm run build
      - name: Run Lighthouse CI
        run: npx lhci autorun
        env:
          LHCI_TOKEN: ${{ secrets.LHCI_TOKEN }}
          LHCI_SERVER_BASE_URL: ${{ secrets.LHCI_SERVER_BASE_URL }}

  webpagetest:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: "20", cache: "npm" }
      - run: npm ci
      - name: WebPageTest gate
        run: node wpt-gate.js
        env:
          WPT_SERVER: ${{ secrets.WPT_SERVER }}
          WPT_API_KEY: ${{ secrets.WPT_API_KEY }}

Run Lighthouse CI as a required check on every PR because it is fast; run WebPageTest as a required check only on routes where network-level control matters, or on a nightly schedule, because its turnaround is measured in minutes rather than seconds. Require the lighthouse-ci check in branch protection so a breach is unmergeable. The diagram below traces one PR through the gate so the ordering is unambiguous — build first, collect a median of three runs, assert against the shared budget, then either post a passing status or exit non-zero and block.

The median of three runs is not a stylistic choice; it is variance control. A single Lighthouse run on a shared CI runner can swing 10 to 20 percent on TBT and LCP because the runner's CPU is contended, so a gate reading one run will flake and teams will learn to re-run until green — which trains everyone to ignore the gate. Taking the median of three (or five on a noisy runner) collapses most of that variance; the deeper statistical treatment, including when three is not enough, is worked in Statistical Noise & Flakiness Reduction.

Observability and Regression Detection

Lab gates catch regressions a developer introduces; they cannot catch regressions that only appear under real device and network diversity, nor slow drift that no single PR is responsible for. Close that gap with two feeds. First, a scheduled synthetic run against production on a fixed cadence, comparing each run to a rolling baseline so slow drift surfaces before it crosses a hard threshold — the scheduling, alerting, and competitor-benchmarking patterns for that cadence live in Continuous Performance Monitoring. Second, real-user telemetry through Custom Performance Beacons & RUM, which captures field LCP, INP, and CLS at P75 and P90 and feeds those percentiles back into the threshold matrix so lab budgets stay anchored to reality.

// minimal field beacon for the metrics the lab gate proxies
import { onLCP, onINP, onCLS } from "web-vitals";

function send(metric) {
  navigator.sendBeacon(
    "/rum",
    JSON.stringify({ name: metric.name, value: metric.value, id: metric.id, path: location.pathname })
  );
}

onLCP(send);
onINP(send);
onCLS(send);

When the field P75 for a route drifts above the lab ceiling, the lab number is stale and should be retightened; when the field is comfortably under but the lab keeps failing, the lab environment is noisier than production and the runner needs attention, not the budget. The distinction matters because the two failures have opposite fixes: one says the budget lies and must move, the other says the measurement lies and must be stabilised. Aggregating raw beacons into stable percentiles is its own pipeline problem — batching, sampling, and P75/P90 rollups are worked in Building P75/P99 Aggregation Pipelines — and once those percentiles land in a dashboard they become the exec-facing trend covered in Dashboarding & Team Adoption.

There is a subtlety in what you point the scheduled synthetic run at. A PR gate necessarily runs against a staging build, because the code is not merged yet, and staging almost always differs from production in cache warmth, feature flags, and data volume. That is acceptable for catching a relative regression — did this PR make staging slower than the last staging run — but it is a poor absolute measure. The scheduled run closes that gap by testing production itself on a fixed cadence, so the absolute P75 you calibrate against is measured where users actually are. Keep the two clearly labelled in storage: staging runs feed the per-PR baseline, production runs feed the calibration percentile, and mixing them silently is how a team ends up gating on numbers that describe a page no user ever loads. If staging variance is itself the problem, the reduction techniques in Reducing Lighthouse CI Variance in Staging address it directly.

Failure Modes and Escalation

A gate that blocks merges is a gate people will try to route around the moment it misfires, so its failure modes deserve as much design as its happy path. The four below cover nearly every "the gate is broken" ticket, and each has a specific fix that is not "lower the budget."

• Flaky gate, no real regression points to variance from a noisy runner or real-network throttling. Raise numberOfRuns to 5, keep throttlingMethod: simulate for determinism, and confirm the metric swings under 10 percent across runs before tightening anything. If the runner itself is contended, pin CPU throttling as covered in Calibrating CPU Throttling for CI Runners.
• Lighthouse passes, WebPageTest fails means the two engines saw different network conditions. Lighthouse simulates the connection; WebPageTest shaped a real one. Treat WebPageTest as authoritative for connection-sensitive metrics and reconcile the two in Comparing Performance Testing Tools.
• Third-party drift breaks the build usually means a vendor shipped a heavier tag with no code change on your side. Pin vendor versions or widen the tolerance deliberately, and track the payload against Third-Party Script Constraints.
• Token or agent unreachable is an infrastructure failure, not a performance failure. An expired LHCI_TOKEN or a down WebPageTest agent fails the upload, not the audit. Fail the upload soft (if: always()) so a storage outage never blocks a clean merge, and page the owning team instead.

The escalation path ties these together: a first breach annotates the PR and is the author's problem to fix before merge; a breach that recurs on main opens a tracked regression ticket with the WebPageTest filmstrip and Lighthouse report attached, owned by the team that merged the change. Writing that policy down — who owns a breach, how long they have, when a budget may be renegotiated — is what turns a gate from a source of friction into a shared contract, and the template for it lives in Writing a Performance Budget Policy.

Frequently Asked Questions