Automating Baseline Promotion Workflows #

A rolling baseline is only as trustworthy as the rule that updates it. If every merge to main overwrites the baseline blindly, the first regression that lands becomes the new normal and the gate quietly stops catching it — the baseline-poisoning failure mode. This guide, part of the Historical Baseline Calibration reference, defines a promotion step that updates the stored baseline automatically, but only after a clean post-merge run on main, and stamps each promotion with the Git SHA that produced it so any bad promotion can be audited and reverted.

The principle is simple: promotion is a privilege, not an event. A run earns the right to become the baseline by passing the existing gate first. Everything below is the mechanics of enforcing that rule in CI.

Promotion Rules #

Promotion is governed by a small set of conditions that must all hold. The table is the authoritative ruleset — a run that fails any row is rejected and the previous baseline is retained.

The delta-sanity row is the backstop against a broken collection (a blank page, a 500) producing an absurdly fast "improvement" that silently relaxes the band. A change larger than 20% is held for manual review rather than auto-promoted.

Diagnostic Steps #

Before automating, confirm what the current baseline is and whether the last main run is eligible to replace it.

# 1. Show the currently stored baseline and its provenance
node scripts/baseline-fetch.js --branch main --print
# → lcp p75=2180ms  sha=4f1c9ad  computedAt=2026-06-19T02:00:00Z  samples=87

# 2. Show the latest main-branch run's median and whether it passed the gate
node scripts/baseline-compare.js --baseline baseline_store.json --run .lighthouseci/ --summary
# → PASS  lcp 2150ms (band 1980-2380)  inp 158ms (band 114-214)  → eligible for promotion

A PASS with a median inside every band, on a main push, is the green light. Anything else and the diagnostic tells you exactly which row of the ruleset blocked it.

Implementation #

The script below promotes the median of a clean run to the baseline. It enforces every rule in the table, recomputes the band from the configured percentile and tolerance, and stamps the Git SHA before writing back to the store.

// scripts/promote-baseline.js  —  run only on a green push to main
import { readFileSync } from "node:fs";
import { execSync } from "node:child_process";

const MIN_SAMPLES = 30;
const MAX_DELTA = 0.20; // delta-sanity guard

function median(xs) {
  const s = [...xs].sort((a, b) => a - b);
  const m = Math.floor(s.length / 2);
  return s.length % 2 ? s[m] : (s[m - 1] + s[m]) / 2;
}

const store = JSON.parse(readFileSync(process.env.STORE_PATH, "utf8"));
const window = JSON.parse(readFileSync(process.env.WINDOW_PATH, "utf8")); // trimmed samples per metric
const gatePassed = process.env.GATE_STATUS === "pass";
const sha = execSync("git rev-parse --short HEAD").toString().trim();

if (!gatePassed) throw new Error("refuse: candidate did not pass the gate");

for (const [metric, cfg] of Object.entries(store.metrics)) {
  const samples = window[metric] ?? [];
  if (samples.length < MIN_SAMPLES) throw new Error(`refuse: ${metric} thin window (${samples.length})`);
  const next = median(samples);
  const prev = cfg.value ?? next;
  if (Math.abs(next - prev) / prev > MAX_DELTA) throw new Error(`refuse: ${metric} delta > 20%`);

  const half = cfg.tolerance.type === "rel" ? next * cfg.tolerance.value : cfg.tolerance.value;
  cfg.value = Math.round(next);
  cfg.band = [Math.round(next - half), Math.round(next + half)];
}

store.provenance = { computedAt: new Date().toISOString(), gitSha: sha };
process.stdout.write(JSON.stringify(store, null, 2));
console.error(`promoted baseline @ ${sha}`);

Pipe the script's stdout back to your store (> baseline_store.json then upload), and let its non-zero exit on any refusal keep the previous baseline in place.

CI Gating Assertion #

Wire promotion to run only on main pushes, after the same gate that protects pull requests has already passed. The if: success() on the promotion step is the load-bearing assertion: a failed gate skips promotion entirely.

name: Promote Baseline
on:
  push:
    branches: [main]

jobs:
  gate-and-promote:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: "20", cache: "npm" }
      - run: npm ci
      - name: Fetch current baseline
        run: node scripts/baseline-fetch.js --branch main --out baseline_store.json
        env: { BASELINE_STORE_URL: "${{ secrets.BASELINE_STORE_URL }}" }
      - name: Collect and gate
        id: gate
        run: |
          npx lhci collect && npx lhci upload --target=filesystem
          node scripts/baseline-compare.js --baseline baseline_store.json --run .lighthouseci/
      - name: Promote (only if gate passed)
        if: success()
        run: |
          GATE_STATUS=pass STORE_PATH=baseline_store.json WINDOW_PATH=trimmed.json \
            node scripts/promote-baseline.js > baseline_next.json
          node scripts/baseline-publish.js --in baseline_next.json --branch main
        env: { BASELINE_STORE_URL: "${{ secrets.BASELINE_STORE_URL }}" }

Verification #

After a merge, confirm the baseline advanced and recorded the new SHA. Re-run the fetch and check that computedAt is fresh and gitSha matches the merge commit.

node scripts/baseline-fetch.js --branch main --print
# → lcp p75=2150ms  sha=9b2e7c1  computedAt=2026-06-20T02:11:00Z  samples=88
git log -1 --format=%h    # → 9b2e7c1  (matches the promoted SHA)

A passing promotion shows the SHA from your latest merge and a computedAt within minutes of it. To roll back, re-publish the prior store entry by its SHA — the stamped provenance is what makes that a one-command operation. With promotion automated, the natural next step is catching regressions statistically rather than only against a fixed band; see Automated Regression Detection.

Frequently Asked Questions #