Synthetic Monitoring vs RUM: Trade-offs for Budget Gating

Teams reach for one number to gate a deploy and discover the two sources disagree: a synthetic Lighthouse run on an emulated Moto G Power over Slow 4G says LCP is 2.1s, while field data reports a P75 of 3.4s on mid-range Android over 4G. Neither is wrong — they measure different populations under different conditions. This page, part of the Comparing Performance Testing Tools reference, breaks down exactly where synthetic (lab) monitoring and real-user monitoring (RUM) diverge, and which signal belongs on which side of your gate. Generic "use both" advice fails because it never says which one blocks the merge — and gating on the wrong one either ships regressions or fails builds on noise you cannot reproduce.

The short answer sets up everything below: gate the merge on synthetic, validate in production with RUM. The rest of this page shows how to quantify the gap between the two, how to derive one from the other, and where each signal quietly lies to you if you trust it in the wrong place.

Where the Two Signals Diverge

Synthetic monitoring runs a controlled test — fixed device, fixed network, warm or cold cache, in CI before deploy. It is deterministic and reproducible, which is exactly what a merge gate needs, but it samples a single environment that may not match your users. RUM collects Custom Performance Beacons & RUM from actual sessions across every device, network, and geography — the ground truth — but it is noisy, arrives after deploy, and cannot block a pull request that has not shipped yet.

The practical split follows from that table: synthetic owns pre-deploy enforcement because it is the only signal that exists before code ships and the only one deterministic enough to fail a build fairly. RUM owns post-deploy verification and long-term drift detection because it is the only signal that reflects what users actually experienced. The diagram below traces a single change from pull request to production and shows where each signal enters the pipeline.

INP and CLS are the exception worth calling out: they depend on real interaction and layout shift accumulated over a full session, so lab values are directional at best. Treat a synthetic INP regression as a warning, and confirm the true P75 against field data calibrated with Percentile-Based Threshold Tuning. If your comparison is specifically synthetic-versus-field for those interaction metrics, the deeper contrast lives in WebPageTest vs RUM for Field Data.

Which Metric Belongs to Which Signal

Not every metric earns a blocking gate, and not every metric should defer to the field. Load metrics that are dominated by bytes on the wire and render-blocking work — LCP, Total Blocking Time, and byte budgets — are reproducible in the lab to within a 3–6% coefficient of variation, so they gate cleanly as errors. Interaction metrics depend on a real human hitting a real main thread that is busy with real third-party work, so their honest value only appears in the field P75. Availability and drift are inherently post-deploy questions. The decision tree below routes each class of metric to the signal that can actually answer for it.

This routing also explains why a "single performance score" gate is a trap. Averaging an error-worthy LCP with a directional INP produces a number that can pass while INP quietly regresses, or fail while nothing users feel has changed. Gate on the individual metrics that each signal can defend, and reserve the composite score for the human-facing scorecard rather than the machine gate.

Diagnostic Steps

Quantify the gap between your own lab and field numbers before deciding how tightly to gate. First, pull the synthetic median from a Lighthouse run:

npx lighthouse https://www.example.com/ \
  --only-categories=performance \
  --output=json --output-path=./lab.json --quiet
node -e "const r=require('./lab.json').audits; \
  console.log('lab LCP', r['largest-contentful-paint'].numericValue|0, 'ms');"
# → lab LCP 2120 ms

Then compute the field P75 for the same metric and URL from your beacon store, and compare:

# lab-vs-field gap: positive means the field is slower than the lab
node -e "const lab=2120, fieldP75=3400; \
  console.log('gap', (((fieldP75-lab)/lab)*100).toFixed(0)+'%');"
# → gap 60%

A gap above roughly 30–40% means your synthetic profile is too optimistic — tighten the lab throttling toward a mid-range device on Fast 3G so the gate is honest, rather than tightening the threshold and failing every build. The chart below plots the two values from this example so the size of the gap is legible at a glance, along with the derived lab budget the gate will actually enforce.

Implementation

Make synthetic the blocking gate and RUM the post-deploy check, with the lab ceiling derived from the field P75 minus a calibration margin. The script below turns a measured field P75 into the lab budget the gate should enforce:

// derive-lab-budget.js — set the synthetic gate from field reality
const FIELD_P75 = { lcp: 3400, inp: 240 }; // ms, from your RUM store (mobile, 4G)
const LAB_MARGIN = 0.85; // lab runs ~15% faster than field P75

const labBudget = Object.fromEntries(
  Object.entries(FIELD_P75).map(([metric, p75]) => [
    metric,
    Math.round(p75 * LAB_MARGIN),
  ]),
);

console.log(JSON.stringify(labBudget));
// → {"lcp":2890,"inp":204}
// LCP gates as an error (lab-faithful); INP stays a warning (needs real input)

The margin is not a guess you keep forever. Recompute it every time you build a fresh field distribution from your P75/P99 aggregation pipeline, because device mix and network conditions drift with your audience. A margin that held at 0.85 for a mostly-desktop audience can slide to 0.75 as mobile share grows, and a stale margin silently loosens the gate.

CI Gating Assertion

Gate the merge on the synthetic LCP (deterministic) and keep synthetic INP as a non-blocking warning, since its real value only emerges from field interaction. Add this to lighthouserc.json:

{
  "ci": {
    "assert": {
      "assertions": {
        "metric-lcp": ["error", { "maxNumericValue": 2890 }],
        "interaction-to-next-paint": ["warn", { "maxNumericValue": 204 }],
        "cumulative-layout-shift": ["warn", { "maxNumericValue": 0.1 }]
      }
    }
  }
}

Verification

Confirm the split works end to end. Pre-deploy, the synthetic LCP assertion should fail a deliberately regressed branch — push an oversized hero image and watch the gate exit non-zero with metric-lcp failure. Post-deploy, the field P75 LCP for that route should move in the same direction within a traffic window of a few hours; if the lab passed but the field P75 climbs past the 2890 ms budget on mobile over 4G, your synthetic profile is still too optimistic and needs recalibration. A healthy setup shows the lab-to-field gap holding steady below roughly 30% release over release — track it on the same dashboard you build in Visualizing Budget Trends with Grafana, and route sustained widening of that gap into Alerting on Synthetic Performance Drift so nobody has to eyeball the chart.

Common Failure Modes

Three patterns account for most misfires when teams first split the signals. The first is gating on a synthetic INP that reads 90 ms on a fast emulated CPU while the field P75 sits at 260 ms on a mid-range Android over 4G — the gate stays green while users suffer, because the lab main thread never carries the third-party and hydration cost that real sessions do. The fix is the routing above: INP is a warning in the lab and an error only against field data. The second is a synthetic profile that is too fast — running unthrottled on a fast CI runner produces a lab LCP near 1.2s that no real user will ever see, so the derived budget is meaningless. Pin the emulation to a mid-range device on Fast 3G or Slow 4G and keep it stable across runs. The third is treating a single noisy RUM window as truth and rolling back on it; a field P75 built from a few hundred sessions swings 15–25% run to run, so require two consecutive breaching windows before any automated action. If you are weighing whether to keep the whole RUM pipeline at all versus leaning harder on beacons, the head-to-head is in Lighthouse CI vs Custom RUM Beacons.

Frequently Asked Questions