Device & Network Emulation Weighting

A Lighthouse run on a fast desktop with no throttling tells you almost nothing about the user on a mid-range Android phone over Fast 3G, yet that user may be the majority of your traffic. Emulation weighting closes that gap: it maps each synthetic device-and-network profile to its real share of traffic, then collapses the per-profile metrics into a single weighted figure the gate can compare. This guide is part of the Threshold Calibration & Baseline Management reference, and it owns the first stage of that pipeline — making sure the raw runs that feed calibration represent the users you actually have. Everything downstream, from Percentile-Based Threshold Tuning to baseline promotion, inherits whatever bias you bake into these profiles.

The discipline has three coupled parts: choosing presets that match your audience, throttling each profile correctly so the timings are deterministic, and assigning weights that sum to one so high-traffic profiles dominate the verdict. Get the throttling wrong and every downstream percentile is built on noise; get the weights wrong and the gate optimizes for a user who barely exists. The recurring failure mode teams hit is the "green on desktop, miserable on mobile" gate — a single unthrottled desktop run passes with a P75 LCP near 1500 ms while the P75 mobile experience on a moto-g4 over Fast 3G sits above 4000 ms and never enters the verdict. Weighting is the structural fix: it forces the slow-majority profile into the number the gate reads.

Throughout this guide, treat every ceiling as a P75 target measured against a named device and connection unless stated otherwise. A budget written as "LCP ≤ 3000 ms" is meaningless without "at P75 on a mid-range mobile (moto-g4) over Fast 3G" attached, because the same 3000 ms is trivially green on desktop Ethernet and a hard miss on a throttled phone. The weighting matrix exists precisely so those context-bound numbers can be combined honestly.

Architecture of the Weighting Matrix

Construct a deterministic profile matrix that maps emulation presets to observed traffic and normalizes to exactly 1.0. The matrix is the contract between your real-user distribution and your lab gate: each cell is a (device × network) profile, and its weight is that profile's share of traffic. The diagram shows how the matrix routes parallel runs into a single weighted aggregate.

To build the matrix: extract RUM traffic distribution by device class (mobile, tablet, desktop) and connection type, map each bucket to a standard emulation preset (moto-g4, desktop-chrome, 3G-fast, 4G-good), normalize every weight so the column sums to exactly 1.0, and validate that the resulting concurrent-run count fits your CI runner capacity. Pull those shares from your own field data rather than a vendor default — the Custom Performance Beacons & RUM pipeline is the cleanest source because it already segments by device and effective connection type. Store the matrix as committed JSON so the weights are versioned alongside the code and every change lands in a reviewable diff.

{
  "profiles": [
    { "id": "mobile-3g",  "device": "moto-g4",        "network": "3G-fast",  "weight": 0.45 },
    { "id": "mobile-4g",  "device": "moto-g4",        "network": "4G-good",  "weight": 0.25 },
    { "id": "desktop-4g", "device": "desktop-chrome", "network": "4G-good",  "weight": 0.20 },
    { "id": "desktop-eth","device": "desktop-chrome", "network": "desktop",  "weight": 0.10 }
  ],
  "metadata": { "version": "1.2.0", "normalized_sum": 1.0, "source": "rum_traffic_q3" }
}

A common early mistake is to weight by device sales figures or gut feel rather than session share. Weight by the traffic you actually serve at the percentile you gate on: if 62% of your P75 sessions arrive on mid-range Android over a 3G-class effective connection, that profile deserves a weight near 0.62, no matter how many desktops sit in the office. If you serve genuinely distinct audiences — logged-in power users versus anonymous first-time visitors — a single blended matrix hides the split, and you are better off maintaining separate matrices per audience the way Segmenting Baselines by Page Type segments its baselines.

How Simulated Throttling Shapes Each Profile

Before weights matter, every profile has to produce a stable timing, and that stability comes entirely from how Lighthouse throttles. Each profile passes the raw page load through two independent throttle stages: a network stage that imposes round-trip latency (rttMs) and a bandwidth cap (throughputKbps), and a CPU stage that multiplies main-thread work by cpuSlowdownMultiplier. Simulated throttling applies both in software after a fast unthrottled trace, so the reported number never depends on the runner's real bandwidth. The diagram traces one page load through both stages and contrasts the mobile and desktop settings.

The two stages are independent, and confusing them is the source of most emulation bugs. A slow LCP on the mobile-3g profile could be network-bound (the hero image cannot arrive within the 1600 kbps cap) or CPU-bound (a hydration script blocks the main thread under the 4× multiplier), and only splitting the two stages tells you which. The network numbers are portable because they are enforced in software; the CPU multiplier is not, because "4× slower than this runner" means different absolute speeds on a shared ubuntu-latest box versus a pinned self-hosted agent. That single fact is why the CPU stage needs its own calibration, covered end to end in Calibrating CPU Throttling for CI Runners.

Prerequisites and Environment

Weighting depends on deterministic emulation, which depends on a stable runner. Before wiring weights, confirm the following are in place:

• @lhci/cli ≥ 0.13 or the Lighthouse CLI ≥ 11, with Chrome ≥ 120 available on the runner. Pin exact versions so throttling semantics do not drift between minors.
• RUM or CrUX traffic shares for at least the last full quarter, broken down by device class and connection type. Without this, the weights are guesses.
• A pinned-CPU runner. Variable runner hardware silently changes the effective CPU throttle, which is the single most common source of emulation jitter — the fix is in Calibrating CPU Throttling for CI Runners.
• throttlingMethod: simulate as the default. Simulated throttling models the network in software, so timings do not depend on the runner's real bandwidth.
• A parallel CI matrix with capacity for every profile at once, wired the way GitHub Actions Performance Matrices lays out its fan-out jobs, so profiles collect concurrently instead of serially inflating wall-clock time.

Verify determinism before you trust a single weight: run the same profile three times on a quiet runner and confirm the P75 LCP spread stays under roughly 5% on a mid-range mobile (moto-g4) over Fast 3G. If the spread is wider, fix the environment first — no weighting scheme survives a profile whose own runs disagree by hundreds of milliseconds.

Configuration Reference

Generate the Lighthouse throttling config from the weighting matrix rather than hand-writing per-profile settings, so the network parameters and the weights can never disagree. Every field below is load-bearing: rttMs and throughputKbps define the simulated network, and cpuSlowdownMultiplier defines how much slower the runner pretends to be than itself.

// lighthouse-emulation-config.js — derive throttling from the weighting matrix.
const matrix = require("./weighting-matrix.json");

module.exports = matrix.profiles.map((p) => ({
  id: p.id,
  lighthouseFlags: {
    chromeFlags: ["--headless", "--no-sandbox", "--disable-dev-shm-usage"],
    formFactor: p.device.includes("desktop") ? "desktop" : "mobile",
    throttlingMethod: "simulate",
    throttling: {
      // RTT and throughput model the network in software (deterministic).
      rttMs: p.network === "3G-fast" ? 150 : p.network === "4G-good" ? 40 : 0,
      throughputKbps:
        p.network === "3G-fast" ? 1600
        : p.network === "4G-good" ? 9000
        : 10000000,
      // Mobile presets emulate a slow CPU; desktop runs unthrottled CPU.
      cpuSlowdownMultiplier: p.device.includes("desktop") ? 1 : 4,
    },
  },
}));

cpuSlowdownMultiplier: 4 is the Lighthouse mobile default, chosen to make a fast runner behave like a mid-range phone — but "4" is only correct relative to a specific runner speed, which is exactly why it needs per-runner calibration. Treat the throughputKbps: 10000000 desktop value as an intentional "effectively unthrottled" sentinel rather than a real link speed; it exists so the desktop profile is bound by CPU and render work, not by an artificial bandwidth cap.

Step-by-Step Implementation

1. Run profiles in parallel using a CI matrix so all four collect concurrently. Expected output per profile is a lh-<id>.json artifact uploaded for the aggregation job.

   name: Weighted Performance Gate
   on: [pull_request]
   jobs:
     emulate:
       runs-on: ubuntu-latest
       strategy:
         matrix:
           profile: [mobile-3g, mobile-4g, desktop-4g, desktop-eth]
         fail-fast: false
       steps:
         - uses: actions/checkout@v4
         - name: Run Lighthouse
           run: npx lighthouse https://staging.example.com --config-path=./lighthouse-emulation-config.js --output=json --output-path=./results/${{ matrix.profile }}.json
         - uses: actions/upload-artifact@v4
           with:
             name: lh-${{ matrix.profile }}
             path: ./results/${{ matrix.profile }}.json

2. Denoise each profile before weighting. Run three iterations per profile, take the median of each metric, and retry any single run more than 15% off the median. This keeps a flaky run from contaminating the weighted aggregate, following the Statistical Noise & Flakiness Reduction protocol.

3. Aggregate with weights in a job that depends on the matrix. Expected output is three printed weighted metrics and a non-zero exit on breach.

   // aggregate-weights.js — collapse per-profile medians into one weighted metric.
   const fs = require("fs");
   const path = require("path");
   const matrix = require("../weighting-matrix.json");
   
   let weightedLCP = 0, weightedINP = 0, weightedCLS = 0;
   matrix.profiles.forEach((p) => {
     const report = JSON.parse(
       fs.readFileSync(path.join("./results", `lh-${p.id}.json`), "utf8"));
     const { lcp, inp, cls } = report.audits;
     weightedLCP += lcp.numericValue * p.weight;
     weightedINP += inp.numericValue * p.weight;
     weightedCLS += cls.numericValue * p.weight;
   });
   
   console.log(`Weighted LCP: ${weightedLCP.toFixed(2)}ms`);
   console.log(`Weighted INP: ${weightedINP.toFixed(2)}ms`);
   console.log(`Weighted CLS: ${weightedCLS.toFixed(4)}`);
   
   if (weightedLCP > 3000 || weightedINP > 200 || weightedCLS > 0.1) process.exit(1);

   Expected output: Weighted LCP: 2607.50ms, Weighted INP: 188.75ms, Weighted CLS: 0.0925, exit 0.

Reading the Weighted Aggregate

The weighted LCP is a weighted sum, so it is worth seeing where each millisecond of the final number comes from. Take the worked example above: with per-profile median LCPs of 3300 ms (mobile-3g), 2450 ms (mobile-4g), 1750 ms (desktop-4g), and 1600 ms (desktop-eth) — all P75 figures against their respective device and connection — the weighted total of 2607.5 ms decomposes into the contributions below. The mobile-3g profile alone supplies more than half the final number despite being one of four runs, because its 0.45 weight and its slow timing compound.

This decomposition is the reason the weighted gate behaves the way you want: a regression concentrated in the mobile-3g profile moves the aggregate hard, while the same absolute regression on desktop-eth barely nudges it. It also exposes the risk. Because the low-weight profiles contribute so little, a serious desktop-eth regression — say P75 LCP jumping from 1600 ms to 2600 ms on desktop Ethernet — adds only about 100 ms to the aggregate and can pass unnoticed. That is why the weighted error gate always ships alongside per-profile warn floors, so a minority profile cannot rot silently behind a green aggregate.

Threshold Calibration

The throttling numbers are not arbitrary — each profile maps to a measured device and link. The table below is the calibration reference: it pairs each preset with its CPU multiplier and the RTT and throughput that define its simulated network, plus a representative weighted-target LCP. Re-derive the weights from your own traffic, and verify the CPU multiplier against your runner per Calibrating CPU Throttling for CI Runners.

Every ceiling in that column is a P75 target measured against the device and connection on the same row: the 3000 ms mobile-3g ceiling means "P75 LCP on a moto-g4 over Fast 3G," and the 1500 ms desktop-eth ceiling means "P75 LCP on desktop Chrome over Ethernet." They are not interchangeable. The mobile profiles carry the heaviest weights and the loosest ceilings on purpose: they represent the slowest realistic hardware most of your users hold, so the weighted figure leans toward their experience. How those mobile and desktop ceilings diverge in the budget itself is covered in Mobile vs Desktop Budget Divergence.

A note on the aggregate ceiling versus the per-profile ceilings. The weighted error gate in aggregate-weights.js fires at 3000 ms, but that is the ceiling for the blended number, not for any one profile. Because the mobile profiles dominate the weighting, the blended ceiling sits close to the mobile-3g ceiling by construction — which is the intended behavior. If your traffic were desktop-heavy, the same math would pull the aggregate ceiling downward, and you would re-derive it rather than reuse 3000 ms.

Choosing Device and Network Presets

Adding a profile for every device-and-connection combination in your analytics is a trap: three or four well-chosen profiles capture nearly all of the variance, and each extra profile costs a concurrent runner slot and a share of the weight it steals from more representative buckets. The rule of thumb is to only stand up a profile for a bucket that holds at least 5% of P75 sessions; fold anything thinner into its nearest neighbor. The decision tree formalizes that choice.

Connection tier is the second axis. Map the bucket's median effective connection type to a preset — a 3g or slow-4g effective type lands on the 3G-fast row (150 ms RTT, 1600 kbps), while 4g lands on the 4G-good row (40 ms RTT, 9000 kbps). Where a bucket straddles two tiers, prefer the slower one; gating on the more pessimistic P75 profile is cheaper than shipping a regression that only your slowest users feel. Whether you should target P75 or the stricter P90 for these ceilings is a separate calibration decision, unpacked in Choosing Between P75 and P90 Budget Targets.

CI Enforcement Snippet

Gate on the weighted aggregate, not on any single profile, and require the check in branch protection. This step runs after the matrix and the denoise step, failing the build only when the weighted metric crosses its calibrated ceiling.

  aggregate:
    needs: emulate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/download-artifact@v4
        with:
          path: ./results
      - name: Validate weight normalization
        run: node -e "const m=require('./weighting-matrix.json');const s=m.profiles.reduce((a,p)=>a+p.weight,0);if(Math.abs(s-1)>0.001){console.error('weights sum to '+s);process.exit(1)}"
      - name: Weighted aggregation gate
        run: node ./scripts/aggregate-weights.js

The normalization check fails fast if the matrix has been edited to sum to anything other than 1.0, which would silently skew every weighted metric. For network isolation beyond simulated throttling, route the slowest profile through a dedicated agent per WebPageTest Private Instance Setup.

Troubleshooting and Edge Cases

• Weighted metric jitters between builds → the CPU throttle is drifting with runner hardware; pin the runner and calibrate the multiplier per Calibrating CPU Throttling for CI Runners.
• Weights no longer match traffic → inject a fresh matrix via a TRAFFIC_MATRIX_OVERRIDE environment variable and fall back to the committed JSON on parse failure, so a stale matrix never blocks merges.
• One profile is consistently flaky → it is likely the 3G profile saturating the simulated link; raise its iteration count to five and median-filter before weighting.
• Aggregate passes but a single mobile profile is terrible → weighted gating can mask a profile-level failure; add a per-profile warn floor alongside the weighted error gate so regressions in a minority profile are still visible.
• Concurrent runs exceed runner capacity → reduce matrix parallelism or stagger profiles; oversubscribed runners inflate CPU contention and corrupt the very emulation you are trying to control.
• Desktop and mobile share a ceiling → they should not; split the budgets so the unthrottled desktop run is held to a tighter P75 LCP (≤ 1500 ms on desktop Ethernet) than the throttled mobile run (≤ 3000 ms on moto-g4 over Fast 3G).
• A new route regresses only for one audience → a single blended matrix cannot see it; maintain per-audience matrices and compare weighted aggregates side by side rather than averaging the audiences away.

Frequently Asked Questions