Building an Executive Performance Scorecard

An executive performance scorecard is a single page that answers one question in five seconds: is our web experience getting faster or slower, and is that costing us money? It is part of the Performance Budget Reporting and Scorecards reference, and it deliberately hides almost everything an engineer would want. Where an engineering dashboard shows twenty charts, waterfalls, and per-commit regressions, the executive view shows three numbers, a colour, and an arrow. The discipline of a good scorecard is subtraction: every widget you remove makes the surviving signal louder.

This guide walks through the few metrics leaders actually care about, the red/amber/green thresholds that make the page glanceable, the data pipeline that produces those numbers on a schedule, a concrete one-page layout, the reporting cadence, and how to verify the scorecard is telling the truth. Throughout, treat the scorecard as a rollup of the same budget signals your CI already gates on — it should never invent its own definition of "fast."

The Three Metrics Executives Actually Read

Leaders do not read latency histograms. In practice a scorecard needs exactly three things on it. The first is budget compliance percentage: of all monitored routes, what fraction met every Core Web Vitals budget they were assigned, measured at the 75th percentile on mid-range mobile over Fast 3G. The second is revenue-linked route health: the compliance status of the handful of pages that actually make money — the homepage, the product detail page, the cart, and checkout — because a 92% site-wide pass rate means nothing if checkout is the 8% that failed. The third is a trend arrow: is compliance up, flat, or down versus the trailing four-week baseline, so a leader instantly knows whether the situation is improving or eroding.

Everything else — INP breakdowns, third-party attribution, per-commit deltas — belongs on the engineering surface, not here. The scorecard's job is to route attention, not to diagnose. When compliance drops and the arrow turns down, the executive's next action is to ask the engineering lead why, and the lead opens the detailed dashboard. Keep those two surfaces linked but separate.

Setting Red, Amber, and Green Thresholds

Colour is the fastest channel a scorecard has, so the thresholds behind each colour must be defensible and stable. Define them once, write them into the performance budget policy, and do not move the goalposts between reports. A route is counted as compliant when it meets every assigned Core Web Vitals ceiling — LCP at or under 2500 ms at P75 on mid-range mobile over Fast 3G, INP at or under 200 ms at P75 on that same class, and CLS at or under 0.10 at P75 — measured over a rolling seven-day window of real-user data.

The aggregate compliance colour then follows a simple band. Green means at least 90% of monitored routes are compliant; amber means 75% to 89%; red means below 75%. For an individual revenue route, colour is binary-plus: green if it passed all three vitals at P75, amber if it passed at P75 but slipped at P90 on mid-range mobile over Fast 3G (an early-warning state), and red if it failed any vital at P75. The P90 amber tier matters because a route can look healthy at the median-ish P75 while the slowest quarter of real sessions already suffers; surfacing that as amber buys a sprint of lead time before the P75 itself degrades.

The Data Pipeline That Produces It

The scorecard is only credible if it is derived, not hand-assembled. Build a small pipeline that reuses the field data you already collect. Real-user measurements arrive from your custom RUM beacons, land in a metrics store, and get rolled up by the same P75/P99 aggregation pipeline that feeds your engineering dashboards. A nightly job computes per-route percentiles over the trailing seven days, compares each route to its assigned budget, and writes a single compliance row per route. A second query aggregates those rows into the site-wide percentage and the four-week trend.

Compute the percentiles in the store rather than in the reporting layer so every surface agrees on the numbers. A representative aggregation query looks like this:

WITH per_route AS (
  SELECT
    route,
    APPROX_QUANTILES(lcp_ms, 100)[OFFSET(75)]  AS lcp_p75,
    APPROX_QUANTILES(inp_ms, 100)[OFFSET(75)]  AS inp_p75,
    APPROX_QUANTILES(cls,    100)[OFFSET(75)]  AS cls_p75
  FROM rum_events
  WHERE device_class = 'mid-range-mobile'
    AND effective_connection = 'fast-3g'
    AND event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY)
  GROUP BY route
)
SELECT
  route,
  lcp_p75, inp_p75, cls_p75,
  (lcp_p75 <= 2500 AND inp_p75 <= 200 AND cls_p75 <= 0.10) AS is_compliant
FROM per_route;

The reporting layer then only has to count green rows and format them. Keeping the compliance boolean in the query means the scorecard, the weekly email, and the engineering dashboard cannot disagree about what "compliant" meant on a given day. A tiny transform turns the rows into the scorecard payload:

function buildScorecard(routeRows, revenueRoutes, priorPct) {
  const total = routeRows.length;
  const passing = routeRows.filter((r) => r.is_compliant).length;
  const compliancePct = Math.round((passing / total) * 100);
  const band =
    compliancePct >= 90 ? "green" : compliancePct >= 75 ? "amber" : "red";
  const revenue = revenueRoutes.map((name) => {
    const row = routeRows.find((r) => r.route === name);
    return { route: name, status: row && row.is_compliant ? "green" : "red" };
  });
  const delta = compliancePct - priorPct;
  const trend = delta > 1 ? "up" : delta < -1 ? "down" : "flat";
  return { compliancePct, band, revenue, trend, delta };
}

A One-Page Layout That Fits on a Slide

Design the scorecard to survive being pasted into a slide deck or a phone email preview. Use a three-column top row for the headline metrics, a compact revenue-route strip below it, and a single trailing sparkline that shows compliance over the last twelve weeks. Everything above the fold must be legible at thumbnail size. Reserve the bottom third for a one-line "what changed and what we are doing about it" note written by the engineering lead — leaders trust a scorecard far more when a human annotates the anomaly rather than leaving them to guess.

Keep the palette to the three status colours plus neutral text, and never encode meaning in colour alone: pair every colour with a word or icon so the page still works when printed in greyscale or viewed by a colour-blind reader. The sparkline connects this static snapshot to the fuller story of tracking budget compliance over time, which is where a leader clicks through when the trend arrow raises a question.

Cadence, Ownership, and Distribution

A scorecard nobody receives on a rhythm becomes a dashboard nobody opens. Publish it weekly, on the same weekday and hour, so it becomes a fixture rather than an event. The nightly rollup runs every night, but the executive summary snapshots once a week and is delivered as a static image plus a link — most of your audience will read it in an email preview and never click. That weekly push pairs naturally with the weekly performance budget email digests that carry the engineering-facing detail.

Assign a named owner — usually the performance lead — who is accountable for the annotation line and for the numbers being current. When compliance goes red, the owner adds one sentence of context before the scorecard ships; an unexplained red erodes trust faster than the regression itself. Archive every weekly snapshot so the trend arrow is reproducible and so a quarter-over-quarter narrative can be assembled without re-querying raw events. Render the sparkline and status tiles in Grafana if that is where your panels already live, then export the panel image for the email.

Verifying the Scorecard Tells the Truth

Before a scorecard drives decisions, prove it matches ground truth. Spot-check three routes each week by pulling their raw P75 for LCP, INP, and CLS straight from the store and confirming the compliance flag on the scorecard agrees. Confirm the device and connection filter is applied — a scorecard computed over unfiltered desktop-and-mobile traffic will read artificially green because desktop over cable easily clears budgets that mid-range mobile over Fast 3G does not. Finally, assert that the sample size behind each route is large enough that the P75 is stable; a route with only a few dozen sessions in the window will jitter and produce a jumpy trend arrow.

# Verify one route's scorecard flag against raw field data.
curl -s "https://metrics.internal/api/route?path=/checkout&days=7&device=mid-range-mobile&conn=fast-3g" \
  | jq '{
      route: .path,
      samples: .count,
      lcp_p75: .lcp_p75,
      inp_p75: .inp_p75,
      cls_p75: .cls_p75,
      expected_compliant: (.lcp_p75 <= 2500 and .inp_p75 <= 200 and .cls_p75 <= 0.10),
      enough_data: (.count >= 1000)
    }'

If expected_compliant disagrees with the colour on the scorecard, or enough_data is false, hold the route out of the aggregate until the pipeline is fixed. A scorecard that is occasionally wrong is worse than no scorecard, because it teaches leaders to distrust every green they see.

Frequently Asked Questions