Visualizing Budget Trends With Grafana

A performance budget that lives only in a CI assertion file is invisible until it breaks the build, and by then the regression is already merged and context away from the engineer who can fix it. This guide, part of the Dashboarding & Team Adoption reference, turns budgets into something a whole team watches: a Grafana board where LCP, INP, CLS, and byte budgets trend over time with a horizontal threshold line on every panel, broken down per route, so a creeping P75 is caught at the slope rather than at the cliff.

The work has three coupled concerns — where the numbers come from (a time-series data source fed by RUM and CI), how they are shaped into panels (queries that compute percentiles per route), and how the budget is drawn (a threshold line and color regions that make a breach unmistakable). Get the ingestion model wrong and every panel inherits gaps; get the query granularity wrong and per-route signal drowns in a site-wide average. The sections below build the board from the store outward, then close the loop by pushing CI numbers back into the same store so the trend and the gate never disagree.

Architecture Overview

Grafana never collects metrics itself — it queries a backing store. Two producers write into that store: a real-user-monitoring beacon emitting field metrics from browsers, and Lighthouse CI emitting lab metrics from each pipeline run. Both land in a time-series database (Prometheus for counters and gauges, or Postgres/TimescaleDB when you need exact percentiles over raw samples), and Grafana reads from there. Keeping the two producers on distinct, labelled series is not a nicety — it is the difference between a board a team trusts and one they learn to ignore.

The two producers answer different questions and must be kept on separate series. RUM tells you what users actually experience and is the source of truth for budget calibration; the lab signal from your Lighthouse pipeline tells you what a controlled environment measured and is what the gate enforces. Feed both, label them, and never average them together. A single mixed series that blends a throttled desktop lab run at 1400 ms with a field P75 of 2600 ms on mid-range Android over 4G produces a number that describes no real user and no real test — the worst of both worlds.

Prerequisites and Environment

• Grafana ≥ 10.x — self-hosted or cloud. The provisioning files and unified-alerting semantics referenced here assume 10.x. On older releases the threshold mode names differ and the panel JSON below will not import cleanly.
• A time-series data source — Prometheus (with the Pushgateway or a remote-write target for CI batch jobs) or PostgreSQL/TimescaleDB. Postgres is preferred when you need exact percentiles from raw beacon rows rather than histogram approximations.
• A RUM ingestion path — the beacon and aggregation pipeline from Custom Performance Beacons & RUM, writing one row or sample per metric per route. If you have not settled on a field pipeline yet, the P75/P99 aggregation work there is the prerequisite that makes exact percentile panels possible.
• An LHCI source — if you store lab runs in the Self-Hosting the Lighthouse CI Server Postgres database, Grafana can query it directly; otherwise push CI numbers to the time-series store explicitly (shown in the CI Enforcement section).

Provision the data source as code so the board is reproducible across environments and a rebuilt Grafana instance comes back with the same connections:

# /etc/grafana/provisioning/datasources/perf.yaml
apiVersion: 1
datasources:
  - name: PerfTSDB
    type: postgres
    access: proxy
    url: timescale.internal:5432
    user: grafana_ro
    jsonData:
      database: perf_metrics
      sslmode: require
      postgresVersion: 1500
      timescaledb: true
    secureJsonData:
      password: ${PERF_DB_PASSWORD}

Grant the Grafana user read-only access. A dashboard credential that can only SELECT cannot be turned into a data-exfiltration or mutation path if the instance is ever exposed, and it keeps a rogue panel query from locking rows the ingestion writers need.

Choosing the Backing Store

The single largest design decision is which store sits behind the board, because it dictates how you compute percentiles. Prometheus stores pre-bucketed histograms and answers with histogram_quantile, which interpolates the P75 from bucket boundaries — cheap, fast, and precise enough for trend shape, but only as accurate as your bucket layout. Postgres with TimescaleDB stores raw samples and answers with percentile_cont(0.75), an exact interpolated percentile over the actual observations, at the cost of heavier queries you tame with continuous aggregates. The decision tree below is the short version.

Most teams that already run a beacon into a SQL store should stay in Postgres — the exact percentile and the freedom to slice by any label (device class, connection type, logged-in state) outweigh the query cost once continuous aggregates are in place. Reach for Prometheus when Web Vitals are already exported as histograms alongside your infrastructure metrics and you would rather not run a second database.

Configuration Reference

A Grafana panel is JSON. The block below is one time-series panel that plots the P75 LCP trend for a route and draws the budget as a threshold step with a colored region above it. Every field that matters for budget visualization is annotated after the block.

{
  "title": "LCP P75 — /checkout",
  "type": "timeseries",
  "datasource": { "type": "postgres", "uid": "PerfTSDB" },
  "fieldConfig": {
    "defaults": {
      "unit": "ms",
      "custom": { "lineWidth": 2, "fillOpacity": 8, "thresholdsStyle": { "mode": "line+area" } },
      "thresholds": {
        "mode": "absolute",
        "steps": [
          { "value": null, "color": "green" },
          { "value": 2500, "color": "red" }
        ]
      }
    }
  },
  "targets": [
    {
      "refId": "A",
      "format": "time_series",
      "rawSql": "SELECT time_bucket('1 hour', ts) AS time, percentile_cont(0.75) WITHIN GROUP (ORDER BY value) AS lcp_p75 FROM web_vitals WHERE metric = 'LCP' AND route = '/checkout' AND source = 'field' AND $__timeFilter(ts) GROUP BY 1 ORDER BY 1"
    }
  ]
}

The thresholds.steps array is what makes the budget visible: a green floor and a red step at 2500 ms — the P75 LCP ceiling for a mid-range Android phone on a 4G connection. With thresholdsStyle.mode set to line+area Grafana draws a horizontal line at the budget and shades the breach region red, so a rising trend that crosses it is impossible to miss. time_bucket('1 hour', ...) and percentile_cont(0.75) compute an exact hourly P75 — do not substitute avg(), which hides the tail that the budget actually governs. Note the explicit source = 'field' filter: it keeps this panel on the RUM series so a lab spike from CI never distorts the field trend.

For a Prometheus source the same panel uses a PromQL target instead, reading a pre-aggregated histogram:

histogram_quantile(0.75, sum by (le, route) (rate(web_vitals_lcp_bucket{route="/checkout"}[1h])))

Anatomy of a Budget Panel

The reason a threshold line beats a raw number is that it turns a budget from a pass/fail assertion into a distance. A panel reading "LCP P75 = 2470 ms" against a 2500 ms budget looks fine; the same value plotted as the last point on a line that has climbed 60 ms a week for six weeks reads as a breach that is two deploys away. The chart below is the shape you are building toward: a per-route P75 trend rising into a dashed budget line, with the breach highlighted the moment the line crosses.

A panel like this earns its place because it changes the conversation from "did we pass" to "where are we heading". The week-5 value of 2380 ms is technically inside budget, but the trend makes clear that at roughly 80 ms of drift per week the route will breach within two sprints unless something changes. That early warning is the entire point of trending instead of gating alone, and it is why this board pairs naturally with the scheduled synthetic runs described in Continuous Performance Monitoring — the board shows drift, the schedule guarantees fresh points to draw it from.

Step-by-Step Implementation

1. Connect the data source. Apply the provisioning file (or add it in Connections → Data sources) and click Save & test.

   curl -s -u admin:$GRAFANA_PW http://grafana.internal/api/datasources/name/PerfTSDB | jq '.type,.id'

   Expected output: "postgres" and a numeric id, confirming Grafana resolved the source.

2. Build the panel. Create a new dashboard, add a Time series panel, select PerfTSDB, and paste the rawSql from the configuration reference. The graph should render a continuous P75 line for the selected route.

3. Add the budget threshold. In the panel's Thresholds section add a red step at the route's budget value and set Show thresholds to As lines (dashed) and regions. The dashed line and red region appear immediately.

   Expected result: the panel shows the live trend below a dashed budget line, with any historical breach already shaded red.

4. Templatize the route. Add a dashboard variable route of type Query (SELECT DISTINCT route FROM web_vitals) and replace the literal /checkout in the SQL with $route. One panel now serves every route, and a repeating row gives a per-route grid.

5. Add a device-class filter. Add a second variable device (SELECT DISTINCT device_class FROM web_vitals) and append AND device_class = '$device' to the query. Because a 2500 ms LCP budget for mid-range mobile on 4G is a different line from the 1200 ms desktop-broadband P75 budget, the filter lets one panel honor both without averaging device classes into a meaningless blend.

6. Save with a JSON model in version control. Export the dashboard JSON and commit it beside your infrastructure code. A board that is code, not clicks, survives a Grafana rebuild and reviews like anything else.

Threshold Calibration

Pull the budget value for each threshold line from field data, not the lab number. Read the P75 of each metric from your RUM store over a trailing 28-day window, then set the line at the budget you are committing to — usually field-P75 rounded to the nearest target band. The windows below control how much smoothing each panel applies; tighter buckets surface regressions sooner but show more noise.

Set the threshold line from a Grafana dashboard variable rather than hardcoding it per panel, so a budget change is one edit that propagates across every route in the repeating grid. For the percentile method behind these values — why P75 and not P90 for LCP, and when a P90 ceiling is the honest choice — see Percentile-Based Threshold Tuning. When the trend line crossing the budget should also page someone rather than merely display, wire the same threshold into Alerting on Performance Budget Regressions so the panel and the alert rule read the same number.

CI Enforcement

A trend board is only honest if the lab signal it shows is the same number the gate enforces. After Lighthouse CI asserts, push the median metrics to the same store the panels read, tagged with the commit so a step appears on the trend exactly when a change lands. The flow is four short stages, and the diagram makes the ownership of each explicit.

- name: Push LHCI metrics to TSDB
  if: always()
  run: |
    LCP=$(jq '.audits["largest-contentful-paint"].numericValue' .lighthouseci/lhr-*.json | sort -n | awk '{a[NR]=$1} END{print a[int(NR/2)+1]}')
    psql "$PERF_DB_URL" -c "INSERT INTO web_vitals(ts, metric, route, value, source, sha)
      VALUES (now(), 'LCP', '/checkout', ${LCP}, 'lab', '${{ github.sha }}');"
  env:
    PERF_DB_URL: ${{ secrets.PERF_DB_URL }}

Tagging rows with source = 'lab' keeps the CI series separate from RUM on the panel, and the sha column lets a Grafana annotation query mark each deploy on the timeline. That annotation is what makes a step interpretable: a 300 ms jump on the lab trend that lines up exactly with a deploy marker points straight at the pull request to bisect, which is precisely the kind of signal that Automated Regression Detection formalizes into a statistical trigger rather than an eyeball read.

Making the Board a Team Habit

A dashboard nobody opens is as invisible as a budget in an assertion file. The board earns attention when it is wired into the rituals a team already has. Pin the per-route grid to a wall display or a team channel snapshot so the trend is ambient, not sought out. Add a scorecard row at the top — one stat panel per core metric showing the current field P75 against its budget as a colored value — so a glance answers "are we green" before anyone reads a single line chart; that rollup is the seed of the exec-facing view built in Performance Budget Reporting and Scorecards. Tie a breach to an owner: the same threshold that shades a panel red should map to a named route owner, which is exactly the accountability model that Driving Team Performance Budget Adoption turns into a written policy.

Teams standardized on a different observability stack do not need to abandon this design — the pattern of one labelled store, per-route percentile panels, and a threshold line drawn from field P75 transfers directly. If your organization already lives in Datadog, the same field-versus-lab discipline and per-route threshold monitors are covered in Integrating Performance Budgets With Datadog; the tool changes but the budget-as-a-visible-line principle does not.

Troubleshooting and Edge Cases

• Panel shows gaps → the producer stopped writing or a route had no traffic in a bucket; set Connect null values to Threshold and verify the beacon is still emitting. A low-traffic route with an hourly bucket will show gaps that are real absence of data, not failure — widen the bucket for such routes.
• P75 looks too flat → you are reading an avg() not a percentile; switch to percentile_cont(0.75) or histogram_quantile. An average of LCP will sit far below the P75 the budget governs and will lie about compliance.
• Threshold line missing → Show thresholds defaults to Off on new panels; set it to lines+regions explicitly, and confirm the threshold mode is absolute not percentage.
• Lab and field series diverge wildly → that is expected; the lab is throttled and synthetic while the field is real devices. A lab P75 of 1900 ms against a field P75 of 2600 ms on mid-range mobile over 4G is normal. Keep them on separate panels, not one overlay.
• Slow dashboard on Postgres → add a composite index on (metric, route, source, ts) and rely on Timescale continuous aggregates instead of querying raw rows on every refresh.
• Route cardinality explodes → normalize dynamic segments (/product/123 → /product/:id) in the beacon before storage, or the variable dropdown becomes unusable and the per-route grid renders hundreds of near-empty panels.
• Deploy annotations cluster unreadably → filter the annotation query to the main branch only, so feature-branch CI pushes do not litter the production trend with markers.

Frequently Asked Questions