Visualizing Budget Trends With Grafana #

A performance budget that lives only in a CI assertion file is invisible until it breaks the build, and by then the regression is already merged context away from the engineer who can fix it. This guide, part of the Dashboarding & Team Adoption reference, turns budgets into something a whole team watches: a Grafana board where LCP, INP, CLS, and byte budgets trend over time with a horizontal threshold line on every panel, broken down per route, so a creeping P75 is caught at the slope rather than at the cliff.

The work has three coupled concerns — where the numbers come from (a time-series data source fed by RUM and CI), how they are shaped into panels (queries that compute percentiles per route), and how the budget is drawn (a threshold line and color regions that make a breach unmistakable). Get the ingestion model wrong and every panel inherits gaps; get the query granularity wrong and per-route signal drowns in a site-wide average.

Architecture Overview #

Grafana never collects metrics itself — it queries a backing store. Two producers write into that store: a real-user-monitoring beacon emitting field metrics from browsers, and Lighthouse CI emitting lab metrics from each pipeline run. Both land in a time-series database (Prometheus for counters and gauges, or Postgres/TimescaleDB when you need exact percentiles over raw samples), and Grafana reads from there.

The two producers answer different questions and must be kept on separate series. RUM tells you what users actually experience and is the source of truth for budget calibration; the lab signal from your Lighthouse pipeline tells you what a controlled environment measured and is what the gate enforces. Feed both, label them, and never average them together.

Prerequisites & Environment #

• Grafana ≥ 10.x — self-hosted or cloud. Provisioning files and unified alerting referenced here assume 10.x semantics.
• A time-series data source — Prometheus (with the Pushgateway or a remote-write target for CI batch jobs) or PostgreSQL/TimescaleDB. Postgres is preferred when you need exact percentiles from raw beacon rows rather than histogram approximations.
• A RUM ingestion path — the beacon and aggregation pipeline from Custom Performance Beacons & RUM, writing one row or sample per metric per route.
• An LHCI source — if you store lab runs in the Self-Hosting the Lighthouse CI Server Postgres database, Grafana can query it directly; otherwise push CI numbers to the time-series store explicitly (shown below).

Provision the data source as code so the board is reproducible across environments:

# /etc/grafana/provisioning/datasources/perf.yaml
apiVersion: 1
datasources:
  - name: PerfTSDB
    type: postgres
    access: proxy
    url: timescale.internal:5432
    user: grafana_ro
    jsonData:
      database: perf_metrics
      sslmode: require
      postgresVersion: 1500
      timescaledb: true
    secureJsonData:
      password: ${PERF_DB_PASSWORD}

Configuration Reference #

A Grafana panel is JSON. The block below is one time-series panel that plots the P75 LCP trend for a route and draws the budget as a threshold step with a colored region above it. Every field that matters for budget visualization is annotated after the block.

{
  "title": "LCP P75 — /checkout",
  "type": "timeseries",
  "datasource": { "type": "postgres", "uid": "PerfTSDB" },
  "fieldConfig": {
    "defaults": {
      "unit": "ms",
      "custom": { "lineWidth": 2, "fillOpacity": 8, "thresholdsStyle": { "mode": "line+area" } },
      "thresholds": {
        "mode": "absolute",
        "steps": [
          { "value": null, "color": "green" },
          { "value": 2500, "color": "red" }
        ]
      }
    }
  },
  "targets": [
    {
      "refId": "A",
      "format": "time_series",
      "rawSql": "SELECT time_bucket('1 hour', ts) AS time, percentile_cont(0.75) WITHIN GROUP (ORDER BY value) AS lcp_p75 FROM web_vitals WHERE metric = 'LCP' AND route = '/checkout' AND $__timeFilter(ts) GROUP BY 1 ORDER BY 1"
    }
  ]
}

The thresholds.steps array is what makes the budget visible: a green floor and a red step at 2500 ms (the high-end-mobile 4G P75 LCP ceiling). With thresholdsStyle.mode set to line+area Grafana draws a horizontal line at the budget and shades the breach region red, so a rising trend that crosses it is impossible to miss. time_bucket('1 hour', ...) and percentile_cont(0.75) compute an exact hourly P75 — do not substitute avg(), which hides the tail that the budget actually governs.

For a Prometheus source the same panel uses a PromQL target instead, reading a pre-aggregated histogram:

histogram_quantile(0.75, sum by (le, route) (rate(web_vitals_lcp_bucket{route="/checkout"}[1h])))

Step-by-Step Implementation #

1. Connect the data source. Apply the provisioning file (or add it in Connections → Data sources) and click Save & test.

   curl -s -u admin:$GRAFANA_PW http://grafana.internal/api/datasources/name/PerfTSDB | jq '.type,.id'

   Expected output: "postgres" and a numeric id, confirming Grafana resolved the source.

2. Build the panel. Create a new dashboard, add a Time series panel, select PerfTSDB, and paste the rawSql from the configuration reference. The graph should render a continuous P75 line for the selected route.

3. Add the budget threshold. In the panel's Thresholds section add a red step at the route's budget value and set Show thresholds to As lines (dashed) and regions. The dashed line and red region appear immediately.

   Expected result: the panel shows the live trend below a dashed budget line, with any historical breach already shaded red.

4. Templatize the route. Add a dashboard variable route of type Query (SELECT DISTINCT route FROM web_vitals) and replace the literal /checkout in the SQL with $route. One panel now serves every route, and a repeating row gives a per-route grid.

Threshold Calibration #

Pull the budget value for each threshold line from field data, not the lab number. Read the P75 of each metric from your RUM store over a trailing 28-day window, then set the line at the budget you are committing to — usually field-P75 rounded to the nearest target band. The windows below control how much smoothing each panel applies; tighter buckets surface regressions sooner but show more noise.

Set the threshold line from a Grafana dashboard variable rather than hardcoding it per panel, so a budget change is one edit. For the percentile method behind these values, see Percentile-Based Threshold Tuning.

CI Enforcement #

A trend board is only honest if the lab signal it shows is the same number the gate enforces. After Lighthouse CI asserts, push the median metrics to the same store the panels read, tagged with the commit so a step appears on the trend exactly when a change lands.

- name: Push LHCI metrics to TSDB
  if: always()
  run: |
    LCP=$(jq '.audits["largest-contentful-paint"].numericValue' .lighthouseci/lhr-*.json | sort -n | awk '{a[NR]=$1} END{print a[int(NR/2)+1]}')
    psql "$PERF_DB_URL" -c "INSERT INTO web_vitals(ts, metric, route, value, source, sha)
      VALUES (now(), 'LCP', '/checkout', ${LCP}, 'lab', '${{ github.sha }}');"
  env:
    PERF_DB_URL: ${{ secrets.PERF_DB_URL }}

Tagging rows with source = 'lab' keeps the CI series separate from RUM on the panel, and the sha column lets a Grafana annotation query mark each deploy on the timeline.

Troubleshooting & Edge Cases #

• Panel shows gaps → the producer stopped writing or a route had no traffic in a bucket; set Connect null values to Threshold and verify the beacon is still emitting.
• P75 looks too flat → you are reading an avg() not a percentile; switch to percentile_cont(0.75) or histogram_quantile.
• Threshold line missing → Show thresholds defaults to Off on new panels; set it to lines+regions explicitly.
• Lab and field series diverge wildly → that is expected; the lab is throttled and synthetic. Keep them on separate panels, not one overlay.
• Slow dashboard on Postgres → add a composite index on (metric, route, ts) and rely on Timescale continuous aggregates instead of querying raw rows.
• Route cardinality explodes → normalize dynamic segments (/product/123 → /product/:id) in the beacon before storage, or the variable dropdown becomes unusable.

Frequently Asked Questions #