Configuring WebPageTest API for Automated Testing #

Automated gating needs the WebPageTest API, not the UI: a deterministic submit-poll-parse loop that returns the median metric and a clean exit code CI can act on. This guide drives that loop end to end against the instance you provisioned in WebPageTest Private Instance Setup, so a regression on a controlled connection profile blocks the merge. The work splits into four exact steps — construct a deterministic payload, submit it, poll jsonResult.php until complete, and compare the parsed median against a budget — and each step has a failure mode that silently hangs a pipeline if you skip it.

API Parameter Reference #

The runtest.php endpoint is strict about field types. These are the parameters that matter for a deterministic CI run; omitting location defaults to a shared agent and ruins reproducibility, and firstViewOnly roughly halves execution time while preserving Core Web Vitals.

Diagnostic Steps #

Validate the API key with a cheap call before submitting any test — distinguish 401 (bad key) from 403 (rate-limited or IP-blocked).

curl -s -o /dev/null -w "%{http_code}\n" \
  -H "X-API-Key: $WPT_API_KEY" \
  "$WPT_BASE_URL/getLocations.php?f=json"
# Expected: 200

Confirm the location you intend to submit to actually has idle agents, so the test does not silently queue.

curl -s "$WPT_BASE_URL/getLocations.php?f=json" \
  | jq '.data[] | {location: .id, pending: .PendingTests.Total}'
# Expected: your location present with a low pending count

Implementation #

This script submits one test, polls jsonResult.php every 5 seconds with a hard 180-second timeout (so a stuck agent cannot hang CI), then extracts the median first-view metrics and compares each against a threshold map plus a regression check against a versioned baseline.

#!/usr/bin/env bash
set -euo pipefail

# 1. Submit
ID=$(curl -s -X POST "$WPT_BASE_URL/runtest.php" \
  -H "X-API-Key: $WPT_API_KEY" -H "Content-Type: application/json" \
  -d '{"url":"https://staging.example.com/checkout?wpt_bypass=1","location":"us-east-1:Chrome.Cable","runs":3,"firstViewOnly":true,"connectivity":"Cable","f":"json"}' \
  | jq -r '.data.id')
echo "Submitted test $ID"

# 2. Poll (5s interval, 180s hard timeout)
for i in $(seq 1 36); do
  RESPONSE=$(curl -s "$WPT_BASE_URL/jsonResult.php?test=$ID&f=json")
  STATUS=$(echo "$RESPONSE" | jq -r '.statusCode')
  [ "$STATUS" = "200" ] && break
  [ "$STATUS" = "100" ] && { echo "in progress ($((i*5))s)"; sleep 5; continue; }
  echo "Unexpected status $STATUS"; exit 1
done
[ "$STATUS" = "200" ] || { echo "Timeout"; exit 1; }

# 3. Parse the median first view
LCP=$(echo "$RESPONSE" | jq -r '.data.median.firstView.LargestContentfulPaint')
CLS=$(echo "$RESPONSE" | jq -r '.data.median.firstView.CumulativeLayoutShift')
SI=$(echo "$RESPONSE"  | jq -r '.data.median.firstView.SpeedIndex')

# 4. Budget + regression gate
declare -A T=( ["LCP"]=2500 ["CLS"]=0.1 ["SpeedIndex"]=2500 )
(( $(echo "$LCP > ${T[LCP]}" | bc -l) )) && { echo "FAIL LCP $LCP > ${T[LCP]}"; exit 2; }
(( $(echo "$CLS > ${T[CLS]}" | bc -l) )) && { echo "FAIL CLS $CLS > ${T[CLS]}"; exit 2; }
(( $(echo "$SI  > ${T[SpeedIndex]}" | bc -l) )) && { echo "FAIL SpeedIndex $SI > ${T[SpeedIndex]}"; exit 2; }

BASELINE_LCP=$(jq -r '.LCP' baseline.json)
(( $(echo "$LCP > ($BASELINE_LCP * 1.1)" | bc -l) )) && { echo "REGRESSION: LCP +10% vs baseline"; exit 2; }

echo "All budgets passed (LCP ${LCP}ms, CLS ${CLS}, SI ${SI}ms)."
exit 0

Reserve exit 0 for pass, exit 1 for non-blocking warnings, and exit 2 for a hard fail that blocks the PR.

CI Gating Assertion #

Wire the script into a job so its exit code becomes a required status check. The thresholds enforced here — LCP ≤ 2500 ms, CLS ≤ 0.1, Speed Index ≤ 2500 ms at the P75 of the Cable profile — must match the budget your team agreed.

- name: WebPageTest budget gate
  run: ./scripts/wpt-gate.sh
  env:
    WPT_BASE_URL: ${{ secrets.WPT_SERVER }}
    WPT_API_KEY: ${{ secrets.WPT_API_KEY }}
# Exit 2 from the script fails this step and blocks the merge when the check is
# required in branch protection. Stagger across a matrix at max 5 concurrent
# submissions per key to avoid queue saturation.

Verification #

Confirm the loop works before trusting the gate. A passing run prints the parsed median line and exits 0; a budget breach prints the failing metric and exits 2.

./scripts/wpt-gate.sh; echo "exit=$?"
# Healthy pass:
#   Submitted test 240620_AB_1234
#   All budgets passed (LCP 1820ms, CLS 0.04, SI 1900ms).
#   exit=0
# Hard fail:
#   FAIL LCP 2740 > 2500
#   exit=2

Check three things: the submit step returns a non-empty test ID, the poll exits within the 180-second timeout rather than hanging, and the exit code matches the budget verdict. To cut redundant API calls on unchanged commits, hash the URL plus payload (sha256sum) and skip submission when a cached result for that hash already exists.

Frequently Asked Questions #