Skip to content

feat(perf): store bundle size Measurements for every main commit - #284

Closed
harlan-zw wants to merge 3 commits into
mainfrom
feat/perf-measurements
Closed

harlan-zw wants to merge 3 commits into
mainfrom
feat/perf-measurements

Conversation

@harlan-zw

Copy link
Copy Markdown
Collaborator

❓ Type of change

  • ✨ New feature

📚 Description

Depends on harlan-zw/harlan-agent-kit#355 for the threshold semantics. Until that merges, the perf-review Skill still treats any count movement as a Regression.

I want the perf-review Routine watching skilld.dev's bundle, the same way it watches gscdump's engine. This ports the gscdump harness (run.mjs only changes its temp prefix, perf-comment.yml only its runner) and adapts the two measuring workflows to the desktop pool: both sides are exported with git archive into fresh directories and built through run-with-heartbeat. No skew-protection cache is restored, since that copies earlier deploys' chunks into .output. The cases also drop any chunk latest.json lists as retained, so a stale local cache cannot inflate a number either.

Six count Benchmarks, each with a thresholdPercent:

Benchmark Threshold Why
server/bytes 2% ~200 KB of 9.6 MiB. A new heavy server dependency, not a new route
server/gzip-bytes 2% the number Cloudflare's Worker size limit applies to
client/js-bytes 2% ~44 KB of 2.1 MiB
client/css-bytes 3% Tailwind output grows with every new page, so looser
client/chunk-count 5% 8 of 156 chunks. A lost code split or a split explosion
client/entry-bytes 1% ~2.7 KB. Every page pays for it, so tightest

These are a first guess. The one feature commit I measured (997bbdf) moved server bytes by +0.065% and client JS by +0.011%, well under all of them. The backfill series should say whether 2% is too loose.

"Gzip bytes of the worker entry" became gzip of the whole server bundle. .output/server/index.mjs is a 2.4 KB shim that imports every chunk, so its own size tells nothing.

Local run, 997bbdf against 39f41f2:

server/bytes        head 10027098  parent 10020570  delta +0.065%  noise 0%
server/gzip-bytes   head 3474971   parent 3473032   delta +0.056%  noise 0%
client/js-bytes     head 2190377   parent 2190143   delta +0.011%  noise 0%
client/css-bytes    head 329036    parent 329036    delta 0%       noise 0%
client/chunk-count  head 156       parent 156       delta 0%       noise 0%
client/entry-bytes  head 267596    parent 267596    delta 0%       noise 0%
{ "id": "server/bytes", "kind": "count", "unit": "bytes", "repeats": 1,
  "head": { "min": 10027098, "median": 10027098 },
  "parent": { "min": 10020570, "median": 10020570 },
  "control": { "min": 10027098, "median": 10027098 },
  "deltaPercent": 0.065, "controlPercent": 0, "verified": true }

The builds are not fully deterministic. Three builds of the same commit gave identical client numbers, but server bytes moved by up to 3,135 bytes (0.03%) and gzip by up to 529 bytes (0.015%). Two causes: comark-content emits its raw/*.json.mjs collections in a racy order, and a cold nuxt-og-image font cache builds a slightly different nitro.mjs than a warm one. Both sides in CI start cold, and both are far under 2%. The harness's control column cannot see this, since it re-reads the same build.

perf-review joins .github/routines.yml in report mode. It will only report coverage until ten Measurements exist. The controller accepts it on a maintained repository; only enabled gates Routine sync.

Open question: perf-comment.yml runs on the desktop pool too, to match test.yml. It only calls the API, so a hosted runner would also do.

Architecture: the three workflows, the harness, refs/notes/perf, and the perf-review Routine that reads it

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

The perf-review Routine reads Measurements from refs/notes/perf. Bundle byte counts carry a per-Benchmark thresholdPercent, so normal feature growth is not a Regression.
@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Sep 23, 2026
@harlan-github-agent

harlan-github-agent Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CLOSED

Harlan Agent Kit posted this automated review. It is not Harlan's personal review or approval. AI open source policy. Last updated: 2026-09-23 13:29 UTC.

GitHub closed this pull request. No further automated Review will run.

@harlan-github-agent harlan-github-agent Bot added harlan-agent-blocked The automated Review found a material defect in this head commit. harlan-agent-running An Agent holds a Task on this issue or pull request right now. and removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. harlan-agent-blocked The automated Review found a material defect in this head commit. labels Sep 23, 2026
@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-blocked The automated Review found a material defect in this head commit. and removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-blocked The automated Review found a material defect in this head commit. labels Sep 23, 2026
…ed-chunk guard reads deletedChunks, which nuxt-skew-protection records for JavaScript only. A warm local cache can still inflate css counts; CI's cold exports keep stored
@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-blocked The automated Review found a material defect in this head commit. and removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-blocked The automated Review found a material defect in this head commit. labels Sep 23, 2026
@harlan-zw harlan-zw closed this Sep 23, 2026
@harlan-github-agent harlan-github-agent Bot removed the harlan-agent-running An Agent holds a Task on this issue or pull request right now. label Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant