[kevinjosethomas]

• combine changes and percentages in four-column tables, with one overall summary and no legend. Colour the entire change value from muted to vivid as the percentage grows, capped at 100%.
• replace old tables with a running notice on each push, then publish the latest scan in the same comment.
• verified github rendering, 29 benchmark tests, ruff, and npm run check; follows eng-6043.

---

Note

Low Risk
Changes affect benchmark comment formatting and internal report helpers only; no product runtime, auth, or data paths.

Overview
PR benchmark bot comments are narrowed to four columns (metric, main, PR, change) with delta and percentage in one cell, plus a single Overall count of regressed / improved / no clear change / incomplete / unavailable metrics instead of a legend and separate “Change %” and “Result” columns.

Significant moves get GitHub MathJax coloring: green ↓ for improvements, red ↑ for regressions, with hue intensity scaling by relative change; noise-band deltas stay plain ≈ …. comparison() now returns a Comparison dataclass (outcome + formatted cells) instead of a five-tuple, and metric Definition drops the old better/worse labels.

While status is running or pending-trust, render() omits tables and shows a short status line and run link so a new push can patch the same sticky comment with a running notice instead of leaving the previous scan visible until completion.

^{Reviewed by Cursor Bugbot for commit ef100e2. Bugbot is set up for automated code reviews on this repo. Configure here.}

Note

Make benchmark comments compact and show only the latest scan

• Replaces the old wide metric tables with compact four-column tables and an aggregate outcome summary counting regressions, improvements, neutral, incomplete, and unavailable results.
• Adds a structured Comparison result type with named outcomes (improved, regressed, no clear change, incomplete, unavailable) and signed delta-plus-percentage formatting with MathJax color intensity based on relative change.
• Running and pending-trust comments now show only a status marker and run link, hiding stale metric tables from previous scans.
• A new commit on the same PR replaces the prior bot comment table with a running notice, then fills that same comment on completion; older runs can no longer publish after replacement.
• Risk: report.py comparison formatter returns Comparison objects instead of tuples; any out-of-tree consumers indexing the old tuple shape will break.

^{Macroscope summarized ef100e2.}

[github-actions]

Prime Agent performance — completed

PR ef100e20 compared with main f9c7e06b.
↓ improved · ↑ regressed · ≈ no clear change · — unavailable

Metric	Main	This PR	Change	Change %	Result
Cold startup	2,994.1 ms	3,007.2 ms	≈ +13.1 ms	+0.44%	no clear change
Warm startup	1,779.4 ms	1,771.4 ms	≈ -8.0 ms	-0.45%	no clear change
Installation	29.21 s	26.63 s	≈ -2.58 s	-8.84%	no clear change
Compressed release artifacts	11.10 MB	11.10 MB	≈ +0.00036 MB	+0.00%	no clear change
Installed footprint	597.35 MB	597.35 MB	≈ +0.0015 MB	+0.00%	no clear change
Idle memory, summed RSS	1,015.94 MB	1,103.58 MB	≈ +87.63 MB	+8.63%	no clear change

Python runtime

Metric	Main	This PR	Change	Change %	Result
Python kernel startup	129.3 ms	126.7 ms	≈ -2.6 ms	-1.99%	no clear change
Python cell round trip	0.550 ms	0.499 ms	≈ -0.051 ms	-9.32%	no clear change
Empty bash command	11.9 ms	11.5 ms	≈ -0.5 ms	-3.90%	no clear change
Bash git status	17.4 ms	16.6 ms	≈ -0.8 ms	-4.61%	no clear change
Bash 32 KiB output	12.0 ms	11.7 ms	≈ -0.4 ms	-3.03%	no clear change
35 cells / 9 shell calls	173.1 ms	173.1 ms	≈ +0.0061 ms	+0.00%	no clear change
Python interrupt to done	1.557 ms	1.712 ms	≈ +0.154 ms	+9.91%	no clear change
Python state snapshot	25.3 ms	26.2 ms	≈ +1.0 ms	+3.82%	no clear change
Python state restore	370.8 ms	386.6 ms	≈ +15.8 ms	+4.25%	no clear change
Python idle RSS	32.80 MB	36.04 MB	≈ +3.23 MB	+9.86%	no clear change
Python RSS after pandas workload	94.90 MB	98.22 MB	≈ +3.33 MB	+3.51%	no clear change

Sandbox cost: ~$0.0882 — no inference calls.
Run, logs, and downloadable raw results

Methodology and samples

Main resolved at 2026-09-10T00:45:22.754612+00:00. Harness f9c7e06b.
Linux x64, 4 vCPU, 8 GB RAM, 20 GB disk; region us.
Image: node:24-bookworm@sha256:be23f54a88d34e8824c741b19b91064094f92c1c97b194144bfc8b50d67258e2.
Stock tools, skills, daemon, and Python bootstrap enabled; fresh homes and a fixed Git fixture.
Onboarding is dismissed; the editor starts without a selected model or submitted prompt.
Medians shown. Arrows require a 20% timing/memory change plus absolute floors and IQR.
These practical noise floors are not a statistical significance test.
Cold means stopped Prime processes; OS filesystem caches are not flushed.
No model requests or credentials. Installation excludes build/setup time.
Installer tarballs use loopback; npm/Python downloads use the network with fresh caches.
Artifact size counts release tarballs; footprint after first use includes registry packages.
MB is decimal. Summed RSS can double-count shared pages; PSS is recorded when available.
Provisioning, setup, and build durations are recorded separately in the raw results.
Kernel probes use the installed JSONL runtime, outside the TUI/TypeScript host.
Per trial: 50 Python cells, 5 calls per shell case, and one 35-cell mix (9 git status calls).
Cell/shell values are batch means; other runtime timings are single operations.
State fixture: a 10,000-row × 8-column integer DataFrame and a 10,000-integer list.
Restore runs in a fresh kernel, including pandas imports; kernel startup is excluded.
Kernel RSS covers the isolated Python process; loaded RSS follows the pandas workload.
Costs estimate full sandbox lifetimes at configured rates, including setup and build.
Budget target: $1; not a billing cap. Checks are informational.

Metric	Main successful/attempted	PR successful/attempted	Main spread	PR spread
Cold startup	10/10	10/10	IQR 73.7 ms	IQR 334.3 ms
Warm startup	10/10	10/10	IQR 84.5 ms	IQR 113.8 ms
Installation	3/3	3/3	range 0.75 s	range 0.55 s
Compressed release artifacts	1/1	1/1	—	—
Installed footprint	1/1	1/1	—	—
Idle memory, summed RSS	10/10	10/10	IQR 94.32 MB	IQR 80.50 MB
Python kernel startup	10/10	10/10	IQR 13.5 ms	IQR 9.9 ms
Python cell round trip	10/10	10/10	IQR 0.076 ms	IQR 0.038 ms
Empty bash command	10/10	10/10	IQR 1.3 ms	IQR 0.8 ms
Bash git status	10/10	10/10	IQR 1.9 ms	IQR 0.7 ms
Bash 32 KiB output	10/10	10/10	IQR 0.9 ms	IQR 1.1 ms
35 cells / 9 shell calls	10/10	10/10	IQR 20.0 ms	IQR 15.0 ms
Python interrupt to done	10/10	10/10	IQR 0.068 ms	IQR 0.806 ms
Python state snapshot	10/10	10/10	IQR 1.3 ms	IQR 3.6 ms
Python state restore	10/10	10/10	IQR 15.2 ms	IQR 36.6 ms
Python idle RSS	10/10	10/10	IQR 2.56 MB	IQR 1.87 MB
Python RSS after pandas workload	10/10	10/10	IQR 2.51 MB	IQR 1.77 MB