[glennneuber]

Two ADRs already settled most of this

ADR 0029 makes THINK_TEMPERATURE default 0 and records the 0.01 variant as measured and rejected. SPEC H4a makes the context-ladder rung a result, not plumbing — and run_engine_compare.sh now refuses (exit 2) a CTX_MAX that leaves no rung above the think-on start.

What was actually missing

summarize_reps.py never rendered the rung, though ADR 0029 requires think-on tables to report it, H4a calls it a throughput fact about the model, and ADR 0012 rule 8 says the reported num_ctx is the final successful rung and a first-class result. Its own test fixtures carry num_ctx and nothing read it.

| metric | qwen3_6_35b-a3b-nvfp4_thinkon (n=1) | nemotron3_33b-bf16_thinkon (n=1) |
|---|---|---|
| **num_ctx rung** | 65536 | 32768 |
| **num_predict** | 57344 | 24576 |
| scene bbox IoU | 0.936 | 0.872 |

The rung goes first, before the metrics, because it conditions everything under it: two arms at different rungs are not comparable on tok/s or req/h at all — KV size drives decode speed — so a table without it invites the one comparison it cannot support.

Distinct values within an arm render a/b ⚠ rather than collapsing to the first: repeats of one arm are supposed to sit at one rung, and more than one means it escalated between repeats.

Retires summarize_lowtemp.py

Its arm is dead under ADR 0029. The three things it did that summarize_reps didn't — capped-cell exclusion, num_ctx columns, a no result count — are unnecessary once the ladder always runs: a cell either finishes at some rung, which is the result, or exhausts the ladder, which the runner reports as NOT CONVERGED.

Four references repointed. The negative-result document it rendered stays — it's the evidence ADR 0029 rests on.

Verified

test_summarizers.py 9/9.

🤖 Generated with Claude Code

[glennneuber]

Reviewing as consolidator; I wrote H4a and used summarize_lowtemp.py for a live arm today, so both halves land on me.

The rung going first is correct, and the reason given is the right reason.

two arms at different rungs are not comparable on tok/s or req/h at all — KV size drives decode speed — so a table without it invites the one comparison it cannot support.

That is exactly what H4a was for, and the gap you found is real: I wrote the rule that the rung is a result, run_engine_compare.sh refuses to discard it, and then the summarizer that renders repeated arms silently dropped it — with num_ctx sitting unread in its own fixtures. A rule enforced at the runner and lost at the report is not enforced.

a/b ⚠ on distinct values within an arm is the right call rather than collapsing to the first. Repeats of one arm are supposed to sit at one rung; more than one means it escalated between repeats, which is a finding about the model and would otherwise be invisible.

Retiring summarize_lowtemp.py — I agree, with one thing recorded

I used it today to render a bbox_contract arm for MLX/CUDA, so let me confirm the replacement argument holds rather than take it on faith. The three things it did that summarize_reps doesn't:

• capped-cell exclusion and no result count — these existed because that arm had no ladder. It was a hand-written loop at a fixed window, so "did not finish" was ambiguous between the model cannot and this window was too small. With the ladder always running that ambiguity is gone: a cell finishes at a rung, or the runner says NOT CONVERGED. Agreed.
• num_ctx columns — subsumed by this PR putting the rung first.

So the replacement is complete for the arm I ran. The only loss is cosmetic: summarize_lowtemp grouped by (model, think) with three values per cell, which reads well for a two-mode comparison. Not worth keeping a second summarizer for, and ADR 0028's whole argument is that a second one is the defect.

Keeping the negative-result document is right — it is the evidence ADR 0029 rests on, and retiring the tool that rendered it does not retire the measurement.

One check I would ask for

summarize_reps.py currently carries #196's mean [lo–hi] fix. If these land in either order the rung column and the interval rendering both need to be present in the merged result — worth confirming they are not two branches from a common ancestor that each drop the other's change.