bench: publish C6, the per-request context scenario - #382
Merged
Conversation
C6 has been measured since it was written but never published, sitting on auto-calibration outside the pinned-shape contract C1-C4 use. Pins it to the same 200x1000 shape in all five framework files and gives it its own table against all four rivals: modern-di resolves by reference throughout the body and has no by-type C6 variant, so a split would leave that half mixed-basis. The published set is now derived from report.TABLES rather than hand-listed, so the contract cannot stay scoped to an old set when a table is added. Two fairness corrections to the timed bodies, found in review and moving the numbers in opposite directions: C4 and C6 both called open() on a child that is already open from construction as of 3.1 (81 ns, ~4% of C4's cell and ~6% of C6's), and C6 alone skipped the per-request teardown all four rivals pay inside their timed bodies (~110 ns). C6's cells move against modern-di as a result; C4's dishka ratio moves 1.30 to 1.26. All four tables regenerated from one run afterwards, with every derived figure in the prose recomputed and three structurally stale paragraphs rewritten.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Benchmark
Details
| Benchmark suite | Current: 2c9458d | Previous: c0683ad | Ratio |
|---|---|---|---|
benchmarks/test_guard_by_type.py::test_g16_resolve_by_type |
1778291.9685681886 iter/sec (stddev: 2.4359544045166027e-7) |
2447086.262495338 iter/sec (stddev: 1.6497644199087335e-7) |
1.38 |
benchmarks/test_guard_by_type.py::test_g17_resolve_by_type_large_registry |
2109177.3709060824 iter/sec (stddev: 7.38615589068588e-8) |
3045445.0422145366 iter/sec (stddev: 4.096759850201814e-8) |
1.44 |
benchmarks/test_guard_cold.py::test_g8_cold_first_resolve |
28012.60980142325 iter/sec (stddev: 0.000016702552287647053) |
29460.55380991645 iter/sec (stddev: 0.000026209450565662663) |
1.05 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[1] |
332.79669494135203 iter/sec (stddev: 0.000046289099874175604) |
488.1426242117961 iter/sec (stddev: 0.00011889861932312393) |
1.47 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[2] |
324.9876729338629 iter/sec (stddev: 0.0000505103118333245) |
453.50073680809555 iter/sec (stddev: 0.00009287659831508371) |
1.40 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[4] |
297.6558571602398 iter/sec (stddev: 0.00015683665166038222) |
352.1175992839381 iter/sec (stddev: 0.0003297905819275887) |
1.18 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[1] |
2638.6270906625746 iter/sec (stddev: 0.000025354289193365027) |
2925.575124181623 iter/sec (stddev: 0.00002258456630731583) |
1.11 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[2] |
1803.8552445255937 iter/sec (stddev: 0.0003173773707283687) |
1978.9995167941727 iter/sec (stddev: 0.0004185723603133118) |
1.10 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[4] |
1254.06532708415 iter/sec (stddev: 0.00003523083340511343) |
1385.428764513674 iter/sec (stddev: 0.00004537041166387969) |
1.10 |
benchmarks/test_guard_lifecycle.py::test_g6_build_child_container |
624520.6131749683 iter/sec (stddev: 4.875535527309357e-7) |
882618.5512920186 iter/sec (stddev: 3.073878355406602e-7) |
1.41 |
benchmarks/test_guard_lifecycle.py::test_g6b_build_child_container_auto_scope |
604852.9302913976 iter/sec (stddev: 4.5738003941852715e-7) |
842331.1972859108 iter/sec (stddev: 2.819198893580552e-7) |
1.39 |
benchmarks/test_guard_lifecycle.py::test_g7_request_lifecycle_batch |
2114.8877438288314 iter/sec (stddev: 0.000014065720023665978) |
2718.8300839534645 iter/sec (stddev: 0.00001733886731029035) |
1.29 |
benchmarks/test_guard_lifecycle.py::test_g7c_event_loop_floor_control |
61749.34273410356 iter/sec (stddev: 0.000001952329629030687) |
87601.7408770678 iter/sec (stddev: 0.000001156471455139014) |
1.42 |
benchmarks/test_guard_lifecycle.py::test_g13_teardown_at_scale |
43629.85536316017 iter/sec (stddev: 0.0000018413664143985542) |
55099.909108907246 iter/sec (stddev: 0.0000017456955557645556) |
1.26 |
benchmarks/test_guard_resolve.py::test_g1_transient_resolve |
1226672.5360113892 iter/sec (stddev: 3.5012112684731927e-7) |
1715417.4104117353 iter/sec (stddev: 2.2350540552844885e-7) |
1.40 |
benchmarks/test_guard_resolve.py::test_g2_cached_resolve |
2564254.4483729657 iter/sec (stddev: 6.587879740904859e-8) |
3823984.0700515443 iter/sec (stddev: 3.2561915292856526e-8) |
1.49 |
benchmarks/test_guard_resolve.py::test_g3_deep_chain |
511083.6408410895 iter/sec (stddev: 5.373772215720167e-7) |
757624.9474645628 iter/sec (stddev: 3.3630904191369485e-7) |
1.48 |
benchmarks/test_guard_resolve.py::test_g4_wide_resolve |
312467.6114354696 iter/sec (stddev: 6.66189937033329e-7) |
431015.58654454054 iter/sec (stddev: 5.269884925328139e-7) |
1.38 |
benchmarks/test_guard_resolve.py::test_g5_cross_scope |
1097187.0289447492 iter/sec (stddev: 3.934502230613184e-7) |
1497807.565427178 iter/sec (stddev: 3.409075507813664e-7) |
1.37 |
benchmarks/test_guard_resolve.py::test_g9_context_resolve |
574545.2932249756 iter/sec (stddev: 5.71815417592075e-7) |
805648.1465809564 iter/sec (stddev: 3.210365063961666e-7) |
1.40 |
benchmarks/test_guard_resolve.py::test_g12_override_active_resolve |
379137.93867381563 iter/sec (stddev: 5.32843377516199e-7) |
582393.7677668326 iter/sec (stddev: 2.3796593257971416e-7) |
1.54 |
benchmarks/test_guard_validate.py::test_g10_validate_deep_chain |
31725.34514233164 iter/sec (stddev: 0.000003012159311677285) |
32979.71312319029 iter/sec (stddev: 0.0000025788526522096015) |
1.04 |
benchmarks/test_guard_validate.py::test_g11_validate_wide |
18674.623500435315 iter/sec (stddev: 0.0000033202512732674443) |
20997.632223123066 iter/sec (stddev: 0.000004316101609119563) |
1.12 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
C6 (per-request context: supply a runtime value by type, resolve a handler needing it plus an app-scoped dep) has been measured by the comparative suite since it was written, but never published — it sat on auto-calibration, outside the pinned-shape contract #380 established for C1-C4. That left
just bench-report, which this page tells readers to run, printing a scenario the page never explained. Rationale: planning/changes/2026-07-27.04-publish-c6-context.md.Promotion. C6 pins to C1-C3's
benchmark.pedanticshape (200 × 1000) in all five framework files, and gets its own table against all four rivals rather than rows in the by-reference/by-type split — for C4's reason: modern-di resolves by reference throughout the body and has no by-type C6 variant, so a split would leave that half mixed-basis.Two fairness corrections to the timed bodies, found in review, pulling in opposite directions. Neither is a library change:
open()on a child they had just built. A container is open from construction as of 3.1, so that timed a redundant lock acquire with no counterpart in any rival body — 81 ns, ~4% of C4's cell and ~6% of C6's. C4 is already published, so this corrects a figure the page was leaning on: its dishka ratio moves 1.30 → 1.26.open()removed for fairness — so the page could not argue the 81 ns mattered while leaving it undisclosed. C6 now closes the child, moving its cells against modern-di (0.56 → 0.59 vs dependency-injector, 1.57 → 1.69 vs dishka).Contract.
_PUBLISHEDis no longer hand-listed; it is derived fromreport.TABLES, so a scenario is published exactly when a table publishes it. Two tests added: one pinning the derivation, one asserting the C6 table publishes undivided — a straydivisor=BATCHwould otherwise publish a number wrong by 100x with everything green. Both were verified to bite by mutation.Republished. All four tables regenerated from one run after the body corrections, with every derived figure in the prose recomputed against it. Three paragraphs were structurally wrong and were rewritten: "what moved" had been measuring against two publications ago and folding drift into a fix narrative; its claim that remaining cells moved less than their published spread was false (7 of 18 moved more); and the C1-vs-that-depends verdict is restated as near-parity, since
±is a full interquartile width, not a half-width, so the prior "spread covers 1.00" reading was never derivable.C5 stays unpublished and the page now says why where a reader reproducing will see it: its cells are not one axis (dependency-injector's is ~98% provider-graph deepcopy; that-depends wires at import with no per-container build).
just test-ci452 passed / 100.00% coverage;just lint-ci,just check-planningandmkdocs build --strictclean. Two rounds of independent review; no code defects were found in either, and both new tests were broken deliberately to confirm they bite.🤖 Generated with Claude Code