docs(performance): re-measure the comparative table for 3.1.0 - #379
Merged
Conversation
Five runs, median-of-medians per scenario, matching the methodology the page already states. Rival pins unchanged, so this refreshes modern-di's column and the ratios derived from it. No verdict crossed 1.0; the dishka/wireup range and the C4-vs-dishka figure were tightened to match. The measurement machine also changed since the previous publication (M2/3.14.4 to M4/3.14.6), so the page now tells readers to compare ratios across revisions rather than nanoseconds - C3 dropped in absolute terms on a scenario a same-machine 3.0.0-vs-3.1.0 guard comparison showed did not move.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Benchmark
Details
| Benchmark suite | Current: 432d3e7 | Previous: 552f7c9 | Ratio |
|---|---|---|---|
benchmarks/test_guard_cold.py::test_g8_cold_first_resolve |
19410.05218150231 iter/sec (stddev: 0.0001528287466197493) |
20501.106615692996 iter/sec (stddev: 0.0002385664721561687) |
1.06 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[1] |
317.0368465697806 iter/sec (stddev: 0.00012674853548938375) |
319.13347597625176 iter/sec (stddev: 0.00005591225341324346) |
1.01 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[2] |
301.90532605791867 iter/sec (stddev: 0.00006873769146441019) |
323.66904621779076 iter/sec (stddev: 0.00003696035094731032) |
1.07 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[4] |
284.61648392309587 iter/sec (stddev: 0.00011159300338704306) |
295.92740259502506 iter/sec (stddev: 0.0001869359183473183) |
1.04 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[1] |
1379.7177263098847 iter/sec (stddev: 0.001183510809353734) |
1192.8435944970065 iter/sec (stddev: 0.0021649039909886585) |
0.86 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[2] |
1235.021364994624 iter/sec (stddev: 0.0005545565267556137) |
1259.5191174899107 iter/sec (stddev: 0.0006782309481742352) |
1.02 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[4] |
799.8293430787867 iter/sec (stddev: 0.0014271871767045315) |
901.6730904860332 iter/sec (stddev: 0.0007326898557464034) |
1.13 |
benchmarks/test_guard_lifecycle.py::test_g6_build_child_container |
479077.40436114086 iter/sec (stddev: 0.000009713386407877722) |
430241.96604200284 iter/sec (stddev: 0.00006837067718367991) |
0.90 |
benchmarks/test_guard_lifecycle.py::test_g6b_build_child_container_auto_scope |
465203.94237959787 iter/sec (stddev: 0.000009640160385988872) |
449558.3357643676 iter/sec (stddev: 0.000010442471120194737) |
0.97 |
benchmarks/test_guard_lifecycle.py::test_g7_request_lifecycle |
65773.67836882062 iter/sec (stddev: 0.000013895523963619784) |
72758.54678403895 iter/sec (stddev: 0.000013767794611012372) |
1.11 |
benchmarks/test_guard_lifecycle.py::test_g13_teardown_at_scale |
41316.00168152066 iter/sec (stddev: 0.000020311608456818122) |
40991.403991742496 iter/sec (stddev: 0.000020924421405964968) |
0.99 |
benchmarks/test_guard_resolve.py::test_g1_transient_resolve |
1215106.5986841575 iter/sec (stddev: 3.783021624108872e-7) |
1244527.708280102 iter/sec (stddev: 6.010381247138101e-7) |
1.02 |
benchmarks/test_guard_resolve.py::test_g2_cached_resolve |
2768419.334422337 iter/sec (stddev: 5.912380193291223e-8) |
2657837.3355129194 iter/sec (stddev: 6.189431422374186e-8) |
0.96 |
benchmarks/test_guard_resolve.py::test_g3_deep_chain |
487986.22515885206 iter/sec (stddev: 5.228600962928496e-7) |
488386.7102571147 iter/sec (stddev: 8.87661768896005e-7) |
1.00 |
benchmarks/test_guard_resolve.py::test_g4_wide_resolve |
300726.90985739726 iter/sec (stddev: 7.497780766643348e-7) |
306260.45783673355 iter/sec (stddev: 6.975007409867451e-7) |
1.02 |
benchmarks/test_guard_resolve.py::test_g5_cross_scope |
1053747.4115499174 iter/sec (stddev: 0.0000013166442824700342) |
1117644.9044456459 iter/sec (stddev: 4.203654542509246e-7) |
1.06 |
benchmarks/test_guard_resolve.py::test_g9_context_resolve |
572841.0425881288 iter/sec (stddev: 5.317916710091668e-7) |
426591.3917675563 iter/sec (stddev: 8.729055761643911e-7) |
0.74 |
benchmarks/test_guard_resolve.py::test_g12_override_active_resolve |
370346.2948262415 iter/sec (stddev: 4.5237886272696495e-7) |
361859.77176196815 iter/sec (stddev: 8.33311541987878e-7) |
0.98 |
benchmarks/test_guard_validate.py::test_g10_validate_deep_chain |
26425.659251680732 iter/sec (stddev: 0.000022307176112721742) |
26825.449962516916 iter/sec (stddev: 0.00006566129517211644) |
1.02 |
benchmarks/test_guard_validate.py::test_g11_validate_wide |
17205.610033690384 iter/sec (stddev: 0.000009380176784368208) |
17807.445879545092 iter/sec (stddev: 0.000011790803021058317) |
1.03 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The published table was measured 2026-07-17, i.e. against 3.0.0. 3.1.0 changed the top of the resolve path (
resolve_provideropens with an inlineclosedtest rather than an unconditional_raise_if_closed()call;build_child_containerlost its check entirely), so it needed re-measuring. Bundle: planning/changes/2026-07-27.01-refresh-performance-numbers.md.Five runs of
just bench-compare, median-of-medians per scenario — the methodology the page already states. Rival pins unchanged (dishka 1.10.1, dependency-injector 4.49.1, that-depends 4.0.2, wireup 2.12.0), so this refreshes modern-di's column and the ratios derived from it, not a rival bump.No verdict changed — nothing crossed 1.0 either way. Ratios moved modestly in modern-di's favour on C1/C2 (vs dependency-injector on C2: 4.58 → 3.93; vs dishka on C1: 1.30 → 1.28) and were flat on C3/C4. Two prose figures were tightened to match: the dishka/wireup construction-heavy range (1.3–1.9x → 1.3–1.8x) and C4-vs-dishka (1.02 → 1.00).
Two things I added rather than left implicit:
Guard-tier evidence for that claim, run against the
3.0.0tag in a scratch worktree in both orderings to rule out thermal drift:g1_transient_resolve−10.9%/−10.9%,g2_cached_resolve−6.6%/−5.3%,g6_build_child_container−8.0%/−5.4%,g6b−13.3%/−7.0%,g14_concurrent_cached_hit−5.5% to −7.4%. Nothing regressed in both orderings, andg3/g4/g9moved in one ordering only — consistent with their running through compiled resolvers whose guard was already inline on both versions.Deliberately out of scope, noted in the bundle: bumping rival pins, and publishing the C5/C6 scenarios the comparative suite already runs but this page has never shown.
mkdocs build --strict,just lint-ciandjust check-planningall clean. Docs-only; no code touched.🤖 Generated with Claude Code