diff --git a/docs/performance.md b/docs/performance.md index 3988d3a1..6dacd342 100644 --- a/docs/performance.md +++ b/docs/performance.md @@ -6,10 +6,19 @@ Where a launch's seconds go, how to get that as JSON, and how the trend on ## One round trip per question Every `devpod` call costs about 0.45s, far more than `dl` itself spends on -anything. That one figure sets the shape of the whole program: a command reads -the workspace list at most once, and everything a container needs on the way in, -naming it and then the tools probe, rides a single setup pass. So an interactive -`dl ` and a one-shot `dl -- ` cost the same trips. +anything. That figure was measured on the Python build on 2026-08-07 +(`22afd52`) and no host has re-measured it into this prose since, but it is +still the right order: `devpod status` on `dl 0.13.0`, nested inside this +repo's devcontainer, measures 0.43s to 0.67s +([#388](https://github.com/blooop/devlaunch/issues/388)), and the trend's own +Rust `warm / devpod-up` points, 47 of them on GitHub runners, run 0.23s to +0.49s with all but one under 0.45s. So read it as an order rather than as a +stopwatch reading, and not as a floor: the fastest of those points halves it. +That one figure sets the shape of the whole program: a command reads the +workspace list at most once, and everything a container needs on the way in, +naming it and then the tools probe, rides a single setup pass. So an +interactive `dl ` and a one-shot +`dl -- ` cost the same trips. ## Measuring launch time @@ -17,6 +26,26 @@ Set `DEVLAUNCH_TIMING=1` and a `dl` command ends with one summary on stderr, naming each subprocess round trip and the total. Unset (or `0`) records nothing and prints nothing. +**Every second quoted in this section came off one host on 2026-08-14, over +three sessions that day (`d3b8ffb`, `5510b1e`, `461f740`), and that host was +running the Python build `0.1.0` replaced.** The `dl-next` in these captures is +that Python install, not the compiled Rust working tree the name means now. The +formats are current: the stage names are frozen in +`rust/devlaunch-core/src/timing.rs` and pinned by its tests. The seconds are +not. They describe an implementation that no longer ships, and no host has +measured the Rust binary into this prose, which is deliberate rather than a gap +to fill by hand. The Rust build's own per-stage decomposition was re-measured +nested inside this repo's devcontainer +([#388](https://github.com/blooop/devlaunch/issues/388)), which is +docker-in-docker rather than a host, so what it carries across is the per-stage +arithmetic and not anyone's wall clock. + +Current per-commit numbers come from [the trend on main](#the-trend-on-main), +which has published a Rust point on nearly every push since 2026-08-22, two +lost runs aside. Read it forward from `03b7de2`, the first of them, and not +across it: every point before that measures the Python build, and nothing on +the chart says which is which. + Captured from a real warm launch (the launch's own output elided): ```bash @@ -132,8 +161,9 @@ build the image pays more, by an amount this recipe does not measure, and the gap is large: an earlier 3-run median on this same host, reported as its first real launch, was 33.204s. -Every number in the two paragraphs above was copied into this prose by hand. -`--record` is how that stops: it writes the same invocation as one JSON object +Every number in the two paragraphs above was copied into this prose by hand, +which is how they came to outlive the build that produced them. `--record` is +how that stops: it writes the same invocation as one JSON object a trend job can upload without anyone reading it. ```bash @@ -166,7 +196,29 @@ things about it are load-bearing: Every push to `main` runs `.github/workflows/bench.yml`, which benches both shapes on the runner and publishes one point per stage to . It can also be dispatched by -hand. Reading it needs nothing but the chart; what follows is for changing it. +hand. + +**The port is in that series and nothing on the chart marks it**, which is the +one thing worth knowing before reading it. The first 35 points end at `1566c86` +and measure the Python build, because until `1654daf` these steps ran the +console script an editable Python install left in the environment. Publishing +then stopped for two days, the retirement having put a bare `dl` on `PATH` with +no `devpod` beside it, until +[#373](https://github.com/blooop/devlaunch/pull/373) fixed it. The Rust series +starts at `03b7de2` on 2026-08-22 and has a point for nearly every push since. +Two are missing, `5573f77` and `7be9986`: the workflow serialises on one +concurrency group, GitHub keeps only one run queued behind the one it is +running, and five merges landed inside four minutes that day, so the runs in +the middle were cancelled before they benched anything. A missing point is +therefore a cancelled run rather than a skipped commit, and the comparison +across one spans two commits' worth of change. A comparison within either run +of points is otherwise sound, and one that straddles the port is not: it reads +a whole change of implementation as a regression. +[#292](https://github.com/blooop/devlaunch/issues/292) is open on that, and on +its record-keeping instruction rather than on a red workflow: the commit of the +first Rust point has to be written into the map's decisions so nobody later +reads that step as something `dl` did. Past that, reading the chart needs +nothing but the chart; what follows is for changing it. `scripts/bench_points.py` is the step between the two formats, bench records in and one flat array of trend cases out: