Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 59 additions & 7 deletions docs/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,46 @@ Where a launch's seconds go, how to get that as JSON, and how the trend on
## One round trip per question

Every `devpod` call costs about 0.45s, far more than `dl` itself spends on
anything. That one figure sets the shape of the whole program: a command reads
the workspace list at most once, and everything a container needs on the way in,
naming it and then the tools probe, rides a single setup pass. So an interactive
`dl <ws>` and a one-shot `dl <ws> -- <cmd>` cost the same trips.
anything. That figure was measured on the Python build on 2026-08-07
(`22afd52`) and no host has re-measured it into this prose since, but it is
still the right order: `devpod status` on `dl 0.13.0`, nested inside this
repo's devcontainer, measures 0.43s to 0.67s
([#388](https://github.com/blooop/devlaunch/issues/388)), and the trend's own
Rust `warm / devpod-up` points, 47 of them on GitHub runners, run 0.23s to
0.49s with all but one under 0.45s. So read it as an order rather than as a
stopwatch reading, and not as a floor: the fastest of those points halves it.
That one figure sets the shape of the whole program: a command reads the
workspace list at most once, and everything a container needs on the way in,
naming it and then the tools probe, rides a single setup pass. So an
interactive `dl <ws>` and a one-shot
`dl <ws> -- <cmd>` cost the same trips.

## Measuring launch time

Set `DEVLAUNCH_TIMING=1` and a `dl` command ends with one summary on stderr,
naming each subprocess round trip and the total. Unset (or `0`) records nothing
and prints nothing.

**Every second quoted in this section came off one host on 2026-08-14, over
three sessions that day (`d3b8ffb`, `5510b1e`, `461f740`), and that host was
running the Python build `0.1.0` replaced.** The `dl-next` in these captures is
that Python install, not the compiled Rust working tree the name means now. The
formats are current: the stage names are frozen in
`rust/devlaunch-core/src/timing.rs` and pinned by its tests. The seconds are
not. They describe an implementation that no longer ships, and no host has
measured the Rust binary into this prose, which is deliberate rather than a gap
to fill by hand. The Rust build's own per-stage decomposition was re-measured
nested inside this repo's devcontainer
([#388](https://github.com/blooop/devlaunch/issues/388)), which is
docker-in-docker rather than a host, so what it carries across is the per-stage
arithmetic and not anyone's wall clock.

Current per-commit numbers come from [the trend on main](#the-trend-on-main),
which has published a Rust point on nearly every push since 2026-08-22, two
lost runs aside. Read it forward from `03b7de2`, the first of them, and not
across it: every point before that measures the Python build, and nothing on
the chart says which is which.

Captured from a real warm launch (the launch's own output elided):

```bash
Expand Down Expand Up @@ -132,8 +161,9 @@ build the image pays more, by an amount this recipe does not measure, and the
gap is large: an earlier 3-run median on this same host, reported as its first
real launch, was 33.204s.

Every number in the two paragraphs above was copied into this prose by hand.
`--record` is how that stops: it writes the same invocation as one JSON object
Every number in the two paragraphs above was copied into this prose by hand,
which is how they came to outlive the build that produced them. `--record` is
how that stops: it writes the same invocation as one JSON object
a trend job can upload without anyone reading it.

```bash
Expand Down Expand Up @@ -166,7 +196,29 @@ things about it are load-bearing:
Every push to `main` runs `.github/workflows/bench.yml`, which benches both
shapes on the runner and publishes one point per stage to
<https://blooop.github.io/devlaunch/dev/bench/>. It can also be dispatched by
hand. Reading it needs nothing but the chart; what follows is for changing it.
hand.

**The port is in that series and nothing on the chart marks it**, which is the
one thing worth knowing before reading it. The first 35 points end at `1566c86`
and measure the Python build, because until `1654daf` these steps ran the
console script an editable Python install left in the environment. Publishing
then stopped for two days, the retirement having put a bare `dl` on `PATH` with
no `devpod` beside it, until
[#373](https://github.com/blooop/devlaunch/pull/373) fixed it. The Rust series
starts at `03b7de2` on 2026-08-22 and has a point for nearly every push since.
Two are missing, `5573f77` and `7be9986`: the workflow serialises on one
concurrency group, GitHub keeps only one run queued behind the one it is
running, and five merges landed inside four minutes that day, so the runs in
the middle were cancelled before they benched anything. A missing point is
therefore a cancelled run rather than a skipped commit, and the comparison
across one spans two commits' worth of change. A comparison within either run
of points is otherwise sound, and one that straddles the port is not: it reads
a whole change of implementation as a regression.
[#292](https://github.com/blooop/devlaunch/issues/292) is open on that, and on
its record-keeping instruction rather than on a red workflow: the commit of the
first Rust point has to be written into the map's decisions so nobody later
reads that step as something `dl` did. Past that, reading the chart needs
nothing but the chart; what follows is for changing it.

`scripts/bench_points.py` is the step between the two formats, bench records
in and one flat array of trend cases out:
Expand Down
Loading