regression (2.5.4): ESP32 elf2image dies on bare `esptool` — version parser never resolves pioarduino URL, and the PATH fallback it relies on broke

[zackees]

Impact

FastLED master has been red on every ESP32 build job since 2026-07-30 — the commit that bumped the pin fbuild==2.5.2 → fbuild==2.5.4 (FastLED 5dd05daff, PR #3770). The ELF links fine; only the firmware.elf → firmware.bin conversion fails.

build error: build failed: esptool not found — cannot convert firmware.elf to firmware.bin.
Error: failed to spawn ["esptool", "--chip", "esp32", "elf2image", ...]: No such file or directory (os error 2)

Affected: esp32dev, esp32c3, esp32s3, esp32dev_i2s, esp32dev_idf33, esp32dev_idf44, esp32dev_idf5_component. esp8266 and all non-ESP boards pass (esp8266 uses objcopy).

Tracked downstream as FastLED#3774.

Cause 1 — esptool provisioning never resolves for platform-espressif32 53.03.10, so it always degrades to a PATH lookup

crates/fbuild-library/src/library/esptool.rs:257 extract_esptool_version() parses the version out of the filename of the tool-esptoolpy metadata URL, per its doc comment:

The URL looks like .../releases/download/0.0.1/esptoolpy-v5.3.0.zip, where 0.0.1 is the registry release tag and the real esptool version lives in the filename — so we parse the filename, not an earlier path segment.

But the platform FastLED resolves publishes the opposite shape — version in the path segment, generic filename:

https://github.com/pioarduino/esptool/releases/download/v4.8.5/esptool.zip

Filename is esptool.zip, which contains no version, so the function returns "unknown". The existing unit test extract_version_falls_back_to_unknown (esptool.rs:302) encodes exactly this case. The tasmota download URL then becomes .../releases/download/vunknown/esptool-linux-amd64.zip → 404 → resolve_esptool() logs a warn and returns None (crates/fbuild-build-esp/src/esp32/orchestrator/packages.rs:139-175) → esptool_bin is None → the linker falls back to a bare esptool name resolved from PATH (crates/fbuild-build-esp/src/esp32/esp32_linker.rs:427-461).

That fallback is why the failing argv is ["esptool", "--chip", ...] with no absolute path. The doc comment acknowledges the design: "Falls back to unknown (which then fails to resolve a release and triggers the PATH fallback)". So for this platform, provisioning has never worked — it has been silently riding the PATH fallback.

Cause 2 — 2.5.4 changed the daemon's environment, and the PATH fallback stopped working

crates/fbuild-cli/src/daemon_client.rs:1038 (added by 84518f8f "adopt running-process 4.6.0 daemon environments", git tag --contains → v2.5.4, matching the regression window exactly):

cmd.env_clear().envs(baseline);
if let Some(path) = launcher_path(std::env::vars_os()) {
    cmd.env("PATH", path);
}

To be precise: launcher_path (daemon_client.rs:1006) does forward the caller's PATH — but only at daemon spawn time. The build, and therefore the elf2image spawn, runs inside the daemon. Any daemon already running from an earlier step — or spawned before the project venv was on PATH — keeps its original environment, and the CLI simply connects to it. Everything else is stripped except FBUILD_*.

FastLED installs esptool>=5.0.2 (pyproject.toml:248, resolving 5.1.0) into the project venv at <project>/.venv/bin/esptool. If the daemon's PATH doesn't include that venv, the bare-name lookup fails — 3 minutes into the build, with a message suggesting pip install esptool even though esptool is installed.

This is not an esptool v4/v5 CLI issue: fbuild already emits the v5 esptool spelling and the venv has 5.1.0.

Requested fixes

1. Parse the path segment when the filename carries no version. .../download/v4.8.5/esptool.zip should resolve 4.8.5. Add that URL shape as a test case alongside the existing two. This alone fixes the reported failure, since provisioning would then succeed and the PATH fallback would never be reached.
2. Make provisioning failure loud. Degrading from a tracing::warn! to a PATH lookup that fails minutes later with a misleading hint hides the real fault. A failed provision should be an actionable error naming the URL it tried.
3. Add a FBUILD_ESPTOOL_PATH override, consistent with the existing FBUILD_WLINK_PATH / FBUILD_WCHISP_PATH / FBUILD_PROBE_RS_PATH / FBUILD_DFU_UTIL_PATH / FBUILD_LPC21ISP_PATH. esptool is currently the only one of these tools with no override, and FBUILD_* is the only prefix that survives the daemon env_clear.
4. Consider forwarding resolved tool paths (or PATH) per-request rather than only at spawn. Binding tool resolution to whichever environment happened to spawn the daemon makes builds depend on daemon lifetime — a warm daemon and a cold one can resolve different toolchains for identical inputs.

Repro

GitHub ubuntu-latest, fbuild build -e esp32dev against pioarduino platform-espressif32 53.03.10, with esptool present only in the project venv.

• Green: FastLED run 30434174442 (fbuild 2.5.2)
• Red: FastLED run 30509160747 (fbuild 2.5.4) — identical environment otherwise; both install the same 142 packages and the same tool-esptoolpy@4.8.5.

[zackees]

Scope split so nothing is lost when PR #1218 closes this:

• PR fix(esptool): parse version from release tag when filename is generic (#1217) #1218 — fault 1, the version parse. Clears the outage. (fixes this issue)
• daemon binds tool resolution to spawn-time PATH for up to 12h — bare-name esptool/python/objcopy spawns resolve nondeterministically #1219 — the daemon binds tool resolution to spawn-time PATH for up to 12h. This is fault 2, generalized: it is not esptool-specific. find_python() (script_runtime.rs:303), objcopy (ch32v_linker.rs:238) and the esp8266 esptool spawn (esp8266_linker.rs:263) all resolve the same way.
• esptool: provisioning failure degrades silently to PATH, and there is no FBUILD_ESPTOOL_PATH override #1220 — esptool provisioning fails silently into a PATH lookup, and is the only provisioned tool with no FBUILD_*_PATH override.

One correction to my original report above, having read the code more carefully: the daemon does not hardcode PATH. launcher_path (daemon_client.rs:1006) forwards the launching CLI's PATH correctly. The defect is lifetime, not stripping — the forward happens only at spawn, and IDLE_TIMEOUT is 12 hours (context.rs:108), so every later CLI that connects to a running daemon silently inherits the first launcher's environment. #1219 has the corrected mechanism and a repro.