Skip to content

feat(template): a declared template rule, and the tmpl applet on the same renderer - #440

Merged
raphaelvigee merged 2 commits into
masterfrom
raphaelvigee/coreutils-template-driver
Oct 18, 2026
Merged

raphaelvigee merged 2 commits into
masterfrom
raphaelvigee/coreutils-template-driver

Conversation

@raphaelvigee

@raphaelvigee raphaelvigee commented Aug 29, 2026 •

Copy link
Copy Markdown
Member

Filling in a config file should not require a shell. template(name, src, out, vars) renders a declared template with declared variables, in-process — inputs and outputs fully declared, no sed pipeline, no quoting rules to get wrong.

One feature, two surfaces

The renderer lives in crates/template so the rule and the tmpl applet are the same code rather than two implementations of one syntax:

  • template() — the rule, and the one to reach for. Its inputs are in the cache key.
  • tmpl — the applet, for the mid-recipe case, and it covers envsubst.

They land together because splitting them would put a crate in trunk with one caller and review the same rendering semantics twice.

Two constraints the renderer carries deliberately

  • No non-deterministic builtins. No now(), no randomness — a template that can render differently from identical inputs is a target that can never be a cache hit.
  • TEMPLATE_FORMAT_VERSION in the def hash, so a minijinja upgrade that changes rendering invalidates rather than silently diverging.

vars is hashed from a BTreeMap because it arrives as a HashMap, and hashing it in iteration order would move the def hash on every run.


vars take deferred values

vars values are Deferred<String>, so vars = {"image": "${read://infra:registry}/app"} works as in an exec run: the host adds a hashed edge at parse and hands over the producer's bytes at run. The def hash takes the text as written, so plain values hash byte-for-byte as before. ${src://…} is refused at parse (this driver has no sandbox), and vars = None still parses — the usual macro-forwarding shape. Hermeticity board: HERMETIC. Covered by unit tests and two crates/e2e/tests/deferred.rs cases (renders the bytes; re-renders when the value moves, def hash unchanged).


The stack

Merge bottom-up, and gh stack sync after each one lands — master is squash-only, so the rebase will conflict and the resolution rule in CLAUDE.md applies.

PR What
5 #446 the toolbox on by default, and the host directories off a target's PATH — breaking
4 #440 the template rule and the tmpl applet
3 #453 grep, find, xargs, sed, tar, gzip, zstd
2 #438 the crate, the entry points, the shim directory
1 #451 the runner PATH seam ← base, targets master

#445 (default-on) was folded into #446: the two flips only mean something together. Only #451 builds automatically: since #449 a stacked PR is skipped unless it carries ci/force-ci. Every layer was checked locally on its own after rebasing onto master — fmt, cargo clippy --workspace --all-targets -D warnings, the touched crates' unit tests, and the e2e + engine suites — not just at the top of the stack.

🤖 Generated with Claude Code

https://claude.ai/code/session_0181d7hhbYWXT42Z1KQPM29Q

@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from d386399 to 168264d Compare August 29, 2026 21:35
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/core-utils-templating-engine branch from a2abb37 to cfb9516 Compare September 3, 2026 16:26
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from 168264d to 7ae3435 Compare September 3, 2026 16:26
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/core-utils-templating-engine branch from cfb9516 to 962ce7c Compare September 3, 2026 16:56
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from 7ae3435 to cceac42 Compare September 3, 2026 16:56
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/core-utils-templating-engine branch from 962ce7c to 11d00de Compare September 3, 2026 17:30
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from cceac42 to 31f29a2 Compare September 3, 2026 17:30
@raphaelvigee
raphaelvigee changed the base branch from raphaelvigee/core-utils-templating-engine to raphaelvigee/coreutils-rest-of-toolbox September 3, 2026 17:32
@raphaelvigee raphaelvigee changed the title feat(template): render a declared template, without a shell feat(template): a declared template rule, and the tmpl applet on the same renderer Sep 3, 2026
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from 31f29a2 to 8c87364 Compare October 16, 2026 17:05
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-rest-of-toolbox branch from 2f656af to 88dd8b5 Compare October 16, 2026 17:05
@raphaelvigee
raphaelvigee removed this pull request from stack #454 October 16, 2026 17:05
@raphaelvigee
raphaelvigee added this pull request to stack #493 October 16, 2026 17:05
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-rest-of-toolbox branch from 88dd8b5 to 12e3328 Compare October 16, 2026 17:34
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from 8c87364 to 67b48cc Compare October 16, 2026 17:35
@raphaelvigee raphaelvigee added the ci/force-ci Run full CI on this stacked PR, which the gate would otherwise skip label Oct 16, 2026
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-rest-of-toolbox branch from 12e3328 to 9bc44ab Compare October 18, 2026 16:23
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from 67b48cc to f674e01 Compare October 18, 2026 16:23
@raphaelvigee
raphaelvigee removed this pull request from stack #493 October 18, 2026 16:23
@raphaelvigee
raphaelvigee added this pull request to stack #497 October 18, 2026 16:23
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from f674e01 to def01c9 Compare October 18, 2026 17:06
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-rest-of-toolbox branch from 9bc44ab to 34439fc Compare October 18, 2026 17:06
@raphaelvigee
raphaelvigee removed this pull request from stack #497 October 18, 2026 18:54
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-rest-of-toolbox branch from 34439fc to 8c77c8e Compare October 18, 2026 18:54
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from def01c9 to caa04e5 Compare October 18, 2026 18:54
@raphaelvigee
raphaelvigee added this pull request to stack #500 October 18, 2026 18:54
@raphaelvigee raphaelvigee removed the ci/force-ci Run full CI on this stacked PR, which the gate would otherwise skip label Oct 18, 2026
@raphaelvigee
raphaelvigee removed this pull request from stack #500 October 18, 2026 21:38
raphaelvigee added a commit that referenced this pull request Oct 18, 2026
…ts past 16 MiB

On aarch64-apple-darwin nearly every function's `__unwind_info` entry points
at its FDE in `__eh_frame` (rustc emits epilogue CFI that compact unwind cannot
express), and that pointer is 24 bits. Debug test binaries now carry more than
16 MiB of `__eh_frame`; ld64 954.16 — nix's, the last open-source release —
writes the offset truncated without a word. libunwind trusts it and parses
whatever sits at the wrapped offset. Depending on the bytes there it falls back
to a full scan or crashes with `malformed DW_CFA_def_cfa_register DWARF unwind,
reg too big` and SIGSEGV — on any panic, or any anyhow error under CI's
RUST_BACKTRACE=1.

It surfaced as the `engine` lib tests crashing on #440 and the `heph` lib
tests on #446, on Test darwin/arm64 only, identically on rerun. Reproduced
with a synthetic binary: ld64 writes hint 0x02e950 for an FDE at 0x102e950
and the walk dies with the CI signature; ld-prime and lld both write "no hint"
and the walk completes.

Xcode's ld-prime links in about half lld's time (heph debug binary 0.59s vs
1.19s, engine lib tests 0.47s vs 0.95s; ld64 1.51s / 1.07s) but is not open
source, so nix cannot pin it. Use it when the machine has it, lld otherwise.

- devenv: `ld64.heph` (`-fuse-ld=heph`) execs `$HEPH_DARWIN_LD`, which
  `enterShell` resolves once per shell — ld-prime (ld-1000+) found with
  `DEVELOPER_DIR` unset, else nix's `ld64.lld` (`pkgs.lld`). Resolving per
  link cost ~40ms; the exec costs ~2ms.
- .cargo/config.toml: `-fuse-ld=heph` for aarch64-apple-darwin, repeating the
  build rustflags since a target entry replaces them.
- tests/darwin_linker.rs: the test binary reads its own LC_BUILD_VERSION and
  fails unless lld or ld >= 1000 linked it. Verified red with the config
  bypassed.
- ci-triage: the signature, as real-not-flake.

This also changes the linker of the shipped macOS binary and cdylibs, and
makes it the runner image's Xcode linker; bin_e2e covers them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0181d7hhbYWXT42Z1KQPM29Q
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-rest-of-toolbox branch from 8c77c8e to 4748115 Compare October 18, 2026 21:42
Base automatically changed from raphaelvigee/coreutils-rest-of-toolbox to master October 18, 2026 21:42
raphaelvigee and others added 2 commits October 18, 2026 23:42
…same renderer

Filling in a config file should not require a shell. `template(name, src, out,
vars)` renders a declared template with declared variables, in-process
(`crates/builtins/src/plugintemplate`) — inputs and outputs fully declared, no
`sed` pipeline, no quoting rules to get wrong.

The renderer itself lives in `crates/template` so the rule and the `tmpl`
applet are the same code rather than two implementations of one syntax. `tmpl`
covers the mid-recipe case (and `envsubst`); the rule is the one to reach for,
because a rule's inputs are in the cache key and a mid-recipe render's are not.

Two constraints the renderer carries deliberately:

- **No non-deterministic builtins.** No `now()`, no randomness — a template
  that can render differently from the same inputs is a target that can never
  be a cache hit.
- **`TEMPLATE_FORMAT_VERSION` in the def hash**, so a minijinja upgrade that
  changes rendering invalidates rather than silently diverging.

`vars` is hashed from a `BTreeMap` because it arrives as a `HashMap`, and
hashing it in iteration order would move the def hash on every run.

The rule and the applet land together because they are one feature: a shared
renderer used two ways. Splitting them would put a crate in trunk with one
caller and review the same rendering semantics twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0181d7hhbYWXT42Z1KQPM29Q
A config file's values usually live somewhere else — a registry host in a
platform repo, a version in a release target — and copying them into
`vars` makes a second source of truth. `vars` values are now
`Deferred<String>`, so `${read://pkg:name}` works there exactly as it does
in an exec `run`: the host turns the reference into a hashed edge at parse
and hands the driver the producer's bytes at run.

The def hash takes each value's text as written. A plain value hashes
byte-for-byte as before, so no existing template target invalidates; a
reference contributes itself, and its producer's content reaches `hashin`
through the edge — `heph inspect def` never builds anything.

`${src://…}` is refused at parse, naming the variable. It is completed only
by the managed-driver layer, and this driver renders in-process with no
sandbox; letting it through would fail at run with a message blaming heph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0181d7hhbYWXT42Z1KQPM29Q
@raphaelvigee
raphaelvigee force-pushed the raphaelvigee/coreutils-template-driver branch from caa04e5 to e776c81 Compare October 18, 2026 21:42
@raphaelvigee
raphaelvigee merged commit 9fa964c into master Oct 18, 2026
@raphaelvigee
raphaelvigee deleted the raphaelvigee/coreutils-template-driver branch October 18, 2026 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant