Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion STANDUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ python3 scripts/host_gate.py # from a hub checkout, against the fleet

**No `--repo` here, and that is the one place in these procedures where it is omitted deliberately.** The flag points the gate at a repo's own `host-tools.json` so its floors are layered over the fleet ones, and at this step there is no repo to point it at: the target does not exist yet, since this section runs before the `git init` in section 0B, and the file itself arrives with the baseline in section 2. So this run checks the fleet floors, which is all that is knowable now.

**Re-run it with `--repo` once section 2 has carried the file**, because a bare run reads no declaration but the one at its own working directory, so any floor the target adds goes unapplied. The gate warns when its working directory sits inside a repo whose overlay it did not read, and no warning can name a target that does not exist yet, so this re-run is the only thing that counts the target's floors:
**Re-run it with `--repo` once section 2 has carried the file**, because a bare run reads no declaration but the one at its own working directory, so any floor the target adds goes unapplied. No warning covers the gap here, because the run above is not inside the target and the target is not an ancestor of the hub checkout it runs from. So this re-run is the only thing that counts the target's floors:

```shell
python3 scripts/host_gate.py --repo <path-to-target-checkout> # after section 2, so the repo's own floors count
Expand Down
2 changes: 1 addition & 1 deletion docs/fleet-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ Each gap's handoff below states who detects it, what closes it, and the test tha
### G2: Host-Tools Repo Overlay Is Silently Skippable (Closed)

- **Gap** - [`scripts/host_gate.py`][host-gate] run without `--repo` read only the declaration at its own working directory, so a run started in a subdirectory of a repo carrying a `host-tools.json` overlay skipped that overlay without a word.
- **Resolution** - A bare run whose working directory sits inside such a repo prints a warning naming the overlay's directory and the `--repo` re-run that counts it, asserted by the `TestBareRunOverlayWarning` cases in `scripts/tests/test_host_gate.py`. An explicit `--repo` and `--no-local` each stay silent, since both are a choice the caller made. [`STANDUP.md`][standup] section 0 states the residual case the warning cannot cover, a target repo that does not exist yet, instead of the workaround sentence.
- **Resolution** - A bare run whose working directory sits inside such a repo prints a warning naming the overlay's directory and the `--repo` re-run for it, asserted by the `TestBareRunOverlayWarning` cases in `scripts/tests/test_host_gate.py`. An overlay the gate cannot parse is named the same way, the failed parse being what leaves open whether it declares anything. An explicit `--repo` and `--no-local` each stay silent, since both are a choice the caller made, and so does an overlay declaring no tool, since every repository carries the file whether or not it adds anything. [`STANDUP.md`][standup] section 0 states the residual case the warning cannot cover, a run that is not inside the target at all, instead of the workaround sentence.

### G3: A Failed Tool Floor Names No Install Remedy (Closed)

Expand Down
7 changes: 5 additions & 2 deletions docs/host-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,13 +52,15 @@ Presence is the weaker half of this contract. Both host defects this fleet has a

Ripgrep also comes from the distribution, where every supported current release meets its floor. It provides `rg` search and `rg --files` discovery for coding agents.

Neither `node`, `dotnet`, nor `pwsh` is in the table above, deliberately: they serve the repositories that need them rather than the fleet contract, and a repository needing one declares it in a `host-tools.json` of its own, which [`scripts/host_gate.py`][host-gate] merges over this one. The merge tightens only, so a repository may raise a floor or add one and may not lower or remove one.
Neither `node`, `dotnet`, nor `pwsh` is in the table above, deliberately: they serve the repositories that need them rather than the fleet contract, and a repository needing one declares it in the `host-tools.json` at its own root, which [`scripts/host_gate.py`][host-gate] merges over this one. The merge tightens only, so a repository may raise a floor or add one and may not lower or remove one.

**A host being stood up needs no Python.** The tooling under [`host-setup/`][host-setup-dir] is shell and PowerShell, deliberately, because requiring an interpreter to upgrade a package or install a tool would make the first step of standing a host up depend on the thing that step exists to provide. The Python floor above is a development requirement, meaning [`scripts/`][scripts-dir] and [`spec/`][spec-dir], and a host that only runs services never has to meet it. `bootstrap.sh` needs `curl` and `tar`, both of which a base install carries or can install without a network tool of its own. `bootstrap.ps1` needs only `tar.exe`, which has shipped with Windows since 1803, and installs its one further dependency, PowerShell 7, itself through `winget`. The one exception is the skills step at the end of a stand-up, which drives the Python installer in [`scripts/`][scripts-dir], and it runs last for exactly that reason: `install-tools` has provided the interpreter by then, and run alone on a host without one it stops and names the tools step as its prerequisite.

**Standing a host up.** [`host-setup/`][host-setup-dir] carries the tooling that makes a host satisfy this contract, and its README is the usage. A host with nothing runs [`host-setup/bootstrap.sh`][bootstrap], which fetches this repository and runs that tooling from the fetched tree. A native Windows host with nothing runs [`host-setup/bootstrap.ps1`][bootstrap-ps1] the same way, which finds or installs PowerShell 7 before it fetches anything, since every script under [`host-setup/windows/`][host-setup-windows] requires it. Neither is called by [`scripts/host_gate.py`][host-gate] and neither calls it: the gate measures a host against the floors above, and the tooling is a remedy a person chooses when the gate reports a gap.

A repository that needs more than the fleet does adds its own `host-tools.json` at its root, which the gate layers over the hub's. It may add a tool nobody else uses, raise a floor, or turn an optional tool required. It may **not** lower a floor or turn a required tool optional, since those edits retire a fleet check from inside the repository it protects, and the gate reports a rejected relaxation rather than dropping it.
**Every repository carries a `host-tools.json` at its root, whether or not it declares anything in it.** [`spec/files.json`][files] lists it as a baseline file of every repository, on the same footing as [`OPERATIONS.md`][operations], so a maintainer who needs to declare a floor finds the file where it already is rather than having to know one may be created. A repository with nothing to add carries an empty `tools` list, and that stub is the conformant state rather than a leftover to remove. [`scripts/host_gate.py`][host-gate] treats an absent one as nothing to report, because the gate measures a host against floors and the audit in [`AUDIT.md`][audit] is what reports a baseline file a repository does not carry. What the gate does say, on a bare run started in a subdirectory, is that an ancestor's overlay went unapplied, and it stays silent about a stub, which has no floor to miss. An overlay it cannot parse is named like a declaring one, since the failed parse is what leaves the question open.

A repository that needs more than the fleet does declares it in its own `host-tools.json`, which the gate layers over the hub's. It may add a tool nobody else uses, raise a floor, or turn an optional tool required. It may **not** lower a floor or turn a required tool optional, since those edits retire a fleet check from inside the repository it protects, and the gate reports a rejected relaxation rather than dropping it.

A repository-only tool can declare constrained package metadata under `install.linux` or `install.windows`. Linux accepts only an `apt` package name. Windows accepts only a `winget` package ID. The installer reads those values as package identifiers and never evaluates `remedy` text from the repository.

Expand Down Expand Up @@ -367,6 +369,7 @@ A host that fails any row is not ready for the procedure that row names, and the
[bootstrap]: ../host-setup/bootstrap.sh
[bootstrap-ps1]: ../host-setup/bootstrap.ps1
[devcontainer]: ./devcontainer.md
[files]: ../spec/files.json
[governance-git-and-commit-rules]: ../GOVERNANCE.md#git-and-commit-rules
[host-gate]: ../scripts/host_gate.py
[host-setup-dir]: ../host-setup/
Expand Down
2 changes: 1 addition & 1 deletion scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ The host contract in [`docs/host-setup.md`][host-setup] as a check, reading the

The three states a tool can be in are kept apart because their remedies differ: **absent** means install it, **unreadable** means the declared pattern is wrong and the fix is in this repo rather than on the host, and **read** means the floor applies. A probe that runs and exits non-zero is not an answer, which is what separates a tool that is missing from one this file cannot parse.

A repository adds its own `host-tools.json` at its root and the gate layers it over the hub's, so a repo needing `ffmpeg`, or needing a tool the fleet calls optional, declares that where it is true. Layering is **tighten-only**: a local entry may add a tool, raise a floor, or turn an optional tool required, and may not lower a floor or turn a required tool optional, because those retire a fleet check from inside the repository it protects. A rejected relaxation is reported rather than dropped.
Every repository carries a `host-tools.json` at its root, an empty `tools` list where it adds nothing, and the gate layers it over the hub's, so a repo needing `ffmpeg`, or needing a tool the fleet calls optional, declares that where it is true. A bare run started in a subdirectory names an ancestor's overlay it did not apply, unless that overlay declares nothing, since every repository carries the file whether or not it adds anything. One it cannot parse is named too, the failed parse being what leaves the question open. Layering is **tighten-only**: a local entry may add a tool, raise a floor, or turn an optional tool required, and may not lower a floor or turn a required tool optional, because those retire a fleet check from inside the repository it protects. A rejected relaxation is reported rather than dropped.

A missing required tool prints its platform source and runnable remedy. An `install` record can derive that remedy through the host installer. The record names only an allowed package manager and package identifier, so repository-authored command text is never executed.

Expand Down
31 changes: 24 additions & 7 deletions scripts/host_gate.py
Original file line number Diff line number Diff line change
Expand Up @@ -413,15 +413,31 @@ def package_source(tool: dict) -> str | None:


def overlay_above(start: Path) -> Path | None:
"""The nearest ancestor of `start` carrying a host-tools.json, or None where none does.
"""The nearest ancestor of `start` carrying a host-tools.json, or None.

A bare run reads only the declaration at the working directory itself, so an overlay at the
root of the repo the run is inside goes unread without a word when the run starts in a
subdirectory. Naming that directory lets the run say what it skipped and which re-run counts it.
A bare run layers only the declaration at the working directory itself, so an overlay at the
root of the repo the run is inside goes unapplied without a word when the run starts in a
subdirectory. Naming that directory lets the run say what it skipped and which re-run applies it.

An overlay declaring no tool is passed over in silence, because every repository carries the
file whether or not it adds anything, so warning on presence alone would tell the operator to
re-run a command that applies a declaration adding nothing. An overlay this cannot parse is
named rather than passed over, since whether it declares anything is exactly what the failed
parse leaves unknown, and the re-run is what turns that into the diagnostic read_declaration
already writes.

The walk stops at the nearest carrier either way, empty or not.
"""
for parent in start.resolve().parents:
if (parent / "host-tools.json").is_file():
overlay = parent / "host-tools.json"
if not overlay.is_file():
continue
tools = read_declaration(overlay, "repository host tool declaration")
# Written out rather than folded into the truthiness below, which a diagnostic already satisfies by being a non-empty string.
# It is what holds if read_declaration ever returns an empty diagnostic, which the truthiness alone would read as a declaration adding nothing.
if isinstance(tools, str):
return parent
return parent if tools else None
return None


Expand Down Expand Up @@ -525,7 +541,8 @@ def main(argv: list[str] | None = None) -> int:
NOTES.clear()
rejected: list[str] = []
local_path = Path(a.repo or ".") / "host-tools.json"
# A repository layering onto the hub is the normal case, and carrying no local file is the common one, so its absence is silent.
# Every repository carries this file, an empty tools list where it adds nothing, so an absence is a conformance miss rather than the ordinary case.
# It is still silent here, because this gate measures a host against floors and spec/audit.py is what reports a baseline file a repository does not carry.
if not a.no_local and local_path.is_file():
local = read_declaration(local_path, "repository host tool declaration")
if isinstance(local, str):
Expand Down Expand Up @@ -558,7 +575,7 @@ def main(argv: list[str] | None = None) -> int:
if skipped is not None:
# Outside --quiet, because a silently skipped overlay is the omission this line exists to name.
print(
f" warning: {skipped} carries a host-tools.json overlay this bare run did not read - re-run with --repo {quote_argument(str(skipped))} so its floors count"
f" warning: {skipped} carries a host-tools.json overlay this bare run did not apply - re-run with --repo {quote_argument(str(skipped))}"
)
return 1 if issues else 0

Expand Down
86 changes: 82 additions & 4 deletions scripts/tests/test_host_gate.py
Original file line number Diff line number Diff line change
Expand Up @@ -526,6 +526,9 @@ class TestBareRunOverlayWarning(unittest.TestCase):
The default --repo reads the working directory alone, so a run started in a subdirectory reads
nothing and used to say nothing, which is the silent skip the warning closes. An explicit
--repo and --no-local each stay silent, since both are a choice the caller made.

An overlay declaring nothing stays silent too. Every repository carries the file whether or not
it adds anything, so warning on presence alone asks for a re-run that would change no floor.
"""

def spec_with_one_passing_tool(self, d):
Expand All @@ -548,9 +551,17 @@ def run_from(self, cwd, argv):
finally:
os.chdir(old)

def repo_with_overlay(self, d):
def repo_with_overlay(self, d, payload=None):
"""A repo root carrying an overlay, and a subdirectory to run from.

The default overlay declares a tool, since an empty one is deliberately not warned about.
That would leave the cases asserting silence passing for the wrong reason, and the cases
asserting a warning failing outright.
"""
root = Path(d)
(root / "host-tools.json").write_text('{"tools": []}', encoding="utf-8")
if payload is None:
payload = json.dumps({"tools": [tool("extra")]})
(root / "host-tools.json").write_text(payload, encoding="utf-8")
sub = root / "scripts"
sub.mkdir()
return root, sub
Expand Down Expand Up @@ -620,18 +631,85 @@ def test_a_spaced_overlay_path_is_quoted_in_the_re_run(self):
root, sub = self.repo_with_overlay(d)
out = self.run_from(sub, ["--spec", self.spec_with_one_passing_tool(d), "--quiet"])
self.assertIn(f"--repo {host_gate.quote_argument(str(root.resolve()))}", out)
self.assertNotIn(f"--repo {root.resolve()} ", out)
self.assertNotIn(f"--repo {root.resolve()}\n", out)

def test_overlay_above_returns_the_nearest_carrier(self):
import tempfile

with tempfile.TemporaryDirectory() as d:
root = Path(d)
(root / "host-tools.json").write_text('{"tools": []}', encoding="utf-8")
(root / "host-tools.json").write_text(
json.dumps({"tools": [tool("ok")]}), encoding="utf-8"
)
deep = root / "a" / "b"
deep.mkdir(parents=True)
self.assertEqual(host_gate.overlay_above(deep), root.resolve())

def test_an_overlay_declaring_nothing_is_not_named(self):
"""The stub every repository carries would otherwise ask for a re-run changing no floor."""
import tempfile

with tempfile.TemporaryDirectory() as d:
_, sub = self.repo_with_overlay(d, '{"tools": []}')
self.assertIsNone(host_gate.overlay_above(sub))
out = self.run_from(sub, ["--spec", self.spec_with_one_passing_tool(d)])
self.assertNotIn("warning:", out)

def test_an_unreadable_overlay_is_named_like_a_declaring_one(self):
"""The failed parse leaves open whether it declares anything, so the run names it."""
import tempfile

for payload in ("{not json", '{"tools": {"gh": {}}}'):
with tempfile.TemporaryDirectory() as d:
root, sub = self.repo_with_overlay(d, payload)
self.assertEqual(host_gate.overlay_above(sub), root.resolve(), payload)
out = self.run_from(sub, ["--spec", self.spec_with_one_passing_tool(d)])
self.assertIn("warning:", out)

def test_the_nearer_of_two_declaring_carriers_is_the_one_returned(self):
"""One carrier proves a carrier is found, so the nearest claim needs two of them."""
import tempfile

with tempfile.TemporaryDirectory() as d:
outer = Path(d)
(outer / "host-tools.json").write_text(
json.dumps({"tools": [tool("outer")]}), encoding="utf-8"
)
inner = outer / "inner"
inner.mkdir()
(inner / "host-tools.json").write_text(
json.dumps({"tools": [tool("inner")]}), encoding="utf-8"
)
deep = inner / "scripts"
deep.mkdir()
self.assertEqual(host_gate.overlay_above(deep), inner.resolve())

def test_an_empty_diagnostic_is_named_rather_than_read_as_an_empty_declaration(self):
"""The one input separating the isinstance branch from the truthiness check under it."""
import tempfile
from unittest.mock import patch

with tempfile.TemporaryDirectory() as d:
_, sub = self.repo_with_overlay(d, '{"tools": []}')
with patch.object(host_gate, "read_declaration", return_value=""):
self.assertEqual(host_gate.overlay_above(sub), Path(d).resolve())

def test_an_empty_overlay_nearer_than_a_declaring_one_stops_the_walk(self):
"""The nearest carrier decides, so an empty one is silence rather than a walk past it."""
import tempfile

with tempfile.TemporaryDirectory() as d:
outer = Path(d)
(outer / "host-tools.json").write_text(
json.dumps({"tools": [tool("ok")]}), encoding="utf-8"
)
inner = outer / "inner"
inner.mkdir()
(inner / "host-tools.json").write_text('{"tools": []}', encoding="utf-8")
deep = inner / "scripts"
deep.mkdir()
self.assertIsNone(host_gate.overlay_above(deep))


class TestMalformedLocalFile(unittest.TestCase):
"""A repository file is validated by nothing upstream, so the gate reports rather than crashes."""
Expand Down