Skip to content

Make the .ashpkg read ceiling a ceiling, on every path that reads one - #491

Merged
cursor[bot] merged 2 commits into
masterfrom
claude/fix-round3-cli
Sep 5, 2026
Merged

cursor[bot] merged 2 commits into
masterfrom
claude/fix-round3-cli

Conversation

@IanFrelinger

@IanFrelinger IanFrelinger commented Sep 2, 2026 •

Copy link
Copy Markdown
Owner

[coordinated-integration]

This PR changes application/ on a master base, which the layer-boundary gate blocks by default. The rationale, for reviewers:

The defect being fixed is a kernel-layer invariant — "a .ashpkg read is bounded and cannot block" — that happens to be implemented entirely in the CLI and daemon layer. The five read paths (import, show, publish, pull, auto-pull) and the mesh serve endpoint all live under application/, so there is no way to make the invariant true from src/ alone. Splitting it would land a guard in one PR and its callers in another, leaving the tree in a state where the CHANGELOG's claim is false — which is precisely the class of defect five review rounds were spent removing here.

The src/ side is untouched. The only cross-layer coupling is that application/src/Ashlar.CLI/Packaging/SafePackageRead.cs now bounds reads that src/Ashlar.Manifest/Packaging/ExtensionPackaging.cs used to backstop after the whole file was already in memory.

The 16 MiB .ashpkg ceiling was applied to two of four read paths — and on those two it was not a ceiling.

FileInfo.Length is metadata. It reports 0 for a FIFO, a socket and a character device, and for a symlink it reports the length of the target path string — 7, 9, 22 bytes. So a symlink to any honest 400 MB package walked straight past the guard, no device files required. Dropped into a synced mesh store, ln -s /dev/zero x.ashpkg OOMed the unbounded read and aborted a whole pull pass, so one planted file denied every legitimate package behind it; mkfifo x.ashpkg hung the pull forever. Each is one command, and neither needs a large file to be synced at all.

SafePackageRead — five gates, none of which trust metadata

gate
G0 metadata only: refuse a directory, refuse anything with a LinkTarget. Length is never read
G1 an open that cannot block: O_RDONLY|O_NONBLOCK|O_CLOEXEC on Unix; \?\ plus a device-namespace refusal on Windows
G2 refuse anything not seekable — fifo, socket, pipe; FILE_TYPE_DISK on Windows
G3 the declared length off the opened handle, advisory only
G4 the cap: never allocate past maxBytes + 1, refuse the moment the total crosses it

G4 is not belt-and-braces. /proc/self/mem is a regular file with st_size 0 that yields unbounded bytes, so a length check can never be the bound.

An empirical probe settled the design rather than guesswork: .NET 10 exposes no st_mode type bits anywhere — File.GetAttributes, UnixFileMode and even FileSystemEntry (the readdir path, where d_type is available to the runtime) all collapse FIFO, socket and chardev to Normal + length 0. And the obvious remediation, "open it and then check", blocks forever on a FIFO — turning the hang inside File.ReadAllTextAsync into a hang inside the guard.

All six read paths

import, show, publish, pull, the auto-pull daemon, and the mesh serve endpoint. That last one was the fifth round's find — it still gated on FileInfo.Length, so:

  • /mesh/v1/index advertised a symlink at 23 bytes and /mesh/v1/pkg/ then served 41,943,040
  • ln -s /etc/passwd x.ashpkg returned /etc/passwd to any peer that reached the port
  • 105 concurrent GETs of one planted FIFO stopped the node answering /hello at all

Tracked as #488, fixed here.

Refusal text is escaped and bounded

ExtensionPackaging interpolates the sender's formatVersion — read before anything is verified — into a message printed by all four verbs. A CR in it overwrites the refusal row with a counterfeit ✓ ADMITTED … sealed by <fingerprint>, i.e. the one part of a row the surrounding comment says a sender cannot choose.

This one is worth dwelling on: an earlier round's skeptics killed "result.Message is an injection sink" 3/3, correctly, by testing the malformed-JSON route — where System.Text.Json hex-escapes control bytes. The real route sets a valid formatVersion, parses cleanly, bypasses that entirely, and reaches the same sink. Every fact they checked was true and the conclusion still didn't follow.

Also here

  • pull --from file://… no longer mangled into a nonsense path
  • progress moved off stdout, so --format-json output parses
  • a command that cannot honour --format-json says so rather than ignoring it
  • a failing row is refused and the pass continues — including a SubmitAsync throw, which sat outside the per-row try
  • two CI filters that matched zero tests (green by vacuity) now run the suite — behind step conditions a pull_request actually selects, which was the second half of that bug

Verification

cert-gate 404/404 · e2e-loop.sh 143/143 · Ashlar.Tests.CLI 207/0 · tier-C filter 61/0

Every attack above was reproduced live against the built binary and then re-run after the fix. New tests were checked to fail against the unfixed code, not merely pass against the fixed one.

How this was reviewed

Five adversarial rounds. Each found a defect the implementer and the verifier had already called clean:

  1. the ceiling did not hold at all
  2. the "Unix-only" e2e block ran on Windows, where MSYS turns mkfifo into a .lnk and ln -s into a 32-byte !<symlink> file
  3. SubmitAsync sat outside the per-row try
  4. the escaping covered two of four sinks while the CHANGELOG claimed four
  5. the serve path was a sixth reader nobody had counted

Independent skeptics then adjudicated the findings — killing 8 of 23 in one round and every HIGH in the next. That filter is why the survivors were worth acting on; without it this PR would have chased several confident, reproduced, wrong claims.

Filed rather than fixed

#483 (pre-existing unescaped sites on the success paths), #484, #485, #486, #487, #488, #489, #490.

🤖 Generated with Claude Code

IanFrelinger and others added 2 commits September 5, 2026 01:00
…--format-json)

Checkpoint before addressing the adversarial review's blockers: the byte
ceiling is bypassable by non-regular files, and the new e2e fixture uses
`truncate`, which macOS does not ship.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 16 MiB guard was applied to two of four read paths, and on those two it
was not a guard. FileInfo.Length is metadata: it reports 0 for a FIFO, a socket
and a character device, and for a SYMLINK it reports the length of the target
PATH STRING — 7, 9, 22 bytes. So `ln -s /dev/zero x.ashpkg` dropped into a
synced mesh store walked past the check and OOMed the unbounded read, aborting
a whole pull pass so one planted file denied every legitimate package behind
it; `mkfifo x.ashpkg` hung the pull forever. Both are one command, and neither
needs a large file to be synced at all.

SafePackageRead replaces the check with five gates that never trust metadata:
refuse a directory or a symlink without reading Length; open non-blocking
(O_NONBLOCK on Unix, \?\ plus a device-namespace refusal on Windows) so a FIFO
cannot block the guard itself; refuse anything not seekable; treat the declared
length as advisory only; and cap the read at maxBytes+1 whatever the metadata
claimed. The cap is not belt-and-braces: /proc/self/mem is a regular file with
st_size 0 that yields unbounded bytes.

All six read paths route through it — import, show, publish, pull, the
auto-pull daemon, and the mesh SERVE endpoint. The serve path was the fifth
round's find: it still gated on FileInfo.Length, so the index advertised
symlinks at 23 bytes and then served 41,943,040; `ln -s /etc/passwd
x.ashpkg` returned /etc/passwd to any peer; and 105 concurrent GETs of one
planted FIFO stopped the node answering /hello at all (issue #488).

Refusal text is escaped and bounded. ExtensionPackaging quotes the sender's
formatVersion — read before anything is verified — into a message printed on
all four verbs, so a CR could overwrite the refusal row with a counterfeit
ADMITTED line carrying a sealer fingerprint, the one part of a row the code
says a sender cannot choose.

Also here: `pull --from file://` is no longer mangled into a nonsense path;
progress moved off stdout so --format-json parses; a command that cannot
honour --format-json now says so instead of ignoring it; a failing row is
refused and the pass continues; and two CI filters that matched zero tests
(green by vacuity) now run the suite — behind step conditions that a
pull_request actually selects, which was the second half of that bug.

cert-gate 402/402. e2e-loop 143/143. Ashlar.Tests.CLI 207/0.

Five adversarial rounds shaped this. Each found a defect the implementer and
the verifier had both called clean: the ceiling did not hold; the "Unix-only"
e2e block ran on Windows, where MSYS turns mkfifo into a .lnk; SubmitAsync sat
outside the per-row try; the escaping covered two of four sinks while the
CHANGELOG said four; and the serve path was a sixth reader nobody had counted.
Independent skeptics then killed 8 of 23 findings in one round and every HIGH
in the next, which is the only reason the rest are worth acting on.

Filed rather than fixed: #483 (pre-existing unescaped sites on the success
paths), #484, #485, #486, #487, #488, #489, #490.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cursor
cursor Bot force-pushed the claude/fix-round3-cli branch from b65a8e1 to 050ac52 Compare September 5, 2026 01:02
@cursor

cursor Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Rebased onto current master (5b74e656, after #500 and #501). The only conflict was CHANGELOG.md: kept the product-split / 0.1.2 Unreleased notes and added the .ashpkg Security / Fixed / Changed entries above [0.1.2].

Local check after rebase: SafePackageRead / UntrustedText / PkgCommand tests 41/41. GitHub now reports MERGEABLE (blocked only on CI).

@cursor
cursor Bot merged commit 220af97 into master Sep 5, 2026
19 checks passed
@cursor
cursor Bot deleted the claude/fix-round3-cli branch September 5, 2026 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant