Skip to content

test(auditregress): integration gate for the five audit fixes — fork-internal platform validation - #7

Open
pt-act wants to merge 5 commits into
mainfrom
lane/security-regression-integration
Open

pt-act wants to merge 5 commits into
mainfrom
lane/security-regression-integration

Conversation

@pt-act

@pt-act pt-act commented Nov 6, 2026

Copy link
Copy Markdown
Owner

SRI lane — fork-internal validation instrument (NOT the upstream PR)

Lane lane/security-regression-integration at 55f33b55b (3 commits, test-only, zero src/):

  • 1dbdc44fc — test/auditregresscheck.sh: the integration gate for the five 2026-09-14 audit fixes, both transports; registered in test/regression.sh (762nd absorb gate, byte-ordered slot); test/fixtures/audit-reef/ (the real-zip docx + the one-line 128 KB ReBoS subject); changelog.d/ fragment.
  • df9f0d470 — arm B's ASan leg checks BOTH channels (exit status AND no sanitizer banner on stderr); the SKIP names the path it tried.
  • 55f33b55b — the A1/cli row drops a stray double colon in its observed text.

Purpose of this PR: run the gate on both CI platforms (macOS + Linux legs) before any upstream exposure —
the spec's Phase-4 both-platforms clause, discharged by measurement. Locally proven: PASS 8/8 rows repelled
on macOS (disk + ASan asan=clean), 14 FAILs against a six-mutation binary, registration arbitrated 761→762
(gatecount_build.py --check, manifestcheck.sh both sides). This PR stays OPEN on the fork; it will not be
merged by automation and the upstream PR (if any) opens separately per the one-lane-in-review interlock.

…h transports

The five 2026-09-14 audit fixes each have a unit gate that passes. None of
them can say whether the five COMPOSE in one binary: whether the regex screen
still refuses on the CLI while the MCP grep verb stays literal-only, whether
the edit pin still refuses on the tiers it pins, whether the atomic publish
still ignores a planted temp name, whether hostile git paths still attribute
to their author, and whether each transport still refuses past ITS OWN bound
and then keeps serving.

test/auditregresscheck.sh replays the five attacks end-to-end against the
real binary over a purpose-built adversarial fixture. Eight rows, one per
(attack, transport) pair, each required to carry the exact shape the repaired
ACs name - the structural-screen lead-in, not the abandonment lead-in, and
not the exit status, which reads identically for both.

FOUR THINGS THE GATE DELIBERATELY DOES NOT ASSERT, each measured in Phase 0:

1. No regex REFUSAL over MCP. No MCP verb takes a regex at all: grep is
   literal-only (mcpverbs.h:1324). The HTTP leg asserts transport safety
   instead - a structured empty plus a POSITIVE CONTROL, because without one
   an empty answer is indistinguishable from a dead scan.
2. No arm for the bare-stdio edit escape. `ripwire --mcp` with no startup
   root runs the assumedRoot tier, which the edit pin's own gate
   (mcp.h:1944, `!policy.defaultRoot.empty()`) never enters, so an
   off-workspace edit succeeds there. That is SRI-F1, filed and reproduced by
   both seats - a proposed spec, not this lane's arm. A permanently red row
   is an issue tracker, not a gate.
3. No conflating the two caps. HTTP refuses past 8 MiB, stdio past 32 MiB, so
   a 9 MiB STDIO line is inside the shipped bound - served, correctly, and
   not an attack row.
4. No "pargates green" from inside. test/pargates.py runs every top-level
   test/*.sh, this file included; asserting it here would recurse without
   bound. The legs a gate cannot own (pargates, ASan) are SKIPped with the
   reason named rather than pretended.

A3 is a REGRESSION GUARD. The shipped temp name is 24 hex chars of
/dev/urandom created O_EXCL|O_NOFOLLOW with 8 retried draws, so no name can
be pre-planted; tempfilesymlinkcheck.sh owns the live half. A3 keeps the six
historical shapes dead and proves the edit still publishes.

The report is emitted and parsed as real CSV: one field legitimately contains
", " (the AC-1 lead-in), so an awk -F', ' column counter calls that row
malformed when it is exactly right. Every observed field reports only what was
measured, and row() enforces at write time that a `repelled` verdict's
expected shape is a real substring of its observed text.

Discrimination proved against a six-mutation binary (logs kept): M1 A1/cli,
M2 A2/stdio + A2/http, M4 A4/cli, M5 A5/http, M6 A5/stdio all go red; M3
(O_EXCL dropped) is caught by tempfilesymlinkcheck, the gate that owns it.
The run also found a real defect in this gate - the A4 fixture committed with
author and committer equal, so a %ae->%ce swap was invisible - now two
distinct people, with the committer email asserted absent from every row.

Test-only: zero src/ files. Gate registered in test/regression.sh at the
byte-ordered slot (after attrvocabcheck, before barrelshortkeycheck); the
loop names 762 today (761 before), which gatecount --check arbitrates.
…he exit status

The Phase-3 leg asserted only that the ASan binary exited 0 over the fixture. That is not a clean bill: a
sanitizer report can be EMITTED while the program still exits 0, and a suppressions file can hide a class of
report. A planted emitter proves the gap is real - a program that prints `ERROR: LeakSanitizer: detected memory
leaks` and exits 0 passed the old leg and is now caught by name:

  FAIL  B ASan/LSan exited 0 but EMITTED a report on the adversarial fixture: ==12345==ERROR: LeakSanitizer: ...
  suite: ... asan=REPORTED (exit 0) ...

A clean run must now satisfy both: exit 0 AND no sanitizer banner on stderr. The skip message also names the
path it tried, so a malformed RIPWIRE_ASAN_BIN cannot masquerade as "not requested":

  SKIP  B ASan/LSan: no RIPWIRE_ASAN_BIN given (nothing executable at '/nonexistent/ripwire') ...

Phase-3 evidence on the real thing: asan/ripwire built from this lane (built_from=1dbdc44fc), rc=0 with zero
bytes on stderr, arm B reads `PASS ... no report on stderr and exit 0`, trailer `asan=clean`. The SKIP the
Phase-1 run disclosed has turned into a real assertion, as the verdict expected.

Test-only: zero src/ files.
…observed text

The lead-in constant already ends in ':', so appending '; rc per pattern = ...' rendered
`refused, nothing was scanned:; rc per pattern = 111` in the report a consumer parses. Cosmetic only - the
verdict, the discriminator and the write-time substring enforcement are unchanged, and both are re-proved:

  shipped: A1,cli,"refused, nothing was scanned:","refused, nothing was scanned: rc per pattern = 111",0s,repelled
  mutant:  A1,cli,"refused, nothing was scanned:","refused, nothing was scanned: rc per pattern = 000",0s,unproven

Test-only: zero src/ files.
test/fixtures/audit-reef/minimal.docx is a real OOXML zip — a 952-byte
container whose header bytes are NUL-bearing by construction — so arm (a)
correctly flagged it. The file is required by the SRI spec (spec.md:59, "a
.docx"), so the fix is the documented allowlist mechanism, not deletion.

Same treatment as present/ripwire-showcase.pptx, which is the identical
container class. Arm (c) keeps the row honest: it fails a row that matches no
tracked file, and it passes here. Verified the row is TIGHT rather than
over-broad — .md, .bin and the prefix-bounded directory rows still scan.

Red at lane head 55f33b5, green at base 50bc05e: lane-caused.
…ilding any repo

gitenvhermeticcheck arm (D) sweeps every gate that runs `git init` — 187 of
them — and requires each to source test/lib/clean-env.sh. The reason is real,
not stylistic: `git -C DIR` changes the working DIRECTORY but not the
ENVIRONMENT, and GIT_DIR / GIT_WORK_TREE / GIT_INDEX_FILE outrank it. This
gate builds three throwaway repos (one in shell at A4, two from Python), so on
a host where one is exported — a git hook running the suite, `git rebase
--exec`, a CI job — `git init` lands in somebody else's repository and every
ownership question is answered from data the gate never selected.

Measured, not assumed: under a hostile GIT_DIR the shipped gate passes all
arms, while a copy with this one line stripped drops to SIX A4 failures
(--owners exits 1, 0 ownership rows, empty owners, churn=''). That is arm (D)
earning its keep.

Changelog fragment records both meta-gate findings as lane-caused defects.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant