Repository navigation
memex build plugin composes the PORTAL's reference set — no fallback to the tester's /app (#3113) - #3130
Conversation
ae753d6 to
1eefbc1
Compare
There was a problem hiding this comment.
🟡 Changes recommended
The PR introduces extensive new async/await usage in src/ (flagged repeatedly) and adds a large temp extraction path without cleanup, both of which are treated as blocking issues under this repo’s operational/concurrency rules.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates the memex build plugin CLI path to follow the same two-image contract as the reusable CI lanes: the tester image executes the build, while the portal image provides the /app reference set + framework identity, with an explicit refusal when --platform-image is not provided.
Changes:
- Require
--platform-imageand refuse early (before any pull) when it’s missing or incorrectly points at the tester image. - Compose and run the gate host using the embedded
.github/scripts/compose-gate-host.sh, and assert tester/platform identity alignment viaframework-identity. - Add a new
MeshWeaver.Cli.Testproject plus targeted regression tests, and update docs/What’s New to reflect the two-image contract.
File summaries
| File | Description |
|---|---|
src/MeshWeaver.Cli/BuildPluginCommand.cs |
Implements the two-image contract, identity checks, gate-host composition, and updated stage execution. |
src/MeshWeaver.Cli/ImageRunner.cs |
Adds composed-host argv helpers, /app extraction, uid:gid resolution, and verbose process capture. |
src/MeshWeaver.Cli/GateHost.cs |
Introduces a single place for composition script embedding/extraction and tester/platform guard helpers. |
src/MeshWeaver.Cli/Program.cs |
Adds the --platform-image option wiring into BuildPluginOptions. |
src/MeshWeaver.Cli/MeshWeaver.Cli.csproj |
Embeds compose-gate-host.sh as an EmbeddedResource (single implementation of composition rules). |
src/MeshWeaver.Cli/README.md |
Documents the two-image contract and the required --platform-image usage. |
src/MeshWeaver.Documentation/Data/Architecture/InMeshBuildAndTest.md |
Updates the “3-line contract” and adds a “Two IMAGES, two roles” section. |
src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-02-plugin-builds-compile-against-the-portal.md |
Adds a What’s New entry (Fix) describing the behavior change and rationale. |
test/MeshWeaver.Cli.Test/MeshWeaver.Cli.Test.csproj |
New test project for CLI seam/regression tests without requiring Docker. |
test/MeshWeaver.Cli.Test/PlatformImageIsRequiredTest.cs |
Tests refusal behavior and “refuse-before-pull” property via empty stdout. |
test/MeshWeaver.Cli.Test/GateHostCompositionTest.cs |
Drift-guard tests for script byte-equality, extraction + executable bit, self-test execution, and composed-host argv. |
test/Directory.Build.props |
Excludes MeshWeaver.Cli.Test from the global MeshWeaver.Messaging using injection. |
MeshWeaver.slnx |
Adds the new MeshWeaver.Cli.Test project to the solution. |
Review details
Suppressed comments (22)
src/MeshWeaver.Cli/BuildPluginCommand.cs:94
- Async/await was added here (pull + write). Repo guidance treats await in src/ as a concurrency hazard; prefer IObservable composition and move the docker interaction behind the IO boundary (IIoPool) instead of awaiting directly.
var pinned = await _runner.PullImage(options.Image, ct);
if (pinned is null) return 4;
await output.WriteLineAsync($"tester image: {pinned}");
src/MeshWeaver.Cli/BuildPluginCommand.cs:97
- Async/await was added here for pulling the platform image and printing output; this repo avoids awaits in src/ and expects reactive composition with IO through IIoPool to avoid deadlocks and context loss.
var platform = await _runner.PullImage(options.PlatformImage, ct);
if (platform is null) return 4;
await output.WriteLineAsync($"platform image: {platform}");
src/MeshWeaver.Cli/BuildPluginCommand.cs:105
- These awaited docker extraction calls add async boundaries in src/. Per repo guidance, prefer composing an IObservable pipeline and executing the extraction work via IIoPool rather than awaiting tasks directly.
if (!await _runner.ExtractApp(pinned, testerApp, ct)) return 10;
if (!await _runner.ExtractApp(platform, portalApp, ct)) return 10;
src/MeshWeaver.Cli/BuildPluginCommand.cs:133
- This new awaited CaptureVerbose call introduces async/await in src/. Repo guidance treats awaits here as a concurrency risk; prefer IObservable composition and route process execution through the sanctioned IO boundary (IIoPool).
var (testerRc, testerLine) = await _runner.CaptureVerbose("docker",
["run", "--rm", "--init", "--entrypoint", "/app/mw-plugin-test", pinned,
"--print-framework-identity"], ct);
src/MeshWeaver.Cli/BuildPluginCommand.cs:142
- This awaited error write is part of an async/await-based control flow introduced in src/. The repo’s standard is to keep product code reactive and avoid awaits to prevent scheduler parking/context loss.
if (testerIdentity.Length == 0)
{
await error.WriteLineAsync(
$"error: could not resolve the tester image's framework identity (got: '{testerLine.Trim()}').");
return 10;
src/MeshWeaver.Cli/BuildPluginCommand.cs:147
- This awaited docker run (framework identity check) adds async/await in src/. Prefer reactive composition (IObservable) and execute the docker work via IIoPool to avoid async continuations on the wrong scheduler.
var (identityRc, identityOut) = await _runner.CaptureVerbose("docker",
["run", "--rm", "--init", "-v", $"{portalApp}:/portal:ro",
"--entrypoint", "/app/mw-plugin-test", pinned,
"framework-identity", "/portal", "--expect", testerIdentity], ct);
src/MeshWeaver.Cli/BuildPluginCommand.cs:156
- This awaited error write participates in the new async/await flow in src/. Repo guidance is to keep product code reactive; avoid awaits and model this as observable composition with explicit error propagation.
if (identityRc != 0 || frameworkIdentity.Length == 0)
{
await error.WriteLineAsync(
"error: the tester image and the platform image do NOT resolve one framework identity "
+ "— they are not one build (the verb's verdict above names the canonical assemblies "
+ "each side lacks). Pin --image and --platform-image from the SAME CD wave.");
src/MeshWeaver.Cli/BuildPluginCommand.cs:166
- Awaiting the bash composition process here introduces an async boundary in src/. The repo’s standard approach is reactive composition with process execution via IIoPool, not direct awaits.
var script = GateHost.ExtractComposeScript(hostRoot);
var composed = await _runner.Exec("bash", [script, portalApp, testerApp, gateHost], ct);
if (composed != 0)
src/MeshWeaver.Cli/BuildPluginCommand.cs:170
- This awaited error write is in the new async/await path added under src/. Repo guidance is to avoid awaits; model refusals/errors as observable failures and keep the IO on IIoPool.
if (composed != 0)
{
await error.WriteLineAsync(
composed == 127
? $"error: could not run {GateHost.ComposeScriptName} — 'bash' is not on PATH. "
src/MeshWeaver.Cli/BuildPluginCommand.cs:177
- ResolveInvokingUser is awaited here, adding another async continuation in src/. Prefer reactive composition and run the host-side
idprobes via the IO boundary (IIoPool).
var invokingUser = await _runner.ResolveInvokingUser(ct);
src/MeshWeaver.Cli/BuildPluginCommand.cs:213
- This awaited upstream seed fetch continues the async/await flow in src/. Repo conventions prefer reactive composition, with the HTTP/IO edge routed through IIoPool instead of awaited directly in command logic.
if (options.UpstreamSeed is { Length: > 0 } upstreams)
{
var seeded = await FetchUpstreamSeed(
options.RegistryUrl!, options.RegistryKey!, upstreams, frameworkIdentity, seedDir, ct);
if (!seeded) return 8;
src/MeshWeaver.Cli/BuildPluginCommand.cs:226
- Awaiting ComposeSealedModules here introduces/extends async control flow in src/. Per repo guidance, avoid await and compose the operation reactively, with IO via IIoPool.
var fetched = options.UpstreamSeed is { Length: > 0 } sources
? await ComposeSealedModules(
options.RegistryUrl!, options.RegistryKey!, sources, frameworkIdentity,
src/MeshWeaver.Cli/BuildPluginCommand.cs:229
- Awaiting FetchRegistryModules here continues the async/await pattern in src/. Repo standard is to keep product code reactive and push network/file IO behind IIoPool.
options.RegistryUrl!, options.RegistryKey!, sources, frameworkIdentity,
options.RegistryModules!, extDir, ct)
: await FetchRegistryModules(
options.RegistryUrl!, options.RegistryKey!, options.RegistryModules!, extDir, ct);
src/MeshWeaver.Cli/BuildPluginCommand.cs:260
- Awaiting RunOnHost here adds another async boundary in src/. Repo guidance expects an observable pipeline (IObservable) and process execution via IIoPool rather than awaiting a Task-returning method.
var compile = await RunOnHost(platform, gateHost, invokingUser, repo, extDir,
mounts: [$"{bake}:/bake"],
env: [],
args: ["compile", "/repo", "--app", "/app",
"--shared-frameworks", GateHost.SharedFrameworks,
..AllowArgs(repo, options), ..moduleArgs, ..options.ExtraArgs,
"--output", "/bake", ..SourceShaArgs(options)],
src/MeshWeaver.Cli/BuildPluginCommand.cs:270
- This awaited error write is part of the Task/await-based flow introduced under src/. Repo conventions prefer reactive error propagation without awaits in product code.
await error.WriteLineAsync(
$"error: compile ran but wrote no bake identity ({BakeIdentityFile} missing or empty) "
+ "— the bake stage regressed. Refusing to test bytes that were never produced.");
return 5;
src/MeshWeaver.Cli/BuildPluginCommand.cs:290
- Returning an awaited RunOnHost continues the async/await pattern in src/. Repo guidance prefers a reactive (IObservable) stage-2 pipeline with explicit Subscribe/error handling and IO via IIoPool.
return await RunOnHost(platform, gateHost, invokingUser, repo, extDir,
mounts: [$"{bake}:/seed:ro"],
env: ["MW_INSTALL_DIFF=1"],
args: ["/repo", "--app", "/app", ..AllowArgs(repo, options), ..moduleArgs,
..options.ExtraArgs, "--seed", "/seed"],
src/MeshWeaver.Cli/ImageRunner.cs:217
- This awaited docker cp path is part of the async/await-based implementation added under src/. Per repo guidance, move process IO behind IIoPool and avoid awaiting tasks directly in product code.
if (await Exec("docker", ["cp", $"{container}:/app", destination], ct) != 0)
{
await error.WriteLineAsync($"error: could not copy /app out of '{image}'.");
return false;
}
src/MeshWeaver.Cli/ImageRunner.cs:222
- Awaiting the docker rm cleanup here adds another async boundary in src/. Repo conventions prefer reactive composition with IO via IIoPool rather than Task/await in product code.
{
await Exec("docker", ["rm", container], ct);
}
src/MeshWeaver.Cli/ImageRunner.cs:237
- ResolveInvokingUser uses async/await in src/ to run host
idcommands. Repo guidance prefers keeping product code reactive and routing OS/process IO via IIoPool rather than awaiting tasks here.
public async Task<string?> ResolveInvokingUser(CancellationToken ct)
{
if (OperatingSystem.IsWindows()) return null;
var uid = (await Capture("id", ["-u"], ct)).Trim();
var gid = (await Capture("id", ["-g"], ct)).Trim();
if (uid.Length > 0 && gid.Length > 0) return $"{uid}:{gid}";
src/MeshWeaver.Cli/ImageRunner.cs:241
- This awaited output write is part of the async/await flow introduced under src/. Repo conventions prefer reactive composition and avoiding awaits in product code (especially around process/IO boundaries).
await output.WriteLineAsync(
"note: could not resolve this user's uid/gid ('id' unavailable) — the run uses the "
+ "platform image's own user, so anything it writes into an output directory will be "
+ "owned by that user.");
src/MeshWeaver.Cli/ImageRunner.cs:269
- CaptureVerbose is implemented with async/await in src/. Repo guidance treats awaits here as a concurrency risk; prefer reactive composition and move the process execution behind IIoPool.
var psi = new ProcessStartInfo(file)
{ UseShellExecute = false, RedirectStandardOutput = true, RedirectStandardError = true };
foreach (var a in args) psi.ArgumentList.Add(a);
await output.WriteLineAsync($"$ {file} {string.Join(' ', psi.ArgumentList)}");
using var p = Process.Start(psi);
if (p is null)
{
await error.WriteLineAsync($"error: could not start '{file}' — is it installed and on PATH?");
return (127, string.Empty);
src/MeshWeaver.Cli/ImageRunner.cs:275
- These awaited stream reads / WaitForExitAsync introduce additional async continuations in src/. Repo conventions expect IO/process waiting to happen via the sanctioned IO boundary (IIoPool) and the higher-level flow to remain reactive.
var stdout = await p.StandardOutput.ReadToEndAsync(ct);
var stderr = await p.StandardError.ReadToEndAsync(ct);
await p.WaitForExitAsync(ct);
if (stderr.Length > 0) await error.WriteAsync(stderr);
return (p.ExitCode, stdout);
- Files reviewed: 13/13 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
`memex build plugin` ran `--entrypoint /app/mw-plugin-test` straight from the tester image, so it compiled against the tester's /app — a strict SUBSET of the portal's (88 vs 219 assemblies on 3.0.0-rc9.ci.7534). Content binding a portal-only assembly could not compile: MeshWeaver.Manufacturing#48 went red on AppleMaps/Gallery and Cornerstone/Pricing with `CS0234 … 'Maps' does not exist in the namespace 'MeshWeaver'` — a CONTENT-shaped failure with an INFRASTRUCTURE cause, on source nobody had changed. The reusable lanes fixed this in #3022/#3071; the CLI path did not, so the defect survived one lane later. The CLI now runs the same two-image contract the lanes do: * `--platform-image` (the PORTAL) is REQUIRED beside `--image` (the TESTER), with no fallback to the tester's /app and an explicit refusal by name when the tester is passed as the platform — both BEFORE anything is pulled; * both images' /app are extracted and their shapes asserted (the tester ships the CLI, the portal ships a non-empty surface manifest); * the tester's own `framework-identity` verb must report that the two resolve ONE identity, and the run's own bake is then checked against it; * the gate host is composed by `.github/scripts/compose-gate-host.sh` — the SAME file the lanes fetch, EMBEDDED here rather than reimplemented, so the ordering and fail-closed rules cannot drift into a second copy; * both stages start the PORTAL image's own `dotnet` on `/host/mw-plugin-test.dll` with `--app /app` (plus `--shared-frameworks` on the compile), as the invoking uid so the bake is not left root-owned; * the upstream seed and the sealed module set are addressed by the PLATFORM's identity, not the tester's — a seed under the tester's address would name a publication the gate never adopts. The extracted script is `chmod +x`'d, as the lanes do: it re-invokes itself by path to prove its own rules, and a 0644 copy fails its self-test on "Permission denied" having verified nothing (caught by the new test, not by inspection). Every string added is an operator diagnostic on a CI tool's stdout/stderr — English by the Localization boundary, like the rest of this CLI (zero `Localize(` calls). New test project test/MeshWeaver.Cli.Test (the CLI had none): the argv is pinned pure, the composition script is pinned byte-for-byte against .github/scripts/ and executed through its own --self-test, and the refusals are asserted to happen before the first `docker pull`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1eefbc1 to
52ecce5
Compare
Test Results (shard 5) 4 files + 1 4 suites +1 1m 26s ⏱️ +15s Results for commit 52ecce5. ± Comparison against base commit 8649999. This pull request removes 194 and adds 221 tests. Note that renamed tests count towards both. |
Test Results (shard 4)2 131 tests +297 2 131 ✅ +297 2m 5s ⏱️ -8s Results for commit 52ecce5. ± Comparison against base commit 8649999. This pull request removes 22 and adds 319 tests. Note that renamed tests count towards both. |
Test Results (shard 3)340 tests - 297 338 ✅ - 299 37s ⏱️ -18s Results for commit 52ecce5. ± Comparison against base commit 8649999. This pull request removes 319 and adds 22 tests. Note that renamed tests count towards both. |
Test Results (shard 2)576 tests - 8 384 ✅ - 198 2m 33s ⏱️ -14s Results for commit 52ecce5. ± Comparison against base commit 8649999. This pull request removes 224 and adds 216 tests. Note that renamed tests count towards both.This pull request removes 2 skipped tests and adds 192 skipped tests. Note that renamed tests count towards both. |
Fixes #3113.
The defect
memex build pluginrandocker run … --entrypoint /app/mw-plugin-test <tester>directly, so itcompiled against the tester image's
/app— a strict SUBSET of the portal's (88 vs 219assemblies on
3.0.0-rc9.ci.7534;MeshWeaver.Maps,.AI,.ContentCollections.Indexing, theBlazor and hosting halves exist only in the portal). Content binding a portal-shipped assembly could
not compile, and the failure read as a CONTENT error on source nobody had changed:
This is the same defect #3022/#3071 fixed for every reusable lane, surviving on the CLI path. It
blocks MeshWeaver.Manufacturing#48, whose
memex build plugin (new build process)job is red onevery run at the current pin.
The fix — the lanes' two-image contract, on the CLI
--platform-imageis REQUIRED beside--image. The tester EXECUTES; the portal supplies thereference set, the framework identity and the runtime. No fallback to the tester's
/app—omitting it is a refusal, and passing the tester as the platform is refused by name. Both happen
BEFORE the first
docker pull, so a misconfigured job fails in seconds naming its input./appare extracted and their shapes asserted — the tester must shipmw-plugin-test.dll, the portal a non-emptymeshweaver-surface.manifest(a host without oneresolves the fallback identity no bake may be keyed to).
framework-identity … --expectverb must report thatthe two images resolve one identity, and its stderr verdict — which names the canonical assemblies
each side lacks (PROD memex: every course cover serves the 'type has not finished compilation' fallback — Store/Plugin build has not settled #1814) — is forwarded rather than swallowed.
.github/scripts/compose-gate-host.sh, and both stages run--entrypoint dotnet <portal> /host/mw-plugin-test.dll … --app /app(plus--shared-frameworks /usr/share/dotnet/sharedon the compile), as the invoking uid withHOME=/tmp. Same shape asnode-repo-gate.yml, deliberately.identity no portal asks for are inert.
tester's. Previously the seed was fetched under the tester's; that addressed a publication the
gate would never adopt.
One implementation of the composition rules, not two
🚨 The composition rules are not reimplemented in C#.
MeshWeaver.Cli.csprojembeds.github/scripts/compose-gate-host.sh— the very file the lanes fetch from the platform at theirpinned ref — and the CLI extracts and runs it.
GateHostCompositionTestpins the two to the sameSHA-256, so a divergent copy fails the build rather than drifting silently.
Verification
dotnet build -c Release -warnaserror:MeshWeaver.Cli,MeshWeaver.Documentation,MeshWeaver.Documentation.Test,MeshWeaver.Cli.Test— 0 Error(s), 0 Warning(s) each.test/MeshWeaver.Cli.Test(new): 16/16, unfiltered.test/MeshWeaver.Documentation.Test:202/202, unfiltered.
--platform-imageguard replaced by a fall-back-to---image, 7 of 15 went red — including thethree refusal tests, which then spent 20 s each actually reaching
docker pull. That is theproperty the tests assert:
ImageRunnerechoes every process it starts, so an empty stdout isthe positive evidence that the refusal happened before anything was pulled.
bash .github/scripts/compose-gate-host.sh --self-test→ OK;bash -nclean;shellcheck -S warningclean. (The script itself is unchanged — it is reused, not edited.) Thenew
TheExtractedScript_PassesItsOwnSelfTestdrives that same self-test from the bytes the CLIputs on disk.
dotnet packsucceeds and the shippedtools/net10.0/any/memex.dllcarries the script.A finding the test produced
The first run of
TheExtractedScript_PassesItsOwnSelfTestfailed withPermission denied:File.WriteAllByteswrites 0644, and the script re-invokes itself by path to prove each of itsrules. The lanes
chmod +xthe script they fetch;ExtractComposeScriptnow does the same. Theproduction path (
bash <script> …) would have worked either way — but a self-test that cannotexecute is a proof that verified nothing, which is exactly the shape this repo's rules forbid.
Strings
Every string added is an operator diagnostic on a CI tool's stdout/stderr — a build refusal
naming an image, a
/app, a framework identity. English by theLocalization
boundary ("a viewer's message vs. an owner's diagnostic"), consistent with the rest of this CLI,
which contains zero
Localize(calls. Nothing here reaches a portal viewer.Docs
Doc/Architecture/InMeshBuildAndTestgains a "Two IMAGES, two roles" section (what the tester'ssubset cannot compile, why there is no fallback, where the one copy of the rules lives);
src/MeshWeaver.Cli/README.mdis updated; a What's New entry (Category: Fix) ships with it.Follow-up (separate repo, after this publishes)
MeshWeaver.Manufacturing#48's
memex build pluginjob must add--platform-image meshweaver.azurecr.io/memex-portal-ai@${{ vars.MW_PORTAL_IMAGE_DIGEST }}once aMeshWeaver.Clicarrying this lands — the variable its ownnode-repo-*lanes already read. Untilthen that job refuses with a named reason instead of failing on
CS0234against innocent content.Pairs-with: none — no public type is removed;
GateHostand theImageRunnermembers are additions,and
PlatformImageis an init-property (not a primary-constructor parameter, so the record'ssynthesized arity is unchanged).
🤖 Generated with Claude Code