From aaee3329317a75247c6c77d11ab40a5f3c920ad5 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 17 Jul 2026 23:57:07 +0000 Subject: [PATCH 01/10] docs: add warm dev loop proposal (mxbuild --serve + reload_model) Draft proposal covering two scenarios: a Docker-free `mxcli dev` warm run loop to replace the docker inner loop, and an iPad split-screen preview workflow (Claude Code Web + browser, preview-before-commit). Built on measured mxbuild --serve incremental builds (~0.8s warm vs ~30-60s one-shot) and the runtime reload_model hot-reload primitive (~0.1s, no restart), with restartRequired branching page/microflow from entity/DDL changes. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../PROPOSAL_mxcli_dev_warm_loop.md | 246 ++++++++++++++++++ 1 file changed, 246 insertions(+) create mode 100644 docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md new file mode 100644 index 000000000..d49277401 --- /dev/null +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -0,0 +1,246 @@ +--- +title: warm dev loop — Docker-free run and iPad split-screen preview +status: draft +date: 2026-07-17 +--- + +# Proposal: Warm dev loop (`mxbuild --serve` + `reload_model`) + +**Status:** Draft +**Date:** 2026-07-17 +**Relates to:** `PROPOSAL_check_mxbuild_gap_heuristics.md` (the static-check gate that +runs *before* this loop ever builds), `.claude/skills/mendix/docker-workflow.md` +(the loop this replaces for inner-loop iteration). + +## Problem Statement + +The current way to run and test a Mendix app with mxcli is `mxcli docker build` / +`mxcli docker run`. It has two costs that dominate the edit→test loop: + +1. **Full build every time.** The one-shot `mxbuild` CLI recompiles the whole + deployment on each invocation. Measured on a one-entity/one-page/one-microflow + app: **~30–60 s per build** (Java compile + full model export + package). +2. **Docker.** The runtime and database run in containers, adding image pulls and + container lifecycle to every cycle — and Docker is **unavailable** in important + contexts: Claude Code on the web, and iPad. In those environments the current + loop cannot run at all. + +Two primitives, both verified to exist and work in Mendix **11.6.3+**, make a far +tighter loop possible: + +- **`mxbuild --serve`** — an HTTP build server (default port 6543) that keeps the + model loaded and rebuilds **incrementally**, skipping unchanged work (including + Java compilation when no Java changed). Each build returns `restartRequired`. +- **`reload_model`** — a runtime admin-port action that swaps the model into the + **running** JVM in-place, draining in-flight actions first (near-zero-downtime). + No process restart. + +This proposal wires those two primitives into mxcli to serve **two scenarios**: + +- **Scenario A — a fast, Docker-free `mxcli dev` loop** to replace `mxcli docker` + for inner-loop iteration. +- **Scenario B — an iPad split-screen workflow**: Claude Code on the web in one + pane, a browser preview in the other, prompt → build → test, all on the iPad, + **without committing to the repo** before testing. + +## BSON Structure + +Not applicable — this is a build/runtime orchestration feature. It touches no +Mendix document serialization. It drives two already-existing interfaces: the +mxbuild HTTP build API and the runtime M2EE admin API. + +## Measured evidence + +All numbers below were measured directly (Mendix 11.6.3, Linux x86-64, JDK 21) on +a one-entity/one-page/one-microflow app during the investigation that motivated +this proposal. + +| Step | Time | Notes | +|------|------|-------| +| `mxbuild` one-shot CLI (full) | **~30–60 s** | current loop; no incremental mode | +| `mxbuild --serve`, cold (first build) | ~13.7 s | loads model into the server | +| `mxbuild --serve`, warm, **no change** | ~1.1 s | model cached | +| `mxbuild --serve`, warm, **microflow change** | **~0.8 s** | Java **not** recompiled | +| `mxbuild --serve`, warm, **entity/attribute change** | ~7.3 s | `restartRequired: true` | +| `reload_model` (hot, no restart) | **~0.07–0.27 s** | `"reload": true`, JVM pid unchanged | +| **Full warm loop** (exec → serve build → reload) | **~1 s** | microflow/page change, no restart | + +`restartRequired` from the serve API cleanly separates the two change classes: + +- **page / microflow change** → `restartRequired: false` → `reload_model` (~0.1 s) +- **entity / domain change** → `restartRequired: true` → `execute_ddl_commands` + + runtime restart (the DDL gate is real and unavoidable — the runtime refuses a + stale schema) + +`mxbuild --serve` and `reload_model` are present in both **11.6.3** and **11.12.1** +(verified). The 11.12 CLI adds no build-skip flags (only `--export-secrets`), so the +incremental path is `--serve`, not a new one-shot flag. + +## Proposed CLI + +### Scenario A: `mxcli dev` — Docker-free warm run loop + +A long-lived local dev supervisor. Boots (or reuses) the standalone runtime + +Postgres, starts `mxbuild --serve`, and on each model change runs +`serve build (Deploy)` → branch on `restartRequired` → `reload_model` or +DDL+restart. + +```bash +# start a warm dev session (no Docker) +mxcli dev -p app.mpr +# → downloads/caches mxbuild + runtime (once), starts Postgres, +# boots the runtime, starts `mxbuild --serve`, prints the local URL, +# and holds everything warm. + +# apply a change and hot-reload (from another shell, or driven by the agent) +mxcli dev reload -p app.mpr # serve-build + restartRequired-aware reload +mxcli dev exec change.mdl -p app.mpr # exec MDL, then reload in one step +mxcli dev status # runtime status, warm-build server health +mxcli dev stop +``` + +`mxcli dev` **subsumes the inner loop of `mxcli docker run`** while keeping +`mxcli docker` for produced-artifact / container-parity builds. + +Optionally a `--watch` mode: watch the `.mpr` (v2 `mprcontents/`) for changes and +auto-run the reload cycle. + +### Scenario B: `mxcli dev serve` + a preview container (iPad) + +The iPad has two panes: **Claude Code on the web** (prompt + edit) and **Safari** +(the running app). The catch, established during investigation: the Claude Code web +sandbox is **egress-only and cannot expose a port** (all reverse tunnels are blocked +by a 443-only + TLS-intercepting egress policy). So the *running* app must live in a +container that has real public ingress — a **Codespace** (public port forwarding → +`https://-8080.app.github.dev`) is the natural fit. + +Two containers, coupled **without a commit** (the explicit requirement — preview WIP +before committing): + +``` +┌─────────────────────────┐ push built deployment ┌──────────────────────────┐ +│ DEV: Claude Code Web │ ── (443, authenticated, no git) ──▶ │ PREVIEW: Codespace │ +│ edits app.mpr │ │ runtime + mxbuild --serve │ +│ mxbuild --serve (build) │ │ Postgres │ +│ │ ◀── agent verifies via public URL ── │ :8080 forwarded PUBLIC │ +└─────────────────────────┘ (curl / Playwright) └──────────────────────────┘ + Safari tab ▲ (iPad) +``` + +- **DEV** (Claude Code Web): prompt Claude → `mxcli` edits `app.mpr` → + `mxbuild --serve` produces the Deploy artifact → `mxcli dev push` uploads it to the + Preview container over 443. +- **PREVIEW** (Codespace): receives the artifact, runs `reload_model` (or DDL+restart + on `restartRequired`), serves the public URL. Booted by a `devcontainer.json` + + `postStart` script that reproduces the standalone-boot recipe. +- **iPad UX:** prompt in the Claude pane → ~1–2 s → refresh the Safari tab → see the + change. Nothing is committed until the user is happy. + +Proposed commands: + +```bash +# in the Preview container (Codespace), started by the devcontainer: +mxcli dev serve -p app.mpr --intake-port 9000 --intake-secret $TOKEN +# → boots runtime + Postgres + mxbuild --serve, forwards 8080 public, +# exposes an authenticated /deploy intake for artifact pushes. + +# in the DEV container (Claude Code Web): +mxcli dev push --to https://-9000.app.github.dev --secret $TOKEN -p app.mpr +# → mxbuild --serve (Deploy) locally, then upload the deployment delta; +# the Preview container reloads and returns the live status. +``` + +### Coupling: options considered + +| Option | Preview WIP w/o commit? | Notes | +|--------|-------------------------|-------| +| **A. Push built deployment artifact over 443** (recommended) | ✅ | robust; runtime in a legit host; ~1–2 s + network | +| B. Push MDL/model, build in the Preview container | ✅ | keeps a synced `.mpr` in the Preview container; heavier transfer for MPR v2 | +| C. Git branch, Codespace pulls | ❌ | requires a commit — **rejected**, violates the core requirement | +| D. Live tunnel (app stays in DEV, Codespace relays) | ✅ | instant, but fragile (chisel over two proxies), ties the app to the ephemeral sandbox, and Codespace-as-relay is an AUP gray area — **not recommended** | + +## Implementation Plan + +Reuse the existing `cmd/mxcli/docker/` plumbing wherever possible — it already +downloads/caches mxbuild + runtime and speaks the M2EE admin API. + +### Files to modify/create + +| File | Change | +|------|--------| +| `cmd/mxcli/dev.go` | New `mxcli dev` command tree (`dev`, `dev reload`, `dev exec`, `dev serve`, `dev push`, `dev status`, `dev stop`). | +| `cmd/mxcli/docker/mxserve.go` | New: `mxbuild --serve` client — start the daemon, `POST /build {target:Deploy}`, parse `status` + `restartRequired`. | +| `cmd/mxcli/docker/admin.go` | Extract the M2EE admin client (auth header `X-M2EE-Authentication: base64(pass)`, actions `update_appcontainer_configuration`, `update_configuration`, `start`, `execute_ddl_commands`, **`reload_model`**, `runtime_status`) out of `docker/oql.go` into a reusable client. | +| `cmd/mxcli/docker/localboot.go` | New: standalone runtime boot (see the config-set below), so `dev` can boot without Docker. | +| `cmd/mxcli/docker/download.go` | Reuse `DownloadMxBuild` / `DownloadRuntime` (no change). | +| `cmd/mxcli/docker/mxenv.go` | Reuse `PrepareMxCommand` (LD_PRELOAD FreeType fix) for the `mx`/`mxbuild` children (no change). | +| `.devcontainer/preview/devcontainer.json` + `boot.sh` | New: Codespace preview container — `forwardPorts: [8080]` public, Postgres, `postStart` → `mxcli dev serve`. | +| `.claude/skills/mendix/docker-workflow.md` | Add a "warm dev loop" section pointing at `mxcli dev`. | +| `cmd/mxcli/dev_test.go` | Tests for the serve client and the `restartRequired` branch logic (mock the serve/admin HTTP endpoints). | + +### The standalone-boot config set (discovered, must be encoded) + +The standalone runtime launcher (`com.mendix.container.boot.Main`, driven via the +admin API) requires a specific config that Studio Pro / the buildpack normally +supply. `mxcli dev` must set all of it or `start` fails: + +- Env: `MX_INSTALL_PATH`, `M2EE_ADMIN_PASS`, `M2EE_ADMIN_PORT`, and the FreeType + `LD_PRELOAD` on the JVM. +- Ensure the `data/{files,tmp,model-upload}` dirs exist under the deployment. +- `update_configuration`: `BasePath` (deployment dir), `RuntimePath` + (`/runtime`), `DatabaseType/Host/Name/UserName/Password` + (`DatabaseHost` includes the port, e.g. `127.0.0.1:5432`), `DTAPMode`, and + **`MicroflowConstants`** (design-time default values are *not* auto-applied + standalone — missing a constant surfaces at runtime as HTTP 530 → login bounce). +- Boot sequence: `update_appcontainer_configuration` (runtime port) → + `update_configuration` → `start` → on `"database has to be updated"`: + `execute_ddl_commands` → `start`. +- Reload cycle: `mxbuild --serve` Deploy build → if `restartRequired` then + `execute_ddl_commands` (if schema changed) + restart, else `reload_model`. + +## Version Compatibility + +- `mxbuild --serve` and `reload_model`: **≥ 11.6.3** (verified on 11.6.3 and + 11.12.1). No new flag needed on 11.12 — the incremental path is `--serve` in both. +- Register the feature in `sdk/versions/mendix-11.yaml` (and 10 if we verify the + admin action name `reload_model` there) with the correct `min_version`, and add a + `checkFeature()` pre-check with an actionable hint. +- **Production-representative testing note:** running `DTAPMode=P` requires the app's + security level to be `CHECKEVERYTHING` (Production) or `start` refuses + (`result:11`). `mxcli dev` should default to `D` for the inner loop and expose a + `--dtap P` flag; when `P` is set, surface the security-level requirement up front. + +## Test Plan + +- `cmd/mxcli/docker/mxserve_test.go` — mock the serve HTTP endpoint; assert request + shape (`{target:Deploy, projectFilePath}`) and `restartRequired` parsing. +- `cmd/mxcli/docker/admin_test.go` — mock the admin endpoint; assert the boot + sequence and the `reload_model` vs DDL+restart branch. +- Integration (gated, needs mxbuild + runtime + Postgres, like the existing docker + tests): boot a fixture app, do a microflow change → assert warm reload with pid + unchanged; do an entity change → assert `restartRequired` → DDL path. +- `mdl-examples/` — a small fixture app the dev-loop tests drive. + +## Open Questions + +1. **Process lifecycle.** Long-lived `mxbuild --serve` and the runtime JVM need + supervision (restart on crash, port cleanup). PID files under `.mxcli/`? A + `dev status`/`dev stop`? (In the sandbox, background JVMs were reaped on idle — + the Codespace/preview host must keep them alive; document the idle-stop caveat.) +2. **Artifact transfer for Scenario B.** Push the whole Deploy `model/` each time, or + compute a delta? MPR v2 is a folder; the *deployment* `model/` is more compact, + but still worth measuring. What's the auth model for the intake endpoint (bearer + secret over the public forward)? +3. **Codespaces AUP.** Running your own app for preview is intended use; the + Codespace-as-relay variant (Option D) is a gray area. The proposal recommends A + (artifact push) precisely to stay clearly on the "previewing your own project" + side. Confirm this framing. +4. **Warm-serve memory.** `mxbuild --serve` holds the model in memory; quantify for + large apps and document guidance. +5. **`reload_model` scope.** Verified for microflow changes (hot) and flagged + correctly for entity changes (`restartRequired`). Not yet exercised: styling-only + (`update_styling` exists as a separate admin action) and multi-unit changes. + Worth a coverage pass before shipping. +6. **Relationship to `check`.** The instant `mxcli check` gate (see + `PROPOSAL_check_mxbuild_gap_heuristics.md`) should run *before* every warm build so + most errors never reach even the ~0.8 s build. `mxcli dev` should call it first. From 673710c901d5e5366207a2c01bbd7ecd78367488 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 00:04:35 +0000 Subject: [PATCH 02/10] docs: add prototype-to-production workflow to warm dev loop proposal Frame the iPad scenario as one leg of the full journey: Claude design prototype -> Mendix model via mxcli/MDL -> production security (ALTER PROJECT SECURITY LEVEL PRODUCTION, per-role grants) -> live warm-loop testing under real security, all on iPad without Studio Pro. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../PROPOSAL_mxcli_dev_warm_loop.md | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index d49277401..39ff5f4d5 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -43,6 +43,40 @@ This proposal wires those two primitives into mxcli to serve **two scenarios**: pane, a browser preview in the other, prompt → build → test, all on the iPad, **without committing to the repo** before testing. +## End-to-end workflow: Claude design prototype → secured production Mendix app (on iPad) + +Scenario B is one leg of a larger journey this loop is meant to unlock — taking an +idea all the way to a **secured, production-grade Mendix app entirely on an iPad**, +with no desktop Studio Pro in the loop: + +1. **Prototype in Claude.** A Claude design Artifact (a clickable HTML/React mockup) + captures the app's screens, data, and flows — fast and throwaway, no Mendix yet. +2. **Translate to Mendix in Claude Code (web).** In the same iPad session, mxcli/MDL + turns the prototype into a real Mendix model: `create entity` / `create page` / + `create microflow` / navigation. The prototype's structure becomes actual `.mpr` + documents — a running Mendix app, not a mockup. +3. **Secure it for production.** Raise the app from prototype to production security + and make it pass the real Mendix consistency rules — the exact path proven during + the investigation behind this proposal: + - `ALTER PROJECT SECURITY LEVEL PRODUCTION` — required before `DTAPMode=P` will + even boot (otherwise `start` returns `result:11 — security must be + CHECKEVERYTHING`). + - `grant` / `revoke` page, entity, and microflow access per module role; define + user roles and their mappings; wire the login and anonymous navigation profiles. + - `mxcli check` (instant) plus the warm `mxbuild` build surface the + production-only errors up front (e.g. *"page needs an allowed role"*, which we + hit and fixed with a single `grant`) instead of discovering them at deploy time. +4. **Test each change live, on the iPad.** The warm loop (§Proposed CLI) previews + every edit in ~1 s in the Safari pane, under **real production security** (login + enforced, anonymous blocked) — so the author iterates prototype → model → secured + app → test without ever leaving the tablet. + +The deliverable is not a throwaway preview but a **secured, production-representative +Mendix app** — the same artifact you would deploy. The warm dev loop is what makes +step 4 fast enough for this to be a genuine authoring experience rather than a batch +process, and Scenario B is what makes it work on a device that has neither Docker nor +Studio Pro. + ## BSON Structure Not applicable — this is a build/runtime orchestration feature. It touches no From a8404b4d04bbab1983b7967a0c098c4da5664e9c Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 02:14:47 +0000 Subject: [PATCH 03/10] docs: add hot-reload scope + DB metamodel-catalog root cause to warm-loop proposal Verified reload_model reloads model store + microflows + i18n only, not the datastorage/entity layer. The runtime reconciles its metamodel catalog (mendixsystem$entity/entityidentifier/attribute/association) at startup, so entity/view-entity/association changes need a restart. Confirmed directly: a view entity added via reload_model never appears in mendixsystem$entity. Adds the two-signal decision matrix (restartRequired x get_ddl_commands) and the key nuance that OQL view entities need restart WITHOUT DDL (query-time, not materialized). Notes the standalone boot lacks preview_execute_oql for functional OQL verification. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../PROPOSAL_mxcli_dev_warm_loop.md | 63 +++++++++++++++++-- 1 file changed, 57 insertions(+), 6 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index 39ff5f4d5..404f78703 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -110,6 +110,50 @@ this proposal. (verified). The 11.12 CLI adds no build-skip flags (only `--export-secrets`), so the incremental path is `--serve`, not a new one-shot flag. +## Hot-reload scope: what `reload_model` can and cannot do (verified) + +`reload_model` reloads exactly three subsystems — the model store +(`ProjectModelStoreLoader.load`), the microflow engine +(`MicroflowEngineModule.reload`), and translations (`I18NProcessor.reload`) — after +draining in-flight actions. It does **not** touch the datastorage / entity layer. + +The runtime maintains a **metamodel catalog inside the app database** — +`mendixsystem$entity`, `mendixsystem$entityidentifier`, `mendixsystem$attribute`, +`mendixsystem$association` — and reconciles it **at startup**, not during a reload. +This was verified directly: a view entity added via `reload_model` (`TaskView`) +returned `Success` but **never appeared in `mendixsystem$entity`**, while the base +`Task` (present at the last startup) has its row (`entity_name` → `table_name` → a +stable identifier GUID). So any change that needs a catalog row — a new/changed +entity, **including OQL view entities**, and associations in +`mendixsystem$association` (e.g. a view entity referencing a persistent entity) — +requires a **restart**, because that is when the catalog is reconciled. + +This makes the apply-decision **two-dimensional**: `restartRequired` (from the serve +build) and `get_ddl_commands` (from the runtime) are **independent** signals. + +| Change class | `restartRequired` | `get_ddl_commands` | Apply via | +|---|---|---|---| +| microflow / page / text | `false` | 0 | `reload_model` (~0.1 s, hot) | +| **OQL view entity** | **`true`** | **0** | **restart** — catalog sync, **no DDL** | +| persistable entity / attribute | `true` | > 0 | `execute_ddl_commands` → restart | +| association (adds FK) | `true` | > 0 | `execute_ddl_commands` → restart | + +The loop must branch on **both** signals: `restartRequired == false` → `reload_model`; +`restartRequired == true` → restart, running `execute_ddl_commands` **only if** +`get_ddl_commands > 0`. Treating "restart" as implying "DDL" is wrong — **OQL view +entities are the counterexample**: they need a restart (to write the +`mendixsystem$entity` catalog row) but no DDL (they are query-time, not materialized — +confirmed: no view/table for `TaskView` in Postgres). + +**Caveat (not fully closed):** `reload_model` *returns* `Success` for a view-entity +change (it doesn't reject it), and I could not run the functional tiebreaker — +querying `TaskView` post-reload — because the standalone hand-boot does not register +the `preview_execute_oql` admin action (it needs the dev-preview flag `mxcli docker` +sets). The conclusion rests on the authoritative `restartRequired` signal plus the +catalog evidence, which agree. Enabling the dev-preview flag on the standalone boot +(so the agent can verify via OQL) is itself a small item worth doing — see Open +Questions. + ## Proposed CLI ### Scenario A: `mxcli dev` — Docker-free warm run loop @@ -229,8 +273,10 @@ supply. `mxcli dev` must set all of it or `start` fails: - Boot sequence: `update_appcontainer_configuration` (runtime port) → `update_configuration` → `start` → on `"database has to be updated"`: `execute_ddl_commands` → `start`. -- Reload cycle: `mxbuild --serve` Deploy build → if `restartRequired` then - `execute_ddl_commands` (if schema changed) + restart, else `reload_model`. +- Reload cycle (two independent signals — see § Hot-reload scope): `mxbuild --serve` + Deploy build → if `restartRequired == false` then `reload_model`; else restart, + running `execute_ddl_commands` first **only if** `get_ddl_commands > 0`. Do **not** + couple "restart" to "DDL" — OQL view entities need a restart with zero DDL. ## Version Compatibility @@ -271,10 +317,15 @@ supply. `mxcli dev` must set all of it or `start` fails: side. Confirm this framing. 4. **Warm-serve memory.** `mxbuild --serve` holds the model in memory; quantify for large apps and document guidance. -5. **`reload_model` scope.** Verified for microflow changes (hot) and flagged - correctly for entity changes (`restartRequired`). Not yet exercised: styling-only - (`update_styling` exists as a separate admin action) and multi-unit changes. - Worth a coverage pass before shipping. +5. **`reload_model` scope (largely resolved — see § Hot-reload scope).** Verified: + microflow changes hot-reload; entity/view-entity/association changes flip + `restartRequired` because the runtime reconciles its DB metamodel catalog + (`mendixsystem$entity` / `entityidentifier` / `attribute` / `association`) at + startup, not on reload. Open sub-items: (a) enable the `preview_execute_oql` + dev flag on the standalone boot so the loop can *functionally* verify a change via + OQL (needed to close the view-entity caveat and to power agentic verification); + (b) exercise styling-only changes (`update_styling` is a distinct admin action — + likely a third hot path); (c) multi-unit and mixed changes. 6. **Relationship to `check`.** The instant `mxcli check` gate (see `PROPOSAL_check_mxbuild_gap_heuristics.md`) should run *before* every warm build so most errors never reach even the ~0.8 s build. `mxcli dev` should call it first. From 4a799cdfb36bbfacef48872accaa3e6dd74e6c33 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 02:41:11 +0000 Subject: [PATCH 04/10] docs: add tunnel-hub scaling (self-registration + admin overview) to warm-loop proposal One static hub fronting many dynamic dev containers, verified via chisel: - chisel fan-in (one server, many clients on distinct ports; --authfile isolation) - why chisel works where ngrok/cloudflared/localtunnel didn't (go install past the gate, single-443 multiplex, system trust store already holds the MITM CA) - mxcli tunnel-hub (static) + mxcli dev --hub (self-registering dynamic side) - registration flow over chisel's authfile-reload + client-declared-remote primitives - admin overview page: fleet list + per-container pending-change summaries - Mendix fleet wrinkles (subdomain ApplicationRootUrl, cookie/DB isolation, slots) - reclassify coupling Option D (live tunnel) from "not recommended" to verified/preferred for the multi-container dev loop, artifact-push kept as durable fallback Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../PROPOSAL_mxcli_dev_warm_loop.md | 95 ++++++++++++++++++- 1 file changed, 94 insertions(+), 1 deletion(-) diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index 404f78703..011414800 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -235,7 +235,81 @@ mxcli dev push --to https://-9000.app.github.dev --secret $TOKEN -p app.mp | **A. Push built deployment artifact over 443** (recommended) | ✅ | robust; runtime in a legit host; ~1–2 s + network | | B. Push MDL/model, build in the Preview container | ✅ | keeps a synced `.mpr` in the Preview container; heavier transfer for MPR v2 | | C. Git branch, Codespace pulls | ❌ | requires a commit — **rejected**, violates the core requirement | -| D. Live tunnel (app stays in DEV, Codespace relays) | ✅ | instant, but fragile (chisel over two proxies), ties the app to the ephemeral sandbox, and Codespace-as-relay is an AUP gray area — **not recommended** | +| D. Live tunnel (app stays in DEV, static relay forwards) | ✅ | **viable — chisel verified** (see § Scaling). Cleanest dev loop: hot-reload stays local, no artifact sync, and the relay is a generic static component. Caveat: the DEV runtime is ephemeral. **Preferred for the multi-container dev loop; A remains the durable/persistent-preview fallback.** | + +## Scaling: tunnel hub with self-registration and admin overview + +The two-container model generalizes to **one static hub fronting many dynamic dev +containers** — parallel agents on different features, or the several apps of one +distributed solution. Verified: a single `chisel server --reverse` fans in to multiple +clients, each on its own server-side port (`server:8001 → container A`, +`server:8002 → container B`), and `--authfile` (auto-reloaded on change) isolates them +per-agent. + +**Why chisel and not the others** (all tested this session): it installs via +`go install` (through `proxy.golang.org`, bypassing the GitHub gate), multiplexes +everything over a **single 443 connection** (no second data port — the wall that killed +localtunnel and cloudflared), and its TLS **passes the sandbox's mandatory MITM** +because it uses the system trust store, which already contains the agent CA (ngrok +failed only because it pins its own roots). The one hop not provable without a live +host: the WebSocket upgrade through Codespaces' `app.github.dev` proxy — a **VPS relay +removes that unknown** and is cleaner AUP-wise for a pure relay. + +### `mxcli tunnel-hub` (static) + `mxcli dev --hub` (dynamic) + +- **`mxcli tunnel-hub`** — the always-on static container. Embeds a chisel server + (`github.com/jpillora/chisel/server` as a library), a host-routing layer + (Caddy/Traefik or built-in), a **registration API**, and an **admin overview page**. + Runs once on a VPS (preferred) or a Codespace. It never changes when an app changes. +- **`mxcli dev --hub `** — the dynamic side self-registers on startup. + +Self-registration flow — chisel has **no native registration API**; this thin control +plane adds one on top of its two primitives (the auto-reloading `--authfile` and the +client-declared reverse remote): + +``` +mxcli dev --hub https://hub.example.com (on container startup) + → POST /register {app, feature, agent} → hub allocates {port, subdomain, token} + appends users.json (chisel reloads) + writes vhost subdomain→port (proxy reloads) + ← {port, subdomain, token} + → chisel client HUB R::localhost:8080 --auth + → set Mendix ApplicationRootUrl = https://.example.com + → heartbeat POST /status {health, last_reload, change_summary} (periodic) +on stop / TTL expiry → POST /deregister (frees the slot) +``` + +### Admin overview page + +The hub serves an authenticated dashboard — the single place to see and reach every +in-flight preview: + +- **One row per dev container:** agent/feature name, app, **clickable public URL**, + runtime health (from the heartbeat), last hot-reload time, uptime. +- **Pending changes per container:** each `mxcli dev` reports its delta — from + `mxcli diff-local` against the committed baseline, or the running list of MDL + statements applied this session — and the hub aggregates them, so a reviewer sees + *what each agent has changed* without opening each session. +- **Use cases:** a PM/reviewer clicks through parallel feature previews; comparing two + agents' takes on the same screen side by side; spotting a container that has drifted + or broken; a distributed-solution operator seeing all constituent apps and their + states at once. + +The change list makes the hub more than a router — it is a **live index of work in +progress across the fleet**, which is exactly the "keep overview" need. + +### Mendix wrinkles at fleet scale + +- **Subdomain, not path**, per container — `ApplicationRootUrl` = the container's own + subdomain. Path-prefix routing fights Mendix's SPA/XAS/cookies; subdomains give each + app a clean origin and free **cookie/session isolation** between agents. +- **DB per container** — each dev container needs its own Postgres/schema + demo data, + or agents overwrite each other. (Sandbox model: each has its own local Postgres.) +- **Slot assignment** — the hub owns the `{port, subdomain, token}` map via the + registration API; `--authfile` pins each agent to its allowed remote. +- **Distributed solution** — constituent apps register as separate containers on + separate subdomains and integrate via their public routes (published REST/OData), + faithfully previewing the real multi-app topology. ## Implementation Plan @@ -254,6 +328,11 @@ downloads/caches mxbuild + runtime and speaks the M2EE admin API. | `cmd/mxcli/docker/mxenv.go` | Reuse `PrepareMxCommand` (LD_PRELOAD FreeType fix) for the `mx`/`mxbuild` children (no change). | | `.devcontainer/preview/devcontainer.json` + `boot.sh` | New: Codespace preview container — `forwardPorts: [8080]` public, Postgres, `postStart` → `mxcli dev serve`. | | `.claude/skills/mendix/docker-workflow.md` | Add a "warm dev loop" section pointing at `mxcli dev`. | +| `cmd/mxcli/tunnelhub/server.go` | New: `mxcli tunnel-hub` — embedded chisel server (`chisel/server` lib), host routing, slot allocation, `--authfile` writer. | +| `cmd/mxcli/tunnelhub/register.go` | New: registration API (`/register`, `/status` heartbeat, `/deregister`); owns the `{port, subdomain, token}` map + TTL cleanup. | +| `cmd/mxcli/tunnelhub/admin.go` | New: authenticated admin overview page — fleet list (URL, health, last reload) + per-container change summaries. | +| `cmd/mxcli/dev.go` | Add `--hub `: self-register on startup, run the chisel client, set `ApplicationRootUrl` from the assigned subdomain, push `change_summary` heartbeats, deregister on stop. | +| `cmd/mxcli/tunnelhub/*_test.go` | Tests: slot allocation/isolation, authfile + vhost reload, register/heartbeat/deregister lifecycle, admin-page rendering. | | `cmd/mxcli/dev_test.go` | Tests for the serve client and the `restartRequired` branch logic (mock the serve/admin HTTP endpoints). | ### The standalone-boot config set (discovered, must be encoded) @@ -329,3 +408,17 @@ supply. `mxcli dev` must set all of it or `start` fails: 6. **Relationship to `check`.** The instant `mxcli check` gate (see `PROPOSAL_check_mxbuild_gap_heuristics.md`) should run *before* every warm build so most errors never reach even the ~0.8 s build. `mxcli dev` should call it first. +7. **Tunnel-hub auth & multi-tenancy.** The hub fronts multiple previews on one host — + who may view the admin page, and how are per-container URLs protected (per-subdomain + auth, shared secret, SSO)? Registration tokens must be scoped so one agent cannot + claim another's slot (`--authfile` allowed-remote regex per user). +8. **Change-summary source & cadence.** `mxcli diff-local` (git baseline) vs a running + MDL-statement log — which is the better per-container "changes" feed for the admin + page, and how often to push it (every reload vs periodic)? +9. **WebSocket through `app.github.dev` unverified.** chisel's reverse tunnel is proven + through the sandbox egress + MITM, but not through a live Codespace forward. Verify + on a real Codespace, or default the hub to a VPS (also the cleaner AUP posture for a + pure relay). +10. **Fleet lifecycle.** TTL/heartbeat-timeout cleanup of dead containers, port/subdomain + reclamation, and what the admin page shows for a container whose sandbox was reaped + (the ephemeral-runtime reality observed this session). From 2c4c1fccc12eeeb0a7e55234f3735887f8677dc0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 02:48:51 +0000 Subject: [PATCH 05/10] docs: add provisioning (new project -> running app, SessionStart bootstrap) How a user goes from a new Claude Code Web project to a running testable app: - entry points: mxcli new (new app) / clone + mxcli init (existing) - gap identified: init scaffolds a devcontainer + .claude tooling but no SessionStart hook; Claude Code Web reaps Postgres/JVM on idle, so it needs an idempotent `mxcli dev up` wired to SessionStart - dev up = setup mxcli/mxbuild/mxruntime (all exist) + Postgres + boot + serve - first-run vs warm cost table; heavy costs are one-time and image-cacheable - adds impl rows (dev up, init SessionStart hook) and an open question on image pre-baking + the SessionStart contract Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../PROPOSAL_mxcli_dev_warm_loop.md | 59 +++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index 011414800..a048bd9d9 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -237,6 +237,58 @@ mxcli dev push --to https://-9000.app.github.dev --secret $TOKEN -p app.mp | C. Git branch, Codespace pulls | ❌ | requires a commit — **rejected**, violates the core requirement | | D. Live tunnel (app stays in DEV, static relay forwards) | ✅ | **viable — chisel verified** (see § Scaling). Cleanest dev loop: hot-reload stays local, no artifact sync, and the relay is a generic static component. Caveat: the DEV runtime is ephemeral. **Preferred for the multi-container dev loop; A remains the durable/persistent-preview fallback.** | +## Provisioning: from new Claude Code Web project to a running testable app + +Two entry points, both on tooling that already exists: + +- **New app:** `mxcli new MyApp --version ` — downloads MxBuild, runs + `mx create-project`, scaffolds `.claude/` tooling + a devcontainer, and installs the + Linux mxcli binary. Optionally seed the blank model from a Claude design prototype + (mxcli/MDL `create entity` / `page` / `microflow`) per § End-to-end workflow. +- **Existing app:** point the Claude Code Web project at a repo containing the `.mpr`, + then `mxcli init` — adds `.claude/` skills/commands/CLAUDE.md and a devcontainer if + missing. + +### Initializing the Claude Code Web session + +`mxcli init` / `new` today scaffold a **devcontainer + `.claude/` tooling** — enough for +Codespaces and local dev containers, but Claude Code Web needs one more piece: a +**SessionStart hook** (or devcontainer `postStart`) that idempotently brings the runtime +up on **every** session, because background processes (Postgres, the JVM) are **reaped +on idle** — observed repeatedly during this investigation. Proposed: `mxcli init` also +emits an `mxcli dev up` bootstrap, wired to SessionStart, that idempotently: + +1. **Ensure mxcli** — prefer a prebuilt binary via `mxcli setup mxcli` (fast) over a + from-source build (needs antlr4 + Go, ~70 s). +2. **Ensure MxBuild + runtime cached** — `mxcli setup mxbuild -p app.mpr` and + `mxcli setup mxruntime -p app.mpr` (both already exist). One-time ~700 MB / ~30–40 s; + **bake into the devcontainer image** so the first session is instant. +3. **Start Postgres + create the app DB** — re-run every session (survives reaping). +4. **Boot the runtime + `mxbuild --serve`** — the standalone-boot recipe + (BasePath / RuntimePath / MicroflowConstants / data-dirs → `start` → DDL if needed), + leaving a warm serve daemon. + +End state: the session comes up **ready to prompt → build → test**. Then: + +- `mxcli dev` — local testable app at `localhost:8080` (the ~1 s warm loop). +- `mxcli dev --hub ` — externally testable: self-registers with the hub and gets a + public subdomain (§ Scaling). + +### First-run vs warm + +| Phase | One-time (first session / image build) | Per session (after reap) | +|-------|----------------------------------------|--------------------------| +| mxcli binary | `setup mxcli` download (or ~70 s build) | cached | +| MxBuild + runtime | ~700 MB, ~30–40 s | cached | +| Postgres + DB | — | start + createdb (~seconds) | +| runtime boot | — | ~40 s cold / seconds warm | + +The heavy costs are one-time and **image-cacheable**; steady-state session startup is +just Postgres + a runtime boot. The only real gap is the **SessionStart bootstrap** — +mxcli already has every ingredient (`new`, `init`, `setup mx*`, the standalone boot); +this proposal wires them into one idempotent `mxcli dev up` that Claude Code Web runs on +session start. + ## Scaling: tunnel hub with self-registration and admin overview The two-container model generalizes to **one static hub fronting many dynamic dev @@ -332,6 +384,8 @@ downloads/caches mxbuild + runtime and speaks the M2EE admin API. | `cmd/mxcli/tunnelhub/register.go` | New: registration API (`/register`, `/status` heartbeat, `/deregister`); owns the `{port, subdomain, token}` map + TTL cleanup. | | `cmd/mxcli/tunnelhub/admin.go` | New: authenticated admin overview page — fleet list (URL, health, last reload) + per-container change summaries. | | `cmd/mxcli/dev.go` | Add `--hub `: self-register on startup, run the chisel client, set `ApplicationRootUrl` from the assigned subdomain, push `change_summary` heartbeats, deregister on stop. | +| `cmd/mxcli/dev.go` (`dev up`) | Idempotent bootstrap for provisioning: `setup mxcli`/`mxbuild`/`mxruntime` if missing, start Postgres + create DB, boot runtime + `mxbuild --serve`. Safe to re-run every session (survives reaping). | +| `cmd/mxcli/init.go` | Emit a **SessionStart hook** (Claude Code Web) and/or devcontainer `postStart` that runs `mxcli dev up`; keep the existing devcontainer + `.claude/` scaffolding. | | `cmd/mxcli/tunnelhub/*_test.go` | Tests: slot allocation/isolation, authfile + vhost reload, register/heartbeat/deregister lifecycle, admin-page rendering. | | `cmd/mxcli/dev_test.go` | Tests for the serve client and the `restartRequired` branch logic (mock the serve/admin HTTP endpoints). | @@ -422,3 +476,8 @@ supply. `mxcli dev` must set all of it or `start` fails: 10. **Fleet lifecycle.** TTL/heartbeat-timeout cleanup of dead containers, port/subdomain reclamation, and what the admin page shows for a container whose sandbox was reaped (the ephemeral-runtime reality observed this session). +11. **Provisioning baseline.** Best mechanism to pre-bake MxBuild + runtime (~700 MB) + into the Claude Code Web image so first session is instant, vs downloading on first + `mxcli dev up`. And the exact SessionStart hook contract on Claude Code Web (does it + run on every resume, with what timeout?) — determines whether `dev up` must be fully + idempotent and how long it may take. From 27aede7c23d7a22609fe35421026eaa150da0ec3 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 03:27:45 +0000 Subject: [PATCH 06/10] docs: make prompt template the primary web/iPad provisioning path Restructure Provisioning into three entry points (prompt template primary for web/iPad, repo template, local CLI). Add the prompt-template subsection with: - the draft bootstrap prompt - rule 1: committing the config is mandatory (survives reaping; produces the repo-template end state without a maintained template) - rule 2: mxcli delivery is an environment-layer concern (github gate blocks release curl; go install works only for a public module) - proven via chisel Adds impl rows (new --emit-template, shipped prompt template, public-module/ env-preinstall delivery) and open question 12 on mxcli delivery into a gated empty repo. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../PROPOSAL_mxcli_dev_warm_loop.md | 62 ++++++++++++++++--- 1 file changed, 54 insertions(+), 8 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index a048bd9d9..17e5a0490 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -239,15 +239,52 @@ mxcli dev push --to https://-9000.app.github.dev --secret $TOKEN -p app.mp ## Provisioning: from new Claude Code Web project to a running testable app -Two entry points, both on tooling that already exists: +Three entry points, ordered by how little you need to start with: + +- **Prompt template (primary for web/iPad — starts from a *truly empty* repo).** Open an + empty repo in Claude Code Web and paste a fixed bootstrap prompt; the agent provisions + the project. No local CLI and no pre-built template repo to maintain — the agent runs + *current* mxcli and can seed the model from a Claude design prototype in the same + prompt. Detailed below. +- **Repo template ("Use this template" on GitHub).** A `mendix-mxcli-starter` marked as a + GitHub template → a new repo with the config already committed. Deterministic, but needs + per-version upkeep; a good fixed, reviewed baseline. Could be generated by + `mxcli new --emit-template` + CI per Mendix version so it is never hand-maintained. +- **Local CLI.** `mxcli new MyApp --version ` (new app: downloads MxBuild, runs + `mx create-project`, scaffolds `.claude/` + a devcontainer, installs the Linux mxcli + binary) or `mxcli init` on an existing repo with an `.mpr`. Requires a machine with the + CLI — **not available on an iPad**. + +### Prompt template (the web/iPad-native path) + +The agent turns an empty repo into a self-sufficient Mendix project in one session: -- **New app:** `mxcli new MyApp --version ` — downloads MxBuild, runs - `mx create-project`, scaffolds `.claude/` tooling + a devcontainer, and installs the - Linux mxcli binary. Optionally seed the blank model from a Claude design prototype - (mxcli/MDL `create entity` / `page` / `microflow`) per § End-to-end workflow. -- **Existing app:** point the Claude Code Web project at a repo containing the `.mpr`, - then `mxcli init` — adds `.claude/` skills/commands/CLAUDE.md and a devcontainer if - missing. +``` +This is an empty repo. Provision it as a Mendix app developed with mxcli: +1. Ensure `mxcli` is available (should be pre-installed by the environment; else + `go install github.com/mendixlabs/mxcli/cmd/mxcli@latest`). +2. `mxcli new App --version 11.6.3` at the repo root (or `mxcli init` if an .mpr exists). +3. Add `.claude/settings.json` with a SessionStart hook running `./mxcli dev up -p App.mpr`, + then COMMIT .mpr + .devcontainer/ + .claude/ so future sessions self-bootstrap. +4. `mxcli dev up -p App.mpr` — cache MxBuild+runtime, start Postgres, boot the runtime. +5. Confirm HTTP 200 locally and report. +(Optional) Seed the domain model, pages, and microflows from this prototype: . +``` + +Two rules make this robust: + +- **Committing the config (step 3) is mandatory.** The prompt is a *one-time seed*; its + output must be committed (`.mpr` + `.devcontainer/` + `.claude/` SessionStart hook) so + that after idle **reaping** the next session self-bootstraps from files, not from + re-prompting. Steady state is then file-driven and deterministic — the prompt template + *produces* the repo-template end state without a maintained template. +- **mxcli delivery is an *environment*-layer concern, not the prompt's.** Step 1 is the + fragile part: GitHub is gated in web sessions (a release-binary `curl` may be blocked), + and `go install` via `proxy.golang.org` works **only if mxcli is a public Go module** + (the mechanism was proven for chisel this session). Robust fix: the Claude Code Web + **environment setup script / image pre-installs mxcli** (and pre-caches MxBuild+runtime), + so the prompt only does logic (`new` → commit → `dev up`), never delivery. Publishing + mxcli as a public Go module is the fallback. ### Initializing the Claude Code Web session @@ -386,6 +423,9 @@ downloads/caches mxbuild + runtime and speaks the M2EE admin API. | `cmd/mxcli/dev.go` | Add `--hub `: self-register on startup, run the chisel client, set `ApplicationRootUrl` from the assigned subdomain, push `change_summary` heartbeats, deregister on stop. | | `cmd/mxcli/dev.go` (`dev up`) | Idempotent bootstrap for provisioning: `setup mxcli`/`mxbuild`/`mxruntime` if missing, start Postgres + create DB, boot runtime + `mxbuild --serve`. Safe to re-run every session (survives reaping). | | `cmd/mxcli/init.go` | Emit a **SessionStart hook** (Claude Code Web) and/or devcontainer `postStart` that runs `mxcli dev up`; keep the existing devcontainer + `.claude/` scaffolding. | +| `cmd/mxcli/new.go` | Add `--emit-template`: write a GitHub-template-ready repo (config + starter `.mpr`), for CI-per-version publishing of `mendix-mxcli-starter`. | +| `docs-site/.../bootstrap-prompt.md` | Ship the canonical **prompt template** (the web/iPad seed prompt) as documented, copy-pasteable text. | +| release / go.mod | Ensure mxcli is deliverable into a gated web session — **public Go module** (`go install`) and/or an **environment setup-script** that pre-installs it; do not rely on a GitHub release `curl` (may be gate-blocked). | | `cmd/mxcli/tunnelhub/*_test.go` | Tests: slot allocation/isolation, authfile + vhost reload, register/heartbeat/deregister lifecycle, admin-page rendering. | | `cmd/mxcli/dev_test.go` | Tests for the serve client and the `restartRequired` branch logic (mock the serve/admin HTTP endpoints). | @@ -481,3 +521,9 @@ supply. `mxcli dev` must set all of it or `start` fails: `mxcli dev up`. And the exact SessionStart hook contract on Claude Code Web (does it run on every resume, with what timeout?) — determines whether `dev up` must be fully idempotent and how long it may take. +12. **mxcli delivery into a gated empty repo (the prompt-template blocker).** GitHub is + gated in web sessions, so a release-binary `curl` may be blocked; `go install` via + `proxy.golang.org` only works if mxcli is a **public** module. Decide the canonical + delivery: publish a public Go module, pre-install mxcli in the environment image, or + both. This determines whether the prompt-template path works unassisted or must lean + on the environment layer. From a7f41e4d68b1b1ea9ca59fc146171477171df6bd Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 03:38:08 +0000 Subject: [PATCH 07/10] docs: add delivery-slices section to warm-loop proposal Keep one proposal but show how to ship it incrementally as four independent, value-ordered slices: (1) warm local loop, (2) provisioning, (3) single-app external preview, (4) tunnel hub. Recommends 1->2 first (solo dev, no external parts), 3 for single-app preview, 4 as a separately-reviewed scale-out. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../PROPOSAL_mxcli_dev_warm_loop.md | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md index 17e5a0490..9865ef09a 100644 --- a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -451,6 +451,29 @@ supply. `mxcli dev` must set all of it or `start` fails: running `execute_ddl_commands` first **only if** `get_ddl_commands > 0`. Do **not** couple "restart" to "DDL" — OQL view entities need a restart with zero DDL. +## Delivery slices + +This stays one proposal, but it should **ship as four independent slices** — each +valuable on its own, each a candidate for its own PR. Ordered by value-to-risk; each +builds on the previous. + +| # | Slice | Delivers | Depends on | Size / risk | +|---|-------|----------|------------|-------------| +| 1 | **Warm local loop** — `mxcli dev` (serve daemon + M2EE admin client + `restartRequired` × `get_ddl_commands` branching) | Docker-free ~1 s edit→test loop, locally | nothing new | medium / low — highest bang-for-buck | +| 2 | **Provisioning** — `mxcli dev up` bootstrap + `mxcli init` SessionStart hook + prompt template | a fresh Claude Code Web session comes up testable; iPad-native start | slice 1 | small / low | +| 3 | **Single-app external preview** — `mxcli dev serve` + chisel client → one static relay + `ApplicationRootUrl` wiring | a shareable live preview URL (the iPad two-container flow) | slice 1 | medium / medium — the `app.github.dev` WebSocket hop is unverified (a VPS relay avoids it) | +| 4 | **Tunnel hub** — `mxcli tunnel-hub` + `mxcli dev --hub` + registration API + admin overview | many dev containers behind one ingress; fleet overview + per-container change lists | slice 3 | large / higher — a product in its own right, with a multi-tenant auth surface | + +Recommended sequencing: **1 → 2** delivers the complete solo dev experience (Scenario A +plus provisioning) with no external moving parts, and can ship first. **3** adds external +preview for a single app. **4** is the scale-out and deserves its own design/security +review — build it only once 1–3 are proven. Slices 1–2 stand alone if the tunnel/hub work +is deferred indefinitely. + +Cross-cut: the instant `mxcli check` gate (`PROPOSAL_check_mxbuild_gap_heuristics.md`) +should front slice 1's build on every iteration, so most errors never reach even the +~0.8 s warm build. + ## Version Compatibility - `mxbuild --serve` and `reload_model`: **≥ 11.6.3** (verified on 11.6.3 and From 88252c5407d98bb4df8db308789f7b28450d7b52 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 03:50:50 +0000 Subject: [PATCH 08/10] docs: regenerate proposals README index (add warm-loop + 4 others) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ran scripts/gen_proposals_readme.py. Indexes 5 previously-missing active proposals: mxcli_dev_warm_loop, check_mxbuild_gap_heuristics, ai_capability_dataset, architecture_graph_visualization, playwright_session_reuse. Side effect, flagged: PROPOSAL_marketplace_modules (status: shipping) and UNIFIED_SCHEMA_REGISTRY (status: descoped) drop from the index because their statuses are non-canonical; the generator only buckets canonical statuses. They remain on disk and re-index once their statuses are normalised (maintainer's call — not changed here). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- docs/11-proposals/README.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/docs/11-proposals/README.md b/docs/11-proposals/README.md index 0ad4dd8cc..7f24ebe14 100644 --- a/docs/11-proposals/README.md +++ b/docs/11-proposals/README.md @@ -26,10 +26,10 @@ for display in this README): -## Active Proposals (78) +## Active Proposals (83) -### In Progress (partial) (8) +### In Progress (partial) (7) | Proposal | Status | Summary | |----------|--------|---------| @@ -38,11 +38,10 @@ for display in this README): | [Implement `mxcli structure` Command](mxcli-structure-proposal.md) | Partial | mxcli is a Go CLI tool for working with Mendix projects. | | [Multi-Version Support: Consolidated Architecture & Status](MULTI_VERSION_SUPPORT.md) | Partial | Mendix projects vary along three versioning axes, and mxcli must handle all of them correctly: | | [mxcli Linter - Implementation Proposal (Revised)](PROPOSAL_mxcli_linter.md) | Partial | Add an extensible linting framework to mxcli that leverages the existing SQLite-based catalog system. | -| [mxcli marketplace — Download & Manage Marketplace Modules](PROPOSAL_marketplace_modules.md) | Partial | After four rounds of spiking (scripts/auth-discovery-spike.sh), the | | [Navigation Support in MDL](navigation-support.md) | Partial | Every Mendix project has exactly one navigation$NavigationDocument — a project-level singleton containing navigation profiles (Responsive, P | | [Podman Support as Docker Alternative](PROPOSAL_podman_support.md) | Partial | Docker Desktop requires a paid subscription for larger organizations. | -### Proposed (36) +### Proposed (35) | Proposal | Status | Summary | |----------|--------|---------| @@ -77,22 +76,24 @@ for display in this README): | [SHOW/DESCRIBE Scheduled Events](show-describe-scheduled-events.md) | Proposed | Document type: ScheduledEvents$ScheduledEvent | | [SHOW/DESCRIBE Support for Missing Pluggable (React) Widgets](show-describe-pluggable-widgets.md) | Proposed | Scope: Improve DESCRIBE PAGE/SNIPPET output for pluggable widgets that currently fall through to generic formatting | | [Unified mxcli & MDL Documentation Site](proposal-documentation-site.md) | Proposed | mxcli has extensive documentation — 119 markdown files, a complete language specification, architecture docs, examples, and user guides — bu | -| [Unified Schema Registry](UNIFIED_SCHEMA_REGISTRY.md) | Descoped | Serialization core retired into engalar's roundtrip codec; survives as thin inspection/migration features over `gen/*` — see [backend strategy § Version handling](PROPOSAL_backend_strategy.md#version-handling--the-schema-registry-overlap-investigation-2026-06-05) | | [Version-Aware MDL and BSON Serialization](version-aware-mdl.md) | Proposed | The Mendix metamodel evolves across versions. | | [VS Code MDL Visualizations](PROPOSAL_vscode_visualizations.md) | Proposed | Add visual diagram previews to the VS Code MDL extension, enabling users to see entity-relationship diagrams, microflow flowcharts, page wir | | [VS Code Search — Quick Pick + Workspace Symbol](PROPOSAL_vscode_search.md) | Proposed | Full-text search exists in mxcli (mxcli search) but is only accessible via the terminal. | | [Workflow Improvements: ALTER WORKFLOW + Cross-References](PROPOSAL_workflow_improvements.md) | Proposed | Workflow support in mxcli has full CREATE/DESCRIBE/DROP/SHOW coverage with 13 activity types and BSON round-trip fidelity. | -### Draft (33) +### Draft (38) | Proposal | Status | Summary | |----------|--------|---------| | [Agent Document Type Support in MDL](PROPOSAL_agent_document_support.md) | Draft | Mendix 11.9 introduces Agents as a first-class concept for building agentic AI applications. | +| [Architecture Graph Visualization (communities, layers, god-nodes)](PROPOSAL_architecture_graph_visualization.md) | Draft | Issue: TBD (file before implementation) | | [Association Mapping in IMPORT](PROPOSAL_import_associations.md) | Draft | Parent: PROPOSAL_mxcli_sql.md (Phase 3 extension) | | [Backend Strategy — adopt engalar's modelsdk base + multi-backend (MCP first)](PROPOSAL_backend_strategy.md) | Draft | - Adopt engalar's modelsdk foundation as the base rather than merging 1109 | | [Bulk Change Custom Widget Properties](PROPOSAL_bulk_widget_property_updates.md) | Draft | Custom widgets (pluggable widgets) in Mendix have complex nested property structures. | | [Bulk External Action Support from OData Contracts](PROPOSAL_external_actions_bulk_create.md) | Draft | Issue #143 requests importing all entities and actions from a consumed OData service. | +| [check heuristics for constructs MxBuild rejects](PROPOSAL_check_mxbuild_gap_heuristics.md) | Draft | Relates to: PROPOSAL_expression_type_checking.md (shares the ModelResolver | | [Claude Code Prompt: Sprotty-based Domain Model Visualization PoC](proposal_sprotty_visualization.md) | Draft | I'm building a VS Code extension that visualizes Mendix Domain Models using Sprotty. | +| [Deterministic AI-Capability Dataset & Report Generator](PROPOSAL_ai_capability_dataset.md) | Draft | We publish an AI Capabilities report (ai-capabilities-report-.html) that | | [Entity Positioning in ALTER ENTITY](PROPOSAL_entity_position.md) | Draft | Entity positions in the domain model canvas can only be set during create entity via the @position annotation. | | [Expression Type Checking for mxcli check](PROPOSAL_expression_type_checking.md) | Draft | mxcli check is currently a syntactic validator only. | | [Extend UPDATE WIDGETS to Built-in Widgets via Schema Registry](PROPOSAL_update_builtin_widget_properties.md) | Draft | update widgets currently only works for pluggable widgets (ComboBox, DataGrid2, etc.) — it cannot modify properties on built-in widgets like | @@ -115,11 +116,13 @@ for display in this README): | [mxcli auth — Mendix Platform Authentication](PROPOSAL_platform_auth.md) | Draft | A growing set of mxcli features need to talk to Mendix platform APIs on behalf of the user: | | [mxcli catalog — Mendix Catalog Integration](PROPOSAL_catalog_integration.md) | Draft | ⚠️ TERMINOLOGY NOTE: This proposal covers the external Mendix Catalog service at catalog.mendix.com (CLI: mxcli catalog search), which is se | | [mxcli Playground](mxcli-playground.md) | Draft | A public GitHub repository (mendixlabs/mxcli-playground) containing a ready-to-use Mendix project pre-configured with mxcli, Claude Code ski | +| [Playwright Session Reuse and Lifecycle Control](PROPOSAL_playwright_session_reuse.md) | Draft | Builds on proposal-playwright-cli.md, which | | [Project Brain — Persistent Knowledge and Session Scaffolding for Long-Term AI Collaboration](PROPOSAL_project_brain.md) | Draft | mxcli is developed with significant AI involvement, yet the project's knowledge infrastructure is built for static human reading rather than | | [RENAME with Reference Refactoring](PROPOSAL_rename_refactoring.md) | Draft | Renaming entities, microflows, pages, and modules is one of the most common refactoring operations. | | [Replace Generated Playwright Tests with playwright-cli](proposal-playwright-cli.md) | Draft | The current approach (documented in proposal-playwright-testing.md) has Claude Code generate TypeScript test files (.spec.ts), then run them | | [Self-Describing Syntax Feature Registry](syntax-feature-registry.md) | Draft | Branch: research/recursive-help-discovery | | [Version-Aware Agent Support](PROPOSAL_version_aware_agent_support.md) | Draft | Three use cases require mxcli to be version-aware at the MDL level: | +| [warm dev loop — Docker-free run and iPad split-screen preview](PROPOSAL_mxcli_dev_warm_loop.md) | Draft | Relates to: PROPOSAL_check_mxbuild_gap_heuristics.md (the static-check gate that | ## Archived (24) From 2bfb306edf687ca3a7b349a6785f039305794bc3 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 03:53:11 +0000 Subject: [PATCH 09/10] docs: normalise two non-canonical proposal statuses, reindex README Lint category-2 fix (maintainer-approved) so both re-appear in the index: - PROPOSAL_marketplace_modules: status shipping -> partial (active); tidy date - UNIFIED_SCHEMA_REGISTRY: status descoped -> abandoned; git mv to archive/ (terminal status belongs in archive/; retired into engalar's codec) Regenerated README via scripts/gen_proposals_readme.py: marketplace now under In Progress, UNIFIED under Archived, warm-loop still indexed. No proposals dropped from the index. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- docs/11-proposals/PROPOSAL_marketplace_modules.md | 4 ++-- docs/11-proposals/README.md | 8 +++++--- .../11-proposals/{ => archive}/UNIFIED_SCHEMA_REGISTRY.md | 3 ++- 3 files changed, 9 insertions(+), 6 deletions(-) rename docs/11-proposals/{ => archive}/UNIFIED_SCHEMA_REGISTRY.md (99%) diff --git a/docs/11-proposals/PROPOSAL_marketplace_modules.md b/docs/11-proposals/PROPOSAL_marketplace_modules.md index 8d6897156..75c7d43d7 100644 --- a/docs/11-proposals/PROPOSAL_marketplace_modules.md +++ b/docs/11-proposals/PROPOSAL_marketplace_modules.md @@ -1,7 +1,7 @@ --- title: mxcli marketplace — Download & Manage Marketplace Modules -status: shipping -date: 2026-03-23 (initial), revised 2026-04-16 (spike), 2026-06-05 (download unblocked) +status: partial +date: 2026-06-05 --- # Proposal: `mxcli marketplace` — Download & Manage Marketplace Modules diff --git a/docs/11-proposals/README.md b/docs/11-proposals/README.md index 7f24ebe14..0cabcc190 100644 --- a/docs/11-proposals/README.md +++ b/docs/11-proposals/README.md @@ -26,10 +26,10 @@ for display in this README): -## Active Proposals (83) +## Active Proposals (82) -### In Progress (partial) (7) +### In Progress (partial) (8) | Proposal | Status | Summary | |----------|--------|---------| @@ -38,6 +38,7 @@ for display in this README): | [Implement `mxcli structure` Command](mxcli-structure-proposal.md) | Partial | mxcli is a Go CLI tool for working with Mendix projects. | | [Multi-Version Support: Consolidated Architecture & Status](MULTI_VERSION_SUPPORT.md) | Partial | Mendix projects vary along three versioning axes, and mxcli must handle all of them correctly: | | [mxcli Linter - Implementation Proposal (Revised)](PROPOSAL_mxcli_linter.md) | Partial | Add an extensible linting framework to mxcli that leverages the existing SQLite-based catalog system. | +| [mxcli marketplace — Download & Manage Marketplace Modules](PROPOSAL_marketplace_modules.md) | Partial | Unblocking path #1 below has happened: the content API now returns a | | [Navigation Support in MDL](navigation-support.md) | Partial | Every Mendix project has exactly one navigation$NavigationDocument — a project-level singleton containing navigation profiles (Responsive, P | | [Podman Support as Docker Alternative](PROPOSAL_podman_support.md) | Partial | Docker Desktop requires a paid subscription for larger organizations. | @@ -125,13 +126,14 @@ for display in this README): | [warm dev loop — Docker-free run and iPad split-screen preview](PROPOSAL_mxcli_dev_warm_loop.md) | Draft | Relates to: PROPOSAL_check_mxbuild_gap_heuristics.md (the static-check gate that | -## Archived (24) +## Archived (25) Terminal-status proposals. See [archive/](archive/). | Proposal | Status | Summary | |----------|--------|---------| +| [Unified Schema Registry](archive/UNIFIED_SCHEMA_REGISTRY.md) | Abandoned | Mendix project metadata (BSON document schemas, widget property structures, MDL keyword | | [Architecture Improvements for Agentic Development and Reduced Merge Conflicts](archive/PROPOSAL_agentic_architecture_improvements.md) | Done | Two related friction points have been identified: | | [Augment Widget Templates from .mpk at Runtime](archive/PROPOSAL_mpk_widget_augmentation.md) | Done | When mxcli creates pluggable widgets (ComboBox, DataGrid2, etc.), it uses static JSON templates extracted from a Mendix 11.6.0 project. | | [Business Events Support in mxcli](archive/PROPOSAL_business_events_support.md) | Done | Business Events is a Mendix feature for asynchronous event-driven integration, allowing applications to publish and subscribe to business ev | diff --git a/docs/11-proposals/UNIFIED_SCHEMA_REGISTRY.md b/docs/11-proposals/archive/UNIFIED_SCHEMA_REGISTRY.md similarity index 99% rename from docs/11-proposals/UNIFIED_SCHEMA_REGISTRY.md rename to docs/11-proposals/archive/UNIFIED_SCHEMA_REGISTRY.md index cfa2befac..28d19a891 100644 --- a/docs/11-proposals/UNIFIED_SCHEMA_REGISTRY.md +++ b/docs/11-proposals/archive/UNIFIED_SCHEMA_REGISTRY.md @@ -1,6 +1,7 @@ --- title: Unified Schema Registry -status: descoped +status: abandoned +date: 2026-06-05 --- # Proposal: Unified Schema Registry From 13f01686c5d02053a38724a3ac31101f0d8eb761 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 18 Jul 2026 04:20:25 +0000 Subject: [PATCH 10/10] docs: capture diagnostics-catalog research + commit extraction script Document the first-part research (mxcli check improvements) as a proposal: - where Mendix's diagnostics catalog lives (Mendix.Modeler.Texts.dll PO files) - structure: CODE__SUMMARY msgids, {param} templates, severity via CE/CW/CI prefix, LOCATION + MX_DOCS_CAT tags - findings: 1318 codes (11.12.1), static-checkability tiering, mxcli already reimplements ~77-81, mx check is same-floor as a build (Go check is the only speedup) - approach: versioned data asset, port by tier/frequency, harvest-against-oracle - verbatim examples (small fair-use sample) Commit scripts/extract-diagnostics-catalog.sh (the methodology). Its OUTPUT is Mendix's proprietary corpus and is NOT committed (see IP open question). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01JXnEgoc2NQP1Y2TWMCMXC4 --- .../PROPOSAL_check_diagnostics_catalog.md | 181 ++++++++++++++++++ docs/11-proposals/README.md | 5 +- scripts/extract-diagnostics-catalog.sh | 100 ++++++++++ 3 files changed, 284 insertions(+), 2 deletions(-) create mode 100644 docs/11-proposals/PROPOSAL_check_diagnostics_catalog.md create mode 100755 scripts/extract-diagnostics-catalog.sh diff --git a/docs/11-proposals/PROPOSAL_check_diagnostics_catalog.md b/docs/11-proposals/PROPOSAL_check_diagnostics_catalog.md new file mode 100644 index 000000000..05a87de61 --- /dev/null +++ b/docs/11-proposals/PROPOSAL_check_diagnostics_catalog.md @@ -0,0 +1,181 @@ +--- +title: mxcli check — mine Mendix's diagnostics catalog to close the check↔mxbuild gap +status: draft +date: 2026-07-18 +--- + +# Proposal: mine Mendix's diagnostics catalog for `mxcli check` + +**Status:** Draft +**Date:** 2026-07-18 +**Relates to:** `PROPOSAL_check_mxbuild_gap_heuristics.md` (one *tactical* slice of this — +6 specific constructs), `PROPOSAL_expression_type_checking.md` (Tier B enabler), +`PROPOSAL_mxcli_dev_warm_loop.md` (this catalog fronts the warm loop's build). + +## Problem Statement + +`mxcli check` reports "Check passed" for many constructs that the real MxBuild +(`docker build` / `mx check`) then rejects. Each miss costs a full build round-trip +(~30–60 s) to discover — and in the warm-dev-loop world (`PROPOSAL_mxcli_dev_warm_loop.md`) +it's the difference between an instant gate and a ~0.8 s build. The gap is currently +closed **reactively**, one rule at a time, whenever someone hits a build error. + +Mendix already ships the complete list of things it will reject: a **diagnostics catalog** +embedded in `Mendix.Modeler.Texts.dll`. This proposal mines it into a **target list** so +`check`/`lint` coverage becomes a measured, prioritised program instead of a reactive one. + +## BSON Structure + +Not applicable — this is a static-analysis/tooling proposal. It reads Mendix's build-tool +assemblies (design-time), not app `.mpr` documents. + +## Research findings (verified this session) + +**Where the catalog lives.** `Mendix.Modeler.Texts.dll` embeds GNU-gettext `.po` resources +named `problem-descriptions_.po` (en-US primary, plus deprecated/test variants and +5 translations). They *are* the catalog. + +**Format — already structured, not free text.** +- `msgid` = `CODE__SUMMARY`, e.g. `CE0001__ASSOCIATION_BETWEEN_PERSISTABLE_...`. +- `msgstr` = a **parameterised template** with named `{PLACEHOLDER}` tokens (and + pluralisation tokens like `{S_IF_PLURAL}`), not runtime-assembled fragments. +- **Severity is encoded in the prefix**: `CE` = Consistency Error, `CW` = Warning, + `CI` = Info. No inference needed. +- Each entry carries a `# LOCATION: .cs` comment and an optional + `# NOTE: MX_DOCS_CAT:` docs-deep-link tag. + +**Scale (Mendix 11.12.1, primary en-US PO):** **1318 codes** — 1265 `CE` + 53 `CW` +(`CI` codes live mostly in the deprecated/test POs); **727/1318 are parameterised**. +Present identically in 11.6.3 and 11.12.1. + +**Static-checkability tiering** (heuristic, by message + source subsystem): + +| Tier | Share | Nature | `mxcli check` reach | +|------|-------|--------|---------------------| +| A | ~60% (791) | structural / reference / required-property / naming | direct — model already parsed | +| B | ~26% (338) | expression / type-inference / XPath | needs a type-inference pass (`PROPOSAL_expression_type_checking.md`) | +| C | ~9% (120) | cross-document / datasource / client-compat | whole-model graph analysis | +| D | ~5% (69) | runtime / platform / tooling state | **not** statically checkable — skip | + +Subsystem split (by source file): Microflows 364, Domain model 275, Pages/Widgets 248, +Integration/REST/OData 129, Security 29, Workflows 15, Navigation 7. + +**mxcli already does ~6% of this by hand.** The source references **~77–81 distinct +`CE`/`CW` codes** across ~24 `validate_*.go` files and ~30 lint rules — proof the approach +works; the catalog just turns "port as we hit them" into an ordered backlog of ~1240 left. + +**There is no cheaper Mendix-native shortcut.** `mx check` (consistency only, no build) +measured **~22 s** — *slower* than an incremental build — because both cold-start the .NET +model engine + load the full `.mpr`. So mxcli's Go-native `check` is the **only** path under +the ~18–60 s floor. That is the entire justification for porting rules into Go rather than +shelling out to `mx check`. + +## Extraction methodology + +Reproducible, ~1 min, no Studio Pro. The **script is ours; its output is Mendix's corpus** +(see § Open questions on IP). Committed as `scripts/extract-diagnostics-catalog.sh`: + +```bash +# 1. stream just the one assembly from the Mendix CDN (no full mxbuild download) +curl -s "https://cdn.mendix.com/runtime/mxbuild-${VERSION}.tar.gz" \ + | tar -xz --occurrence=1 modeler/Mendix.Modeler.Texts.dll + +# 2. extract the embedded PO resources (needs mono-utils → monodis) +monodis --manifest Mendix.Modeler.Texts.dll # lists the .po resources +monodis --mresources Mendix.Modeler.Texts.dll # writes each resource to a file + +# 3. parse problem-descriptions_en-US.po into structured rows: +# code, prefix→severity, slug, message-template, {params}, source .cs, MX_DOCS_CAT +# then tier each by static-checkability. (Python parser in the script.) +``` + +`msgid` → `(prefix, number, slug)`; prefix → severity; `{...}` tokens → parameter list; +`# LOCATION:`/`# NOTE:` comments → source file + docs category. Output: `catalog.csv` / +`catalog.json` (local only). Re-run per Mendix version to refresh. + +## Representative examples (verbatim, small fair-use sample) + +``` +CE0001 error An association between a persistable entity and a non-persistable entity + must start in the non-persistable entity and have owner 'Default'. +CE0002 error Owner must be 'Default' for self-referential associations. +CE0006 error An XPath constraint on an access rule on {DESCRIPTION} is not allowed, + because it is not persistable. +CE0190 error Scheduled events of type Legacy are removed; right click for possible + remedies. [# NOTE: MX_DOCS_CAT:ScheduledEventMigration] +CE0529 error The selected page '{PAGE_NAME}' contains {A_IF_SINGULAR}required + parameter{S_IF_PLURAL} and can not be used as home page. +CE0572 error Widget{S_IF_PLURAL} {WIDGETS} {IS_OR_ARE} not supported in React client. +CW0001 warn Property 'Sort order' of the {CONTAINER} has no effect when a page is + used for selecting. +CW0040 warn Action activity that has a side effect on the client is not recommended + here because the microflow is used as a data source for + {CONTAINER_TYPE_AND_NAME}. +CI1561 info Path parameter 'x' does not appear in the path ''. +``` + +Plus the production-security rule this session actually hit and fixed with one `grant`: +*"At least one allowed role must be selected if the page is used from navigation, a +button, or a URL."* + +## Proposed approach + +1. **Ship the catalog as a versioned data asset** — `sdk/versions/diagnostics-{version}.json`, + produced by the extraction script per Mendix version. Lets `check`/`lint` emit + **Studio-Pro-identical codes + messages + severities**, a UX win even before new rules. + (Subject to the IP decision below — may ship code IDs + our own message text rather than + Mendix's verbatim strings.) +2. **Port by tier then frequency.** Tier A first (cheapest, highest hit-rate), Tier B behind + the type-inference work, Tier C case-by-case, Tier D never. Route `CE`→`check` (hard), + `CW`→`lint` (advisory) — severity is free from the prefix. +3. **Harvest-against-the-oracle loop.** Use `mx check` (~22 s, the full 1318-code truth) as + an **offline oracle** over a corpus of deliberately-broken projects; diff its findings + against `mxcli check`; the misses are the ranked backlog; port; re-run to confirm parity. + This is the build tool for the porting effort, and reuses the "run mxbuild over broken + projects and harvest output" fallback from the very first spike as a *deliberate* tool. +4. `PROPOSAL_check_mxbuild_gap_heuristics.md` is the first tactical instance; this proposal + is the strategic program it fits into. + +## Implementation Plan + +| File | Change | +|------|--------| +| `scripts/extract-diagnostics-catalog.sh` | New: the reproducible extractor (stream DLL → `monodis --mresources` → parse+tier). Committed here. | +| `sdk/versions/diagnostics-{version}.json` | Generated catalog data asset per version (IP decision pending). | +| `mdl/linter/rules/`, `mdl/executor/validate_*.go` | Port Tier-A codes, prioritised by frequency; cite the `CE`/`CW` code in each. | +| `mdl/executor/…` (oracle harness) | Offline `mx check` vs `mxcli check` parity report over a broken-project corpus → ranked backlog. | +| `mdl-examples/check-gap/` | Broken-project fixtures feeding the oracle harness. | + +## Version Compatibility + +- Codes are stable across 11.6.3 / 11.12.1; the extractor is version-parameterised, so the + catalog refreshes per Mendix release. Register generated assets alongside the existing + `sdk/versions/mendix-{9,10,11}.yaml`. +- New/removed codes between versions are themselves a useful signal (what Studio Pro started + or stopped checking). + +## Test Plan + +- Extractor: unit-test the PO parser against a small committed fixture PO (our own, not + Mendix's), asserting code/severity/params/location parsing. +- Oracle harness (gated, needs mxbuild): for each broken fixture, assert `mxcli check` now + reports the same code `mx check` does — parity regression per ported rule. +- Each ported rule: a `mdl-examples/check-gap/` fixture + a `validate_*`/lint test. + +## Open Questions + +1. **IP / licensing (blocking the data-asset shape).** The message *text* is Mendix's + proprietary copyrighted content extracted from their DLL. Committing all 1318 verbatim + into the repo is redistribution. Options: (a) ship only **code IDs + severity + source + + our own paraphrased messages**; (b) generate the full catalog **locally at build time** + from the user's own licensed mxbuild (never committed); (c) confirm redistribution is + permitted. Default: **(b)** — commit the extractor, not the corpus. +2. **`monodis` dependency.** The extractor needs `mono-utils`. Alternatives: `ilspycmd` + (dotnet tool) or a small pure-Go PE manifest-resource reader to drop the mono dep. +3. **Tiering is heuristic.** The A/B/C/D split is keyword+subsystem-based; the real + per-rule trigger predicate lives in Mendix's `.cs` (the `LOCATION` breadcrumb) and must + be reverse-engineered per rule. The catalog gives the *target + message*, not the + predicate — sizing must account for that. +4. **Overlap with `PROPOSAL_check_mxbuild_gap_heuristics.md`.** Fold that proposal's 6 cases + in as the first harvested batch, or keep it separate and cross-link? (Recommend: keep it + as the tactical pilot; this proposal owns the methodology + backlog.) diff --git a/docs/11-proposals/README.md b/docs/11-proposals/README.md index 0cabcc190..2b0df5589 100644 --- a/docs/11-proposals/README.md +++ b/docs/11-proposals/README.md @@ -26,7 +26,7 @@ for display in this README): -## Active Proposals (82) +## Active Proposals (83) ### In Progress (partial) (8) @@ -82,7 +82,7 @@ for display in this README): | [VS Code Search — Quick Pick + Workspace Symbol](PROPOSAL_vscode_search.md) | Proposed | Full-text search exists in mxcli (mxcli search) but is only accessible via the terminal. | | [Workflow Improvements: ALTER WORKFLOW + Cross-References](PROPOSAL_workflow_improvements.md) | Proposed | Workflow support in mxcli has full CREATE/DESCRIBE/DROP/SHOW coverage with 13 activity types and BSON round-trip fidelity. | -### Draft (38) +### Draft (39) | Proposal | Status | Summary | |----------|--------|---------| @@ -116,6 +116,7 @@ for display in this README): | [Multi-Project Tree View](PROPOSAL_multi_project_tree.md) | Draft | Mendix solutions increasingly span multiple applications: a frontend and a backend, or a microservices landscape (product catalog, orders, f | | [mxcli auth — Mendix Platform Authentication](PROPOSAL_platform_auth.md) | Draft | A growing set of mxcli features need to talk to Mendix platform APIs on behalf of the user: | | [mxcli catalog — Mendix Catalog Integration](PROPOSAL_catalog_integration.md) | Draft | ⚠️ TERMINOLOGY NOTE: This proposal covers the external Mendix Catalog service at catalog.mendix.com (CLI: mxcli catalog search), which is se | +| [mxcli check — mine Mendix's diagnostics catalog to close the check↔mxbuild gap](PROPOSAL_check_diagnostics_catalog.md) | Draft | Relates to: PROPOSAL_check_mxbuild_gap_heuristics.md (one tactical slice of this — | | [mxcli Playground](mxcli-playground.md) | Draft | A public GitHub repository (mendixlabs/mxcli-playground) containing a ready-to-use Mendix project pre-configured with mxcli, Claude Code ski | | [Playwright Session Reuse and Lifecycle Control](PROPOSAL_playwright_session_reuse.md) | Draft | Builds on proposal-playwright-cli.md, which | | [Project Brain — Persistent Knowledge and Session Scaffolding for Long-Term AI Collaboration](PROPOSAL_project_brain.md) | Draft | mxcli is developed with significant AI involvement, yet the project's knowledge infrastructure is built for static human reading rather than | diff --git a/scripts/extract-diagnostics-catalog.sh b/scripts/extract-diagnostics-catalog.sh new file mode 100755 index 000000000..4d5285dc6 --- /dev/null +++ b/scripts/extract-diagnostics-catalog.sh @@ -0,0 +1,100 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: Apache-2.0 +# +# extract-diagnostics-catalog.sh — mine Mendix's consistency-diagnostics catalog +# (CE/CW/CI codes) from Mendix.Modeler.Texts.dll into structured, tiered data. +# +# The SCRIPT is ours; its OUTPUT is Mendix's proprietary message corpus — do NOT +# commit the generated catalog.{csv,json}. See PROPOSAL_check_diagnostics_catalog.md. +# +# Usage: scripts/extract-diagnostics-catalog.sh [VERSION] [LOCALE] [OUTDIR] +# Example: scripts/extract-diagnostics-catalog.sh 11.12.1 en-US /tmp/diagcat +# Requires: curl, tar, monodis (apt-get install -y mono-utils), python3 +set -euo pipefail + +VERSION="${1:-11.12.1}" +LOCALE="${2:-en-US}" +OUTDIR="${3:-./diag-catalog-$VERSION}" +DLL="modeler/Mendix.Modeler.Texts.dll" +URL="https://cdn.mendix.com/runtime/mxbuild-${VERSION}.tar.gz" + +command -v monodis >/dev/null || { echo "ERROR: monodis not found (apt-get install -y mono-utils)" >&2; exit 1; } +mkdir -p "$OUTDIR"; cd "$OUTDIR" + +echo "[1/3] streaming $DLL from $URL ..." +# tar --occurrence=1 exits after the one member, closing the pipe early; curl then +# gets a (harmless) SIGPIPE/write-failure. Tolerate it and verify by file existence. +set +o pipefail +curl -fsSL "$URL" 2>/dev/null | tar -xz --occurrence=1 "$DLL" || true +set -o pipefail +[ -f "$DLL" ] || { echo "ERROR: failed to extract $DLL from $URL" >&2; exit 1; } + +echo "[2/3] extracting embedded PO resources via monodis ..." +mkdir -p po && ( cd po && monodis --mresources "../$DLL" >/dev/null 2>&1 ) +PO="$(ls po/*problem-descriptions_${LOCALE}.po | grep -vE 'deprecated|test' | head -1)" +[ -n "$PO" ] || { echo "ERROR: primary $LOCALE PO not found" >&2; exit 1; } + +echo "[3/3] parsing + tiering $PO ..." +python3 - "$PO" "$VERSION" <<'PY' +import re, json, csv, sys, collections +po, version = sys.argv[1], sys.argv[2] +SEV = {"CE": "error", "CW": "warning", "CI": "info"} +# static-checkability tiers (heuristic: message text + source subsystem) +D = ['version control','marketplace','app store','environment','deployment','license', + 'certificate','migration','team server','commit','branch','merge','governance'] +C = ['data source','react client','not supported in','offline','native','reachable', + 'end event','unreachable','navigation','home page','loop'] +B = ['xpath','expression','type of','of type','cannot be converted','incompatible', + 'return type','data type','parameter','must return','regular expression','template'] +def tier(blob): + if any(k in blob for k in D): return 'D' + if any(k in blob for k in C): return 'C' + if any(k in blob for k in B): return 'B' + return 'A' + +code_re = re.compile(r'^(C[EWI])(\d+)__(.*)$') +rows, loc = [], [] +lines = open(po, encoding='utf-8').read().splitlines() +i = 0 +while i < len(lines): + ln = lines[i] + if ln.startswith('# LOCATION:'): + loc.append(re.sub(r'\s*\(in project.*', '', ln.split('LOCATION:', 1)[1]).strip()) + elif ln.startswith('# NOTE:') and 'MX_DOCS_CAT' in ln: + docs = ln.split('MX_DOCS_CAT', 1)[1].lstrip(':(colon) ').strip() + elif ln.startswith('msgid "'): + mid = ln[7:-1] + j = i + 1 + while j < len(lines) and not lines[j].startswith('msgstr'): j += 1 + msg = '' + if j < len(lines): + msg = lines[j][8:].rstrip('"') + k = j + 1 + while k < len(lines) and lines[k].startswith('"'): + msg += lines[k].strip().strip('"'); k += 1 + m = code_re.match(mid) + if m: + params = sorted(set(re.findall(r'\{([A-Z0-9_]+)\}', msg))) + rows.append(dict(code=m.group(1)+m.group(2), severity=SEV[m.group(1)], + slug=m.group(3), message=msg, params=params, + locations=sorted(set(loc)), + docs_cat=locals().get('docs'), + tier=tier((m.group(3) + ' ' + msg + ' ' + ' '.join(loc)).lower()))) + loc, docs = [], None + i += 1 + +json.dump({"version": version, "codes": rows}, open("catalog.json", "w"), indent=1) +with open("catalog.csv", "w", newline='') as f: + w = csv.writer(f); w.writerow(["code","severity","tier","message","params","source","docs_cat"]) + for r in rows: + w.writerow([r['code'], r['severity'], r['tier'], r['message'], + ';'.join(r['params']), ';'.join(r['locations']), r['docs_cat'] or '']) + +sev = collections.Counter(r['severity'] for r in rows) +tiers = collections.Counter(r['tier'] for r in rows) +print(f" codes: {len(rows)} ({dict(sev)})") +print(f" tiers: {dict(sorted(tiers.items()))} (A=structural B=expr/type C=cross-doc D=runtime)") +print(f" parameterised: {sum(1 for r in rows if r['params'])}/{len(rows)}") +print(f" wrote catalog.json + catalog.csv (LOCAL ONLY — Mendix's corpus, do not commit)") +PY +echo "Done: $OUTDIR/catalog.{json,csv}"