diff --git a/docs/11-proposals/PROPOSAL_check_diagnostics_catalog.md b/docs/11-proposals/PROPOSAL_check_diagnostics_catalog.md new file mode 100644 index 000000000..05a87de61 --- /dev/null +++ b/docs/11-proposals/PROPOSAL_check_diagnostics_catalog.md @@ -0,0 +1,181 @@ +--- +title: mxcli check — mine Mendix's diagnostics catalog to close the check↔mxbuild gap +status: draft +date: 2026-07-18 +--- + +# Proposal: mine Mendix's diagnostics catalog for `mxcli check` + +**Status:** Draft +**Date:** 2026-07-18 +**Relates to:** `PROPOSAL_check_mxbuild_gap_heuristics.md` (one *tactical* slice of this — +6 specific constructs), `PROPOSAL_expression_type_checking.md` (Tier B enabler), +`PROPOSAL_mxcli_dev_warm_loop.md` (this catalog fronts the warm loop's build). + +## Problem Statement + +`mxcli check` reports "Check passed" for many constructs that the real MxBuild +(`docker build` / `mx check`) then rejects. Each miss costs a full build round-trip +(~30–60 s) to discover — and in the warm-dev-loop world (`PROPOSAL_mxcli_dev_warm_loop.md`) +it's the difference between an instant gate and a ~0.8 s build. The gap is currently +closed **reactively**, one rule at a time, whenever someone hits a build error. + +Mendix already ships the complete list of things it will reject: a **diagnostics catalog** +embedded in `Mendix.Modeler.Texts.dll`. This proposal mines it into a **target list** so +`check`/`lint` coverage becomes a measured, prioritised program instead of a reactive one. + +## BSON Structure + +Not applicable — this is a static-analysis/tooling proposal. It reads Mendix's build-tool +assemblies (design-time), not app `.mpr` documents. + +## Research findings (verified this session) + +**Where the catalog lives.** `Mendix.Modeler.Texts.dll` embeds GNU-gettext `.po` resources +named `problem-descriptions_.po` (en-US primary, plus deprecated/test variants and +5 translations). They *are* the catalog. + +**Format — already structured, not free text.** +- `msgid` = `CODE__SUMMARY`, e.g. `CE0001__ASSOCIATION_BETWEEN_PERSISTABLE_...`. +- `msgstr` = a **parameterised template** with named `{PLACEHOLDER}` tokens (and + pluralisation tokens like `{S_IF_PLURAL}`), not runtime-assembled fragments. +- **Severity is encoded in the prefix**: `CE` = Consistency Error, `CW` = Warning, + `CI` = Info. No inference needed. +- Each entry carries a `# LOCATION: .cs` comment and an optional + `# NOTE: MX_DOCS_CAT:` docs-deep-link tag. + +**Scale (Mendix 11.12.1, primary en-US PO):** **1318 codes** — 1265 `CE` + 53 `CW` +(`CI` codes live mostly in the deprecated/test POs); **727/1318 are parameterised**. +Present identically in 11.6.3 and 11.12.1. + +**Static-checkability tiering** (heuristic, by message + source subsystem): + +| Tier | Share | Nature | `mxcli check` reach | +|------|-------|--------|---------------------| +| A | ~60% (791) | structural / reference / required-property / naming | direct — model already parsed | +| B | ~26% (338) | expression / type-inference / XPath | needs a type-inference pass (`PROPOSAL_expression_type_checking.md`) | +| C | ~9% (120) | cross-document / datasource / client-compat | whole-model graph analysis | +| D | ~5% (69) | runtime / platform / tooling state | **not** statically checkable — skip | + +Subsystem split (by source file): Microflows 364, Domain model 275, Pages/Widgets 248, +Integration/REST/OData 129, Security 29, Workflows 15, Navigation 7. + +**mxcli already does ~6% of this by hand.** The source references **~77–81 distinct +`CE`/`CW` codes** across ~24 `validate_*.go` files and ~30 lint rules — proof the approach +works; the catalog just turns "port as we hit them" into an ordered backlog of ~1240 left. + +**There is no cheaper Mendix-native shortcut.** `mx check` (consistency only, no build) +measured **~22 s** — *slower* than an incremental build — because both cold-start the .NET +model engine + load the full `.mpr`. So mxcli's Go-native `check` is the **only** path under +the ~18–60 s floor. That is the entire justification for porting rules into Go rather than +shelling out to `mx check`. + +## Extraction methodology + +Reproducible, ~1 min, no Studio Pro. The **script is ours; its output is Mendix's corpus** +(see § Open questions on IP). Committed as `scripts/extract-diagnostics-catalog.sh`: + +```bash +# 1. stream just the one assembly from the Mendix CDN (no full mxbuild download) +curl -s "https://cdn.mendix.com/runtime/mxbuild-${VERSION}.tar.gz" \ + | tar -xz --occurrence=1 modeler/Mendix.Modeler.Texts.dll + +# 2. extract the embedded PO resources (needs mono-utils → monodis) +monodis --manifest Mendix.Modeler.Texts.dll # lists the .po resources +monodis --mresources Mendix.Modeler.Texts.dll # writes each resource to a file + +# 3. parse problem-descriptions_en-US.po into structured rows: +# code, prefix→severity, slug, message-template, {params}, source .cs, MX_DOCS_CAT +# then tier each by static-checkability. (Python parser in the script.) +``` + +`msgid` → `(prefix, number, slug)`; prefix → severity; `{...}` tokens → parameter list; +`# LOCATION:`/`# NOTE:` comments → source file + docs category. Output: `catalog.csv` / +`catalog.json` (local only). Re-run per Mendix version to refresh. + +## Representative examples (verbatim, small fair-use sample) + +``` +CE0001 error An association between a persistable entity and a non-persistable entity + must start in the non-persistable entity and have owner 'Default'. +CE0002 error Owner must be 'Default' for self-referential associations. +CE0006 error An XPath constraint on an access rule on {DESCRIPTION} is not allowed, + because it is not persistable. +CE0190 error Scheduled events of type Legacy are removed; right click for possible + remedies. [# NOTE: MX_DOCS_CAT:ScheduledEventMigration] +CE0529 error The selected page '{PAGE_NAME}' contains {A_IF_SINGULAR}required + parameter{S_IF_PLURAL} and can not be used as home page. +CE0572 error Widget{S_IF_PLURAL} {WIDGETS} {IS_OR_ARE} not supported in React client. +CW0001 warn Property 'Sort order' of the {CONTAINER} has no effect when a page is + used for selecting. +CW0040 warn Action activity that has a side effect on the client is not recommended + here because the microflow is used as a data source for + {CONTAINER_TYPE_AND_NAME}. +CI1561 info Path parameter 'x' does not appear in the path ''. +``` + +Plus the production-security rule this session actually hit and fixed with one `grant`: +*"At least one allowed role must be selected if the page is used from navigation, a +button, or a URL."* + +## Proposed approach + +1. **Ship the catalog as a versioned data asset** — `sdk/versions/diagnostics-{version}.json`, + produced by the extraction script per Mendix version. Lets `check`/`lint` emit + **Studio-Pro-identical codes + messages + severities**, a UX win even before new rules. + (Subject to the IP decision below — may ship code IDs + our own message text rather than + Mendix's verbatim strings.) +2. **Port by tier then frequency.** Tier A first (cheapest, highest hit-rate), Tier B behind + the type-inference work, Tier C case-by-case, Tier D never. Route `CE`→`check` (hard), + `CW`→`lint` (advisory) — severity is free from the prefix. +3. **Harvest-against-the-oracle loop.** Use `mx check` (~22 s, the full 1318-code truth) as + an **offline oracle** over a corpus of deliberately-broken projects; diff its findings + against `mxcli check`; the misses are the ranked backlog; port; re-run to confirm parity. + This is the build tool for the porting effort, and reuses the "run mxbuild over broken + projects and harvest output" fallback from the very first spike as a *deliberate* tool. +4. `PROPOSAL_check_mxbuild_gap_heuristics.md` is the first tactical instance; this proposal + is the strategic program it fits into. + +## Implementation Plan + +| File | Change | +|------|--------| +| `scripts/extract-diagnostics-catalog.sh` | New: the reproducible extractor (stream DLL → `monodis --mresources` → parse+tier). Committed here. | +| `sdk/versions/diagnostics-{version}.json` | Generated catalog data asset per version (IP decision pending). | +| `mdl/linter/rules/`, `mdl/executor/validate_*.go` | Port Tier-A codes, prioritised by frequency; cite the `CE`/`CW` code in each. | +| `mdl/executor/…` (oracle harness) | Offline `mx check` vs `mxcli check` parity report over a broken-project corpus → ranked backlog. | +| `mdl-examples/check-gap/` | Broken-project fixtures feeding the oracle harness. | + +## Version Compatibility + +- Codes are stable across 11.6.3 / 11.12.1; the extractor is version-parameterised, so the + catalog refreshes per Mendix release. Register generated assets alongside the existing + `sdk/versions/mendix-{9,10,11}.yaml`. +- New/removed codes between versions are themselves a useful signal (what Studio Pro started + or stopped checking). + +## Test Plan + +- Extractor: unit-test the PO parser against a small committed fixture PO (our own, not + Mendix's), asserting code/severity/params/location parsing. +- Oracle harness (gated, needs mxbuild): for each broken fixture, assert `mxcli check` now + reports the same code `mx check` does — parity regression per ported rule. +- Each ported rule: a `mdl-examples/check-gap/` fixture + a `validate_*`/lint test. + +## Open Questions + +1. **IP / licensing (blocking the data-asset shape).** The message *text* is Mendix's + proprietary copyrighted content extracted from their DLL. Committing all 1318 verbatim + into the repo is redistribution. Options: (a) ship only **code IDs + severity + source + + our own paraphrased messages**; (b) generate the full catalog **locally at build time** + from the user's own licensed mxbuild (never committed); (c) confirm redistribution is + permitted. Default: **(b)** — commit the extractor, not the corpus. +2. **`monodis` dependency.** The extractor needs `mono-utils`. Alternatives: `ilspycmd` + (dotnet tool) or a small pure-Go PE manifest-resource reader to drop the mono dep. +3. **Tiering is heuristic.** The A/B/C/D split is keyword+subsystem-based; the real + per-rule trigger predicate lives in Mendix's `.cs` (the `LOCATION` breadcrumb) and must + be reverse-engineered per rule. The catalog gives the *target + message*, not the + predicate — sizing must account for that. +4. **Overlap with `PROPOSAL_check_mxbuild_gap_heuristics.md`.** Fold that proposal's 6 cases + in as the first harvested batch, or keep it separate and cross-link? (Recommend: keep it + as the tactical pilot; this proposal owns the methodology + backlog.) diff --git a/docs/11-proposals/PROPOSAL_marketplace_modules.md b/docs/11-proposals/PROPOSAL_marketplace_modules.md index 8d6897156..75c7d43d7 100644 --- a/docs/11-proposals/PROPOSAL_marketplace_modules.md +++ b/docs/11-proposals/PROPOSAL_marketplace_modules.md @@ -1,7 +1,7 @@ --- title: mxcli marketplace — Download & Manage Marketplace Modules -status: shipping -date: 2026-03-23 (initial), revised 2026-04-16 (spike), 2026-06-05 (download unblocked) +status: partial +date: 2026-06-05 --- # Proposal: `mxcli marketplace` — Download & Manage Marketplace Modules diff --git a/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md new file mode 100644 index 000000000..9865ef09a --- /dev/null +++ b/docs/11-proposals/PROPOSAL_mxcli_dev_warm_loop.md @@ -0,0 +1,552 @@ +--- +title: warm dev loop — Docker-free run and iPad split-screen preview +status: draft +date: 2026-07-17 +--- + +# Proposal: Warm dev loop (`mxbuild --serve` + `reload_model`) + +**Status:** Draft +**Date:** 2026-07-17 +**Relates to:** `PROPOSAL_check_mxbuild_gap_heuristics.md` (the static-check gate that +runs *before* this loop ever builds), `.claude/skills/mendix/docker-workflow.md` +(the loop this replaces for inner-loop iteration). + +## Problem Statement + +The current way to run and test a Mendix app with mxcli is `mxcli docker build` / +`mxcli docker run`. It has two costs that dominate the edit→test loop: + +1. **Full build every time.** The one-shot `mxbuild` CLI recompiles the whole + deployment on each invocation. Measured on a one-entity/one-page/one-microflow + app: **~30–60 s per build** (Java compile + full model export + package). +2. **Docker.** The runtime and database run in containers, adding image pulls and + container lifecycle to every cycle — and Docker is **unavailable** in important + contexts: Claude Code on the web, and iPad. In those environments the current + loop cannot run at all. + +Two primitives, both verified to exist and work in Mendix **11.6.3+**, make a far +tighter loop possible: + +- **`mxbuild --serve`** — an HTTP build server (default port 6543) that keeps the + model loaded and rebuilds **incrementally**, skipping unchanged work (including + Java compilation when no Java changed). Each build returns `restartRequired`. +- **`reload_model`** — a runtime admin-port action that swaps the model into the + **running** JVM in-place, draining in-flight actions first (near-zero-downtime). + No process restart. + +This proposal wires those two primitives into mxcli to serve **two scenarios**: + +- **Scenario A — a fast, Docker-free `mxcli dev` loop** to replace `mxcli docker` + for inner-loop iteration. +- **Scenario B — an iPad split-screen workflow**: Claude Code on the web in one + pane, a browser preview in the other, prompt → build → test, all on the iPad, + **without committing to the repo** before testing. + +## End-to-end workflow: Claude design prototype → secured production Mendix app (on iPad) + +Scenario B is one leg of a larger journey this loop is meant to unlock — taking an +idea all the way to a **secured, production-grade Mendix app entirely on an iPad**, +with no desktop Studio Pro in the loop: + +1. **Prototype in Claude.** A Claude design Artifact (a clickable HTML/React mockup) + captures the app's screens, data, and flows — fast and throwaway, no Mendix yet. +2. **Translate to Mendix in Claude Code (web).** In the same iPad session, mxcli/MDL + turns the prototype into a real Mendix model: `create entity` / `create page` / + `create microflow` / navigation. The prototype's structure becomes actual `.mpr` + documents — a running Mendix app, not a mockup. +3. **Secure it for production.** Raise the app from prototype to production security + and make it pass the real Mendix consistency rules — the exact path proven during + the investigation behind this proposal: + - `ALTER PROJECT SECURITY LEVEL PRODUCTION` — required before `DTAPMode=P` will + even boot (otherwise `start` returns `result:11 — security must be + CHECKEVERYTHING`). + - `grant` / `revoke` page, entity, and microflow access per module role; define + user roles and their mappings; wire the login and anonymous navigation profiles. + - `mxcli check` (instant) plus the warm `mxbuild` build surface the + production-only errors up front (e.g. *"page needs an allowed role"*, which we + hit and fixed with a single `grant`) instead of discovering them at deploy time. +4. **Test each change live, on the iPad.** The warm loop (§Proposed CLI) previews + every edit in ~1 s in the Safari pane, under **real production security** (login + enforced, anonymous blocked) — so the author iterates prototype → model → secured + app → test without ever leaving the tablet. + +The deliverable is not a throwaway preview but a **secured, production-representative +Mendix app** — the same artifact you would deploy. The warm dev loop is what makes +step 4 fast enough for this to be a genuine authoring experience rather than a batch +process, and Scenario B is what makes it work on a device that has neither Docker nor +Studio Pro. + +## BSON Structure + +Not applicable — this is a build/runtime orchestration feature. It touches no +Mendix document serialization. It drives two already-existing interfaces: the +mxbuild HTTP build API and the runtime M2EE admin API. + +## Measured evidence + +All numbers below were measured directly (Mendix 11.6.3, Linux x86-64, JDK 21) on +a one-entity/one-page/one-microflow app during the investigation that motivated +this proposal. + +| Step | Time | Notes | +|------|------|-------| +| `mxbuild` one-shot CLI (full) | **~30–60 s** | current loop; no incremental mode | +| `mxbuild --serve`, cold (first build) | ~13.7 s | loads model into the server | +| `mxbuild --serve`, warm, **no change** | ~1.1 s | model cached | +| `mxbuild --serve`, warm, **microflow change** | **~0.8 s** | Java **not** recompiled | +| `mxbuild --serve`, warm, **entity/attribute change** | ~7.3 s | `restartRequired: true` | +| `reload_model` (hot, no restart) | **~0.07–0.27 s** | `"reload": true`, JVM pid unchanged | +| **Full warm loop** (exec → serve build → reload) | **~1 s** | microflow/page change, no restart | + +`restartRequired` from the serve API cleanly separates the two change classes: + +- **page / microflow change** → `restartRequired: false` → `reload_model` (~0.1 s) +- **entity / domain change** → `restartRequired: true` → `execute_ddl_commands` + + runtime restart (the DDL gate is real and unavoidable — the runtime refuses a + stale schema) + +`mxbuild --serve` and `reload_model` are present in both **11.6.3** and **11.12.1** +(verified). The 11.12 CLI adds no build-skip flags (only `--export-secrets`), so the +incremental path is `--serve`, not a new one-shot flag. + +## Hot-reload scope: what `reload_model` can and cannot do (verified) + +`reload_model` reloads exactly three subsystems — the model store +(`ProjectModelStoreLoader.load`), the microflow engine +(`MicroflowEngineModule.reload`), and translations (`I18NProcessor.reload`) — after +draining in-flight actions. It does **not** touch the datastorage / entity layer. + +The runtime maintains a **metamodel catalog inside the app database** — +`mendixsystem$entity`, `mendixsystem$entityidentifier`, `mendixsystem$attribute`, +`mendixsystem$association` — and reconciles it **at startup**, not during a reload. +This was verified directly: a view entity added via `reload_model` (`TaskView`) +returned `Success` but **never appeared in `mendixsystem$entity`**, while the base +`Task` (present at the last startup) has its row (`entity_name` → `table_name` → a +stable identifier GUID). So any change that needs a catalog row — a new/changed +entity, **including OQL view entities**, and associations in +`mendixsystem$association` (e.g. a view entity referencing a persistent entity) — +requires a **restart**, because that is when the catalog is reconciled. + +This makes the apply-decision **two-dimensional**: `restartRequired` (from the serve +build) and `get_ddl_commands` (from the runtime) are **independent** signals. + +| Change class | `restartRequired` | `get_ddl_commands` | Apply via | +|---|---|---|---| +| microflow / page / text | `false` | 0 | `reload_model` (~0.1 s, hot) | +| **OQL view entity** | **`true`** | **0** | **restart** — catalog sync, **no DDL** | +| persistable entity / attribute | `true` | > 0 | `execute_ddl_commands` → restart | +| association (adds FK) | `true` | > 0 | `execute_ddl_commands` → restart | + +The loop must branch on **both** signals: `restartRequired == false` → `reload_model`; +`restartRequired == true` → restart, running `execute_ddl_commands` **only if** +`get_ddl_commands > 0`. Treating "restart" as implying "DDL" is wrong — **OQL view +entities are the counterexample**: they need a restart (to write the +`mendixsystem$entity` catalog row) but no DDL (they are query-time, not materialized — +confirmed: no view/table for `TaskView` in Postgres). + +**Caveat (not fully closed):** `reload_model` *returns* `Success` for a view-entity +change (it doesn't reject it), and I could not run the functional tiebreaker — +querying `TaskView` post-reload — because the standalone hand-boot does not register +the `preview_execute_oql` admin action (it needs the dev-preview flag `mxcli docker` +sets). The conclusion rests on the authoritative `restartRequired` signal plus the +catalog evidence, which agree. Enabling the dev-preview flag on the standalone boot +(so the agent can verify via OQL) is itself a small item worth doing — see Open +Questions. + +## Proposed CLI + +### Scenario A: `mxcli dev` — Docker-free warm run loop + +A long-lived local dev supervisor. Boots (or reuses) the standalone runtime + +Postgres, starts `mxbuild --serve`, and on each model change runs +`serve build (Deploy)` → branch on `restartRequired` → `reload_model` or +DDL+restart. + +```bash +# start a warm dev session (no Docker) +mxcli dev -p app.mpr +# → downloads/caches mxbuild + runtime (once), starts Postgres, +# boots the runtime, starts `mxbuild --serve`, prints the local URL, +# and holds everything warm. + +# apply a change and hot-reload (from another shell, or driven by the agent) +mxcli dev reload -p app.mpr # serve-build + restartRequired-aware reload +mxcli dev exec change.mdl -p app.mpr # exec MDL, then reload in one step +mxcli dev status # runtime status, warm-build server health +mxcli dev stop +``` + +`mxcli dev` **subsumes the inner loop of `mxcli docker run`** while keeping +`mxcli docker` for produced-artifact / container-parity builds. + +Optionally a `--watch` mode: watch the `.mpr` (v2 `mprcontents/`) for changes and +auto-run the reload cycle. + +### Scenario B: `mxcli dev serve` + a preview container (iPad) + +The iPad has two panes: **Claude Code on the web** (prompt + edit) and **Safari** +(the running app). The catch, established during investigation: the Claude Code web +sandbox is **egress-only and cannot expose a port** (all reverse tunnels are blocked +by a 443-only + TLS-intercepting egress policy). So the *running* app must live in a +container that has real public ingress — a **Codespace** (public port forwarding → +`https://-8080.app.github.dev`) is the natural fit. + +Two containers, coupled **without a commit** (the explicit requirement — preview WIP +before committing): + +``` +┌─────────────────────────┐ push built deployment ┌──────────────────────────┐ +│ DEV: Claude Code Web │ ── (443, authenticated, no git) ──▶ │ PREVIEW: Codespace │ +│ edits app.mpr │ │ runtime + mxbuild --serve │ +│ mxbuild --serve (build) │ │ Postgres │ +│ │ ◀── agent verifies via public URL ── │ :8080 forwarded PUBLIC │ +└─────────────────────────┘ (curl / Playwright) └──────────────────────────┘ + Safari tab ▲ (iPad) +``` + +- **DEV** (Claude Code Web): prompt Claude → `mxcli` edits `app.mpr` → + `mxbuild --serve` produces the Deploy artifact → `mxcli dev push` uploads it to the + Preview container over 443. +- **PREVIEW** (Codespace): receives the artifact, runs `reload_model` (or DDL+restart + on `restartRequired`), serves the public URL. Booted by a `devcontainer.json` + + `postStart` script that reproduces the standalone-boot recipe. +- **iPad UX:** prompt in the Claude pane → ~1–2 s → refresh the Safari tab → see the + change. Nothing is committed until the user is happy. + +Proposed commands: + +```bash +# in the Preview container (Codespace), started by the devcontainer: +mxcli dev serve -p app.mpr --intake-port 9000 --intake-secret $TOKEN +# → boots runtime + Postgres + mxbuild --serve, forwards 8080 public, +# exposes an authenticated /deploy intake for artifact pushes. + +# in the DEV container (Claude Code Web): +mxcli dev push --to https://-9000.app.github.dev --secret $TOKEN -p app.mpr +# → mxbuild --serve (Deploy) locally, then upload the deployment delta; +# the Preview container reloads and returns the live status. +``` + +### Coupling: options considered + +| Option | Preview WIP w/o commit? | Notes | +|--------|-------------------------|-------| +| **A. Push built deployment artifact over 443** (recommended) | ✅ | robust; runtime in a legit host; ~1–2 s + network | +| B. Push MDL/model, build in the Preview container | ✅ | keeps a synced `.mpr` in the Preview container; heavier transfer for MPR v2 | +| C. Git branch, Codespace pulls | ❌ | requires a commit — **rejected**, violates the core requirement | +| D. Live tunnel (app stays in DEV, static relay forwards) | ✅ | **viable — chisel verified** (see § Scaling). Cleanest dev loop: hot-reload stays local, no artifact sync, and the relay is a generic static component. Caveat: the DEV runtime is ephemeral. **Preferred for the multi-container dev loop; A remains the durable/persistent-preview fallback.** | + +## Provisioning: from new Claude Code Web project to a running testable app + +Three entry points, ordered by how little you need to start with: + +- **Prompt template (primary for web/iPad — starts from a *truly empty* repo).** Open an + empty repo in Claude Code Web and paste a fixed bootstrap prompt; the agent provisions + the project. No local CLI and no pre-built template repo to maintain — the agent runs + *current* mxcli and can seed the model from a Claude design prototype in the same + prompt. Detailed below. +- **Repo template ("Use this template" on GitHub).** A `mendix-mxcli-starter` marked as a + GitHub template → a new repo with the config already committed. Deterministic, but needs + per-version upkeep; a good fixed, reviewed baseline. Could be generated by + `mxcli new --emit-template` + CI per Mendix version so it is never hand-maintained. +- **Local CLI.** `mxcli new MyApp --version ` (new app: downloads MxBuild, runs + `mx create-project`, scaffolds `.claude/` + a devcontainer, installs the Linux mxcli + binary) or `mxcli init` on an existing repo with an `.mpr`. Requires a machine with the + CLI — **not available on an iPad**. + +### Prompt template (the web/iPad-native path) + +The agent turns an empty repo into a self-sufficient Mendix project in one session: + +``` +This is an empty repo. Provision it as a Mendix app developed with mxcli: +1. Ensure `mxcli` is available (should be pre-installed by the environment; else + `go install github.com/mendixlabs/mxcli/cmd/mxcli@latest`). +2. `mxcli new App --version 11.6.3` at the repo root (or `mxcli init` if an .mpr exists). +3. Add `.claude/settings.json` with a SessionStart hook running `./mxcli dev up -p App.mpr`, + then COMMIT .mpr + .devcontainer/ + .claude/ so future sessions self-bootstrap. +4. `mxcli dev up -p App.mpr` — cache MxBuild+runtime, start Postgres, boot the runtime. +5. Confirm HTTP 200 locally and report. +(Optional) Seed the domain model, pages, and microflows from this prototype: . +``` + +Two rules make this robust: + +- **Committing the config (step 3) is mandatory.** The prompt is a *one-time seed*; its + output must be committed (`.mpr` + `.devcontainer/` + `.claude/` SessionStart hook) so + that after idle **reaping** the next session self-bootstraps from files, not from + re-prompting. Steady state is then file-driven and deterministic — the prompt template + *produces* the repo-template end state without a maintained template. +- **mxcli delivery is an *environment*-layer concern, not the prompt's.** Step 1 is the + fragile part: GitHub is gated in web sessions (a release-binary `curl` may be blocked), + and `go install` via `proxy.golang.org` works **only if mxcli is a public Go module** + (the mechanism was proven for chisel this session). Robust fix: the Claude Code Web + **environment setup script / image pre-installs mxcli** (and pre-caches MxBuild+runtime), + so the prompt only does logic (`new` → commit → `dev up`), never delivery. Publishing + mxcli as a public Go module is the fallback. + +### Initializing the Claude Code Web session + +`mxcli init` / `new` today scaffold a **devcontainer + `.claude/` tooling** — enough for +Codespaces and local dev containers, but Claude Code Web needs one more piece: a +**SessionStart hook** (or devcontainer `postStart`) that idempotently brings the runtime +up on **every** session, because background processes (Postgres, the JVM) are **reaped +on idle** — observed repeatedly during this investigation. Proposed: `mxcli init` also +emits an `mxcli dev up` bootstrap, wired to SessionStart, that idempotently: + +1. **Ensure mxcli** — prefer a prebuilt binary via `mxcli setup mxcli` (fast) over a + from-source build (needs antlr4 + Go, ~70 s). +2. **Ensure MxBuild + runtime cached** — `mxcli setup mxbuild -p app.mpr` and + `mxcli setup mxruntime -p app.mpr` (both already exist). One-time ~700 MB / ~30–40 s; + **bake into the devcontainer image** so the first session is instant. +3. **Start Postgres + create the app DB** — re-run every session (survives reaping). +4. **Boot the runtime + `mxbuild --serve`** — the standalone-boot recipe + (BasePath / RuntimePath / MicroflowConstants / data-dirs → `start` → DDL if needed), + leaving a warm serve daemon. + +End state: the session comes up **ready to prompt → build → test**. Then: + +- `mxcli dev` — local testable app at `localhost:8080` (the ~1 s warm loop). +- `mxcli dev --hub ` — externally testable: self-registers with the hub and gets a + public subdomain (§ Scaling). + +### First-run vs warm + +| Phase | One-time (first session / image build) | Per session (after reap) | +|-------|----------------------------------------|--------------------------| +| mxcli binary | `setup mxcli` download (or ~70 s build) | cached | +| MxBuild + runtime | ~700 MB, ~30–40 s | cached | +| Postgres + DB | — | start + createdb (~seconds) | +| runtime boot | — | ~40 s cold / seconds warm | + +The heavy costs are one-time and **image-cacheable**; steady-state session startup is +just Postgres + a runtime boot. The only real gap is the **SessionStart bootstrap** — +mxcli already has every ingredient (`new`, `init`, `setup mx*`, the standalone boot); +this proposal wires them into one idempotent `mxcli dev up` that Claude Code Web runs on +session start. + +## Scaling: tunnel hub with self-registration and admin overview + +The two-container model generalizes to **one static hub fronting many dynamic dev +containers** — parallel agents on different features, or the several apps of one +distributed solution. Verified: a single `chisel server --reverse` fans in to multiple +clients, each on its own server-side port (`server:8001 → container A`, +`server:8002 → container B`), and `--authfile` (auto-reloaded on change) isolates them +per-agent. + +**Why chisel and not the others** (all tested this session): it installs via +`go install` (through `proxy.golang.org`, bypassing the GitHub gate), multiplexes +everything over a **single 443 connection** (no second data port — the wall that killed +localtunnel and cloudflared), and its TLS **passes the sandbox's mandatory MITM** +because it uses the system trust store, which already contains the agent CA (ngrok +failed only because it pins its own roots). The one hop not provable without a live +host: the WebSocket upgrade through Codespaces' `app.github.dev` proxy — a **VPS relay +removes that unknown** and is cleaner AUP-wise for a pure relay. + +### `mxcli tunnel-hub` (static) + `mxcli dev --hub` (dynamic) + +- **`mxcli tunnel-hub`** — the always-on static container. Embeds a chisel server + (`github.com/jpillora/chisel/server` as a library), a host-routing layer + (Caddy/Traefik or built-in), a **registration API**, and an **admin overview page**. + Runs once on a VPS (preferred) or a Codespace. It never changes when an app changes. +- **`mxcli dev --hub `** — the dynamic side self-registers on startup. + +Self-registration flow — chisel has **no native registration API**; this thin control +plane adds one on top of its two primitives (the auto-reloading `--authfile` and the +client-declared reverse remote): + +``` +mxcli dev --hub https://hub.example.com (on container startup) + → POST /register {app, feature, agent} → hub allocates {port, subdomain, token} + appends users.json (chisel reloads) + writes vhost subdomain→port (proxy reloads) + ← {port, subdomain, token} + → chisel client HUB R::localhost:8080 --auth + → set Mendix ApplicationRootUrl = https://.example.com + → heartbeat POST /status {health, last_reload, change_summary} (periodic) +on stop / TTL expiry → POST /deregister (frees the slot) +``` + +### Admin overview page + +The hub serves an authenticated dashboard — the single place to see and reach every +in-flight preview: + +- **One row per dev container:** agent/feature name, app, **clickable public URL**, + runtime health (from the heartbeat), last hot-reload time, uptime. +- **Pending changes per container:** each `mxcli dev` reports its delta — from + `mxcli diff-local` against the committed baseline, or the running list of MDL + statements applied this session — and the hub aggregates them, so a reviewer sees + *what each agent has changed* without opening each session. +- **Use cases:** a PM/reviewer clicks through parallel feature previews; comparing two + agents' takes on the same screen side by side; spotting a container that has drifted + or broken; a distributed-solution operator seeing all constituent apps and their + states at once. + +The change list makes the hub more than a router — it is a **live index of work in +progress across the fleet**, which is exactly the "keep overview" need. + +### Mendix wrinkles at fleet scale + +- **Subdomain, not path**, per container — `ApplicationRootUrl` = the container's own + subdomain. Path-prefix routing fights Mendix's SPA/XAS/cookies; subdomains give each + app a clean origin and free **cookie/session isolation** between agents. +- **DB per container** — each dev container needs its own Postgres/schema + demo data, + or agents overwrite each other. (Sandbox model: each has its own local Postgres.) +- **Slot assignment** — the hub owns the `{port, subdomain, token}` map via the + registration API; `--authfile` pins each agent to its allowed remote. +- **Distributed solution** — constituent apps register as separate containers on + separate subdomains and integrate via their public routes (published REST/OData), + faithfully previewing the real multi-app topology. + +## Implementation Plan + +Reuse the existing `cmd/mxcli/docker/` plumbing wherever possible — it already +downloads/caches mxbuild + runtime and speaks the M2EE admin API. + +### Files to modify/create + +| File | Change | +|------|--------| +| `cmd/mxcli/dev.go` | New `mxcli dev` command tree (`dev`, `dev reload`, `dev exec`, `dev serve`, `dev push`, `dev status`, `dev stop`). | +| `cmd/mxcli/docker/mxserve.go` | New: `mxbuild --serve` client — start the daemon, `POST /build {target:Deploy}`, parse `status` + `restartRequired`. | +| `cmd/mxcli/docker/admin.go` | Extract the M2EE admin client (auth header `X-M2EE-Authentication: base64(pass)`, actions `update_appcontainer_configuration`, `update_configuration`, `start`, `execute_ddl_commands`, **`reload_model`**, `runtime_status`) out of `docker/oql.go` into a reusable client. | +| `cmd/mxcli/docker/localboot.go` | New: standalone runtime boot (see the config-set below), so `dev` can boot without Docker. | +| `cmd/mxcli/docker/download.go` | Reuse `DownloadMxBuild` / `DownloadRuntime` (no change). | +| `cmd/mxcli/docker/mxenv.go` | Reuse `PrepareMxCommand` (LD_PRELOAD FreeType fix) for the `mx`/`mxbuild` children (no change). | +| `.devcontainer/preview/devcontainer.json` + `boot.sh` | New: Codespace preview container — `forwardPorts: [8080]` public, Postgres, `postStart` → `mxcli dev serve`. | +| `.claude/skills/mendix/docker-workflow.md` | Add a "warm dev loop" section pointing at `mxcli dev`. | +| `cmd/mxcli/tunnelhub/server.go` | New: `mxcli tunnel-hub` — embedded chisel server (`chisel/server` lib), host routing, slot allocation, `--authfile` writer. | +| `cmd/mxcli/tunnelhub/register.go` | New: registration API (`/register`, `/status` heartbeat, `/deregister`); owns the `{port, subdomain, token}` map + TTL cleanup. | +| `cmd/mxcli/tunnelhub/admin.go` | New: authenticated admin overview page — fleet list (URL, health, last reload) + per-container change summaries. | +| `cmd/mxcli/dev.go` | Add `--hub `: self-register on startup, run the chisel client, set `ApplicationRootUrl` from the assigned subdomain, push `change_summary` heartbeats, deregister on stop. | +| `cmd/mxcli/dev.go` (`dev up`) | Idempotent bootstrap for provisioning: `setup mxcli`/`mxbuild`/`mxruntime` if missing, start Postgres + create DB, boot runtime + `mxbuild --serve`. Safe to re-run every session (survives reaping). | +| `cmd/mxcli/init.go` | Emit a **SessionStart hook** (Claude Code Web) and/or devcontainer `postStart` that runs `mxcli dev up`; keep the existing devcontainer + `.claude/` scaffolding. | +| `cmd/mxcli/new.go` | Add `--emit-template`: write a GitHub-template-ready repo (config + starter `.mpr`), for CI-per-version publishing of `mendix-mxcli-starter`. | +| `docs-site/.../bootstrap-prompt.md` | Ship the canonical **prompt template** (the web/iPad seed prompt) as documented, copy-pasteable text. | +| release / go.mod | Ensure mxcli is deliverable into a gated web session — **public Go module** (`go install`) and/or an **environment setup-script** that pre-installs it; do not rely on a GitHub release `curl` (may be gate-blocked). | +| `cmd/mxcli/tunnelhub/*_test.go` | Tests: slot allocation/isolation, authfile + vhost reload, register/heartbeat/deregister lifecycle, admin-page rendering. | +| `cmd/mxcli/dev_test.go` | Tests for the serve client and the `restartRequired` branch logic (mock the serve/admin HTTP endpoints). | + +### The standalone-boot config set (discovered, must be encoded) + +The standalone runtime launcher (`com.mendix.container.boot.Main`, driven via the +admin API) requires a specific config that Studio Pro / the buildpack normally +supply. `mxcli dev` must set all of it or `start` fails: + +- Env: `MX_INSTALL_PATH`, `M2EE_ADMIN_PASS`, `M2EE_ADMIN_PORT`, and the FreeType + `LD_PRELOAD` on the JVM. +- Ensure the `data/{files,tmp,model-upload}` dirs exist under the deployment. +- `update_configuration`: `BasePath` (deployment dir), `RuntimePath` + (`/runtime`), `DatabaseType/Host/Name/UserName/Password` + (`DatabaseHost` includes the port, e.g. `127.0.0.1:5432`), `DTAPMode`, and + **`MicroflowConstants`** (design-time default values are *not* auto-applied + standalone — missing a constant surfaces at runtime as HTTP 530 → login bounce). +- Boot sequence: `update_appcontainer_configuration` (runtime port) → + `update_configuration` → `start` → on `"database has to be updated"`: + `execute_ddl_commands` → `start`. +- Reload cycle (two independent signals — see § Hot-reload scope): `mxbuild --serve` + Deploy build → if `restartRequired == false` then `reload_model`; else restart, + running `execute_ddl_commands` first **only if** `get_ddl_commands > 0`. Do **not** + couple "restart" to "DDL" — OQL view entities need a restart with zero DDL. + +## Delivery slices + +This stays one proposal, but it should **ship as four independent slices** — each +valuable on its own, each a candidate for its own PR. Ordered by value-to-risk; each +builds on the previous. + +| # | Slice | Delivers | Depends on | Size / risk | +|---|-------|----------|------------|-------------| +| 1 | **Warm local loop** — `mxcli dev` (serve daemon + M2EE admin client + `restartRequired` × `get_ddl_commands` branching) | Docker-free ~1 s edit→test loop, locally | nothing new | medium / low — highest bang-for-buck | +| 2 | **Provisioning** — `mxcli dev up` bootstrap + `mxcli init` SessionStart hook + prompt template | a fresh Claude Code Web session comes up testable; iPad-native start | slice 1 | small / low | +| 3 | **Single-app external preview** — `mxcli dev serve` + chisel client → one static relay + `ApplicationRootUrl` wiring | a shareable live preview URL (the iPad two-container flow) | slice 1 | medium / medium — the `app.github.dev` WebSocket hop is unverified (a VPS relay avoids it) | +| 4 | **Tunnel hub** — `mxcli tunnel-hub` + `mxcli dev --hub` + registration API + admin overview | many dev containers behind one ingress; fleet overview + per-container change lists | slice 3 | large / higher — a product in its own right, with a multi-tenant auth surface | + +Recommended sequencing: **1 → 2** delivers the complete solo dev experience (Scenario A +plus provisioning) with no external moving parts, and can ship first. **3** adds external +preview for a single app. **4** is the scale-out and deserves its own design/security +review — build it only once 1–3 are proven. Slices 1–2 stand alone if the tunnel/hub work +is deferred indefinitely. + +Cross-cut: the instant `mxcli check` gate (`PROPOSAL_check_mxbuild_gap_heuristics.md`) +should front slice 1's build on every iteration, so most errors never reach even the +~0.8 s warm build. + +## Version Compatibility + +- `mxbuild --serve` and `reload_model`: **≥ 11.6.3** (verified on 11.6.3 and + 11.12.1). No new flag needed on 11.12 — the incremental path is `--serve` in both. +- Register the feature in `sdk/versions/mendix-11.yaml` (and 10 if we verify the + admin action name `reload_model` there) with the correct `min_version`, and add a + `checkFeature()` pre-check with an actionable hint. +- **Production-representative testing note:** running `DTAPMode=P` requires the app's + security level to be `CHECKEVERYTHING` (Production) or `start` refuses + (`result:11`). `mxcli dev` should default to `D` for the inner loop and expose a + `--dtap P` flag; when `P` is set, surface the security-level requirement up front. + +## Test Plan + +- `cmd/mxcli/docker/mxserve_test.go` — mock the serve HTTP endpoint; assert request + shape (`{target:Deploy, projectFilePath}`) and `restartRequired` parsing. +- `cmd/mxcli/docker/admin_test.go` — mock the admin endpoint; assert the boot + sequence and the `reload_model` vs DDL+restart branch. +- Integration (gated, needs mxbuild + runtime + Postgres, like the existing docker + tests): boot a fixture app, do a microflow change → assert warm reload with pid + unchanged; do an entity change → assert `restartRequired` → DDL path. +- `mdl-examples/` — a small fixture app the dev-loop tests drive. + +## Open Questions + +1. **Process lifecycle.** Long-lived `mxbuild --serve` and the runtime JVM need + supervision (restart on crash, port cleanup). PID files under `.mxcli/`? A + `dev status`/`dev stop`? (In the sandbox, background JVMs were reaped on idle — + the Codespace/preview host must keep them alive; document the idle-stop caveat.) +2. **Artifact transfer for Scenario B.** Push the whole Deploy `model/` each time, or + compute a delta? MPR v2 is a folder; the *deployment* `model/` is more compact, + but still worth measuring. What's the auth model for the intake endpoint (bearer + secret over the public forward)? +3. **Codespaces AUP.** Running your own app for preview is intended use; the + Codespace-as-relay variant (Option D) is a gray area. The proposal recommends A + (artifact push) precisely to stay clearly on the "previewing your own project" + side. Confirm this framing. +4. **Warm-serve memory.** `mxbuild --serve` holds the model in memory; quantify for + large apps and document guidance. +5. **`reload_model` scope (largely resolved — see § Hot-reload scope).** Verified: + microflow changes hot-reload; entity/view-entity/association changes flip + `restartRequired` because the runtime reconciles its DB metamodel catalog + (`mendixsystem$entity` / `entityidentifier` / `attribute` / `association`) at + startup, not on reload. Open sub-items: (a) enable the `preview_execute_oql` + dev flag on the standalone boot so the loop can *functionally* verify a change via + OQL (needed to close the view-entity caveat and to power agentic verification); + (b) exercise styling-only changes (`update_styling` is a distinct admin action — + likely a third hot path); (c) multi-unit and mixed changes. +6. **Relationship to `check`.** The instant `mxcli check` gate (see + `PROPOSAL_check_mxbuild_gap_heuristics.md`) should run *before* every warm build so + most errors never reach even the ~0.8 s build. `mxcli dev` should call it first. +7. **Tunnel-hub auth & multi-tenancy.** The hub fronts multiple previews on one host — + who may view the admin page, and how are per-container URLs protected (per-subdomain + auth, shared secret, SSO)? Registration tokens must be scoped so one agent cannot + claim another's slot (`--authfile` allowed-remote regex per user). +8. **Change-summary source & cadence.** `mxcli diff-local` (git baseline) vs a running + MDL-statement log — which is the better per-container "changes" feed for the admin + page, and how often to push it (every reload vs periodic)? +9. **WebSocket through `app.github.dev` unverified.** chisel's reverse tunnel is proven + through the sandbox egress + MITM, but not through a live Codespace forward. Verify + on a real Codespace, or default the hub to a VPS (also the cleaner AUP posture for a + pure relay). +10. **Fleet lifecycle.** TTL/heartbeat-timeout cleanup of dead containers, port/subdomain + reclamation, and what the admin page shows for a container whose sandbox was reaped + (the ephemeral-runtime reality observed this session). +11. **Provisioning baseline.** Best mechanism to pre-bake MxBuild + runtime (~700 MB) + into the Claude Code Web image so first session is instant, vs downloading on first + `mxcli dev up`. And the exact SessionStart hook contract on Claude Code Web (does it + run on every resume, with what timeout?) — determines whether `dev up` must be fully + idempotent and how long it may take. +12. **mxcli delivery into a gated empty repo (the prompt-template blocker).** GitHub is + gated in web sessions, so a release-binary `curl` may be blocked; `go install` via + `proxy.golang.org` only works if mxcli is a **public** module. Decide the canonical + delivery: publish a public Go module, pre-install mxcli in the environment image, or + both. This determines whether the prompt-template path works unassisted or must lean + on the environment layer. diff --git a/docs/11-proposals/README.md b/docs/11-proposals/README.md index 0ad4dd8cc..2b0df5589 100644 --- a/docs/11-proposals/README.md +++ b/docs/11-proposals/README.md @@ -26,7 +26,7 @@ for display in this README): -## Active Proposals (78) +## Active Proposals (83) ### In Progress (partial) (8) @@ -38,11 +38,11 @@ for display in this README): | [Implement `mxcli structure` Command](mxcli-structure-proposal.md) | Partial | mxcli is a Go CLI tool for working with Mendix projects. | | [Multi-Version Support: Consolidated Architecture & Status](MULTI_VERSION_SUPPORT.md) | Partial | Mendix projects vary along three versioning axes, and mxcli must handle all of them correctly: | | [mxcli Linter - Implementation Proposal (Revised)](PROPOSAL_mxcli_linter.md) | Partial | Add an extensible linting framework to mxcli that leverages the existing SQLite-based catalog system. | -| [mxcli marketplace — Download & Manage Marketplace Modules](PROPOSAL_marketplace_modules.md) | Partial | After four rounds of spiking (scripts/auth-discovery-spike.sh), the | +| [mxcli marketplace — Download & Manage Marketplace Modules](PROPOSAL_marketplace_modules.md) | Partial | Unblocking path #1 below has happened: the content API now returns a | | [Navigation Support in MDL](navigation-support.md) | Partial | Every Mendix project has exactly one navigation$NavigationDocument — a project-level singleton containing navigation profiles (Responsive, P | | [Podman Support as Docker Alternative](PROPOSAL_podman_support.md) | Partial | Docker Desktop requires a paid subscription for larger organizations. | -### Proposed (36) +### Proposed (35) | Proposal | Status | Summary | |----------|--------|---------| @@ -77,22 +77,24 @@ for display in this README): | [SHOW/DESCRIBE Scheduled Events](show-describe-scheduled-events.md) | Proposed | Document type: ScheduledEvents$ScheduledEvent | | [SHOW/DESCRIBE Support for Missing Pluggable (React) Widgets](show-describe-pluggable-widgets.md) | Proposed | Scope: Improve DESCRIBE PAGE/SNIPPET output for pluggable widgets that currently fall through to generic formatting | | [Unified mxcli & MDL Documentation Site](proposal-documentation-site.md) | Proposed | mxcli has extensive documentation — 119 markdown files, a complete language specification, architecture docs, examples, and user guides — bu | -| [Unified Schema Registry](UNIFIED_SCHEMA_REGISTRY.md) | Descoped | Serialization core retired into engalar's roundtrip codec; survives as thin inspection/migration features over `gen/*` — see [backend strategy § Version handling](PROPOSAL_backend_strategy.md#version-handling--the-schema-registry-overlap-investigation-2026-06-05) | | [Version-Aware MDL and BSON Serialization](version-aware-mdl.md) | Proposed | The Mendix metamodel evolves across versions. | | [VS Code MDL Visualizations](PROPOSAL_vscode_visualizations.md) | Proposed | Add visual diagram previews to the VS Code MDL extension, enabling users to see entity-relationship diagrams, microflow flowcharts, page wir | | [VS Code Search — Quick Pick + Workspace Symbol](PROPOSAL_vscode_search.md) | Proposed | Full-text search exists in mxcli (mxcli search) but is only accessible via the terminal. | | [Workflow Improvements: ALTER WORKFLOW + Cross-References](PROPOSAL_workflow_improvements.md) | Proposed | Workflow support in mxcli has full CREATE/DESCRIBE/DROP/SHOW coverage with 13 activity types and BSON round-trip fidelity. | -### Draft (33) +### Draft (39) | Proposal | Status | Summary | |----------|--------|---------| | [Agent Document Type Support in MDL](PROPOSAL_agent_document_support.md) | Draft | Mendix 11.9 introduces Agents as a first-class concept for building agentic AI applications. | +| [Architecture Graph Visualization (communities, layers, god-nodes)](PROPOSAL_architecture_graph_visualization.md) | Draft | Issue: TBD (file before implementation) | | [Association Mapping in IMPORT](PROPOSAL_import_associations.md) | Draft | Parent: PROPOSAL_mxcli_sql.md (Phase 3 extension) | | [Backend Strategy — adopt engalar's modelsdk base + multi-backend (MCP first)](PROPOSAL_backend_strategy.md) | Draft | - Adopt engalar's modelsdk foundation as the base rather than merging 1109 | | [Bulk Change Custom Widget Properties](PROPOSAL_bulk_widget_property_updates.md) | Draft | Custom widgets (pluggable widgets) in Mendix have complex nested property structures. | | [Bulk External Action Support from OData Contracts](PROPOSAL_external_actions_bulk_create.md) | Draft | Issue #143 requests importing all entities and actions from a consumed OData service. | +| [check heuristics for constructs MxBuild rejects](PROPOSAL_check_mxbuild_gap_heuristics.md) | Draft | Relates to: PROPOSAL_expression_type_checking.md (shares the ModelResolver | | [Claude Code Prompt: Sprotty-based Domain Model Visualization PoC](proposal_sprotty_visualization.md) | Draft | I'm building a VS Code extension that visualizes Mendix Domain Models using Sprotty. | +| [Deterministic AI-Capability Dataset & Report Generator](PROPOSAL_ai_capability_dataset.md) | Draft | We publish an AI Capabilities report (ai-capabilities-report-.html) that | | [Entity Positioning in ALTER ENTITY](PROPOSAL_entity_position.md) | Draft | Entity positions in the domain model canvas can only be set during create entity via the @position annotation. | | [Expression Type Checking for mxcli check](PROPOSAL_expression_type_checking.md) | Draft | mxcli check is currently a syntactic validator only. | | [Extend UPDATE WIDGETS to Built-in Widgets via Schema Registry](PROPOSAL_update_builtin_widget_properties.md) | Draft | update widgets currently only works for pluggable widgets (ComboBox, DataGrid2, etc.) — it cannot modify properties on built-in widgets like | @@ -114,21 +116,25 @@ for display in this README): | [Multi-Project Tree View](PROPOSAL_multi_project_tree.md) | Draft | Mendix solutions increasingly span multiple applications: a frontend and a backend, or a microservices landscape (product catalog, orders, f | | [mxcli auth — Mendix Platform Authentication](PROPOSAL_platform_auth.md) | Draft | A growing set of mxcli features need to talk to Mendix platform APIs on behalf of the user: | | [mxcli catalog — Mendix Catalog Integration](PROPOSAL_catalog_integration.md) | Draft | ⚠️ TERMINOLOGY NOTE: This proposal covers the external Mendix Catalog service at catalog.mendix.com (CLI: mxcli catalog search), which is se | +| [mxcli check — mine Mendix's diagnostics catalog to close the check↔mxbuild gap](PROPOSAL_check_diagnostics_catalog.md) | Draft | Relates to: PROPOSAL_check_mxbuild_gap_heuristics.md (one tactical slice of this — | | [mxcli Playground](mxcli-playground.md) | Draft | A public GitHub repository (mendixlabs/mxcli-playground) containing a ready-to-use Mendix project pre-configured with mxcli, Claude Code ski | +| [Playwright Session Reuse and Lifecycle Control](PROPOSAL_playwright_session_reuse.md) | Draft | Builds on proposal-playwright-cli.md, which | | [Project Brain — Persistent Knowledge and Session Scaffolding for Long-Term AI Collaboration](PROPOSAL_project_brain.md) | Draft | mxcli is developed with significant AI involvement, yet the project's knowledge infrastructure is built for static human reading rather than | | [RENAME with Reference Refactoring](PROPOSAL_rename_refactoring.md) | Draft | Renaming entities, microflows, pages, and modules is one of the most common refactoring operations. | | [Replace Generated Playwright Tests with playwright-cli](proposal-playwright-cli.md) | Draft | The current approach (documented in proposal-playwright-testing.md) has Claude Code generate TypeScript test files (.spec.ts), then run them | | [Self-Describing Syntax Feature Registry](syntax-feature-registry.md) | Draft | Branch: research/recursive-help-discovery | | [Version-Aware Agent Support](PROPOSAL_version_aware_agent_support.md) | Draft | Three use cases require mxcli to be version-aware at the MDL level: | +| [warm dev loop — Docker-free run and iPad split-screen preview](PROPOSAL_mxcli_dev_warm_loop.md) | Draft | Relates to: PROPOSAL_check_mxbuild_gap_heuristics.md (the static-check gate that | -## Archived (24) +## Archived (25) Terminal-status proposals. See [archive/](archive/). | Proposal | Status | Summary | |----------|--------|---------| +| [Unified Schema Registry](archive/UNIFIED_SCHEMA_REGISTRY.md) | Abandoned | Mendix project metadata (BSON document schemas, widget property structures, MDL keyword | | [Architecture Improvements for Agentic Development and Reduced Merge Conflicts](archive/PROPOSAL_agentic_architecture_improvements.md) | Done | Two related friction points have been identified: | | [Augment Widget Templates from .mpk at Runtime](archive/PROPOSAL_mpk_widget_augmentation.md) | Done | When mxcli creates pluggable widgets (ComboBox, DataGrid2, etc.), it uses static JSON templates extracted from a Mendix 11.6.0 project. | | [Business Events Support in mxcli](archive/PROPOSAL_business_events_support.md) | Done | Business Events is a Mendix feature for asynchronous event-driven integration, allowing applications to publish and subscribe to business ev | diff --git a/docs/11-proposals/UNIFIED_SCHEMA_REGISTRY.md b/docs/11-proposals/archive/UNIFIED_SCHEMA_REGISTRY.md similarity index 99% rename from docs/11-proposals/UNIFIED_SCHEMA_REGISTRY.md rename to docs/11-proposals/archive/UNIFIED_SCHEMA_REGISTRY.md index cfa2befac..28d19a891 100644 --- a/docs/11-proposals/UNIFIED_SCHEMA_REGISTRY.md +++ b/docs/11-proposals/archive/UNIFIED_SCHEMA_REGISTRY.md @@ -1,6 +1,7 @@ --- title: Unified Schema Registry -status: descoped +status: abandoned +date: 2026-06-05 --- # Proposal: Unified Schema Registry diff --git a/scripts/extract-diagnostics-catalog.sh b/scripts/extract-diagnostics-catalog.sh new file mode 100755 index 000000000..4d5285dc6 --- /dev/null +++ b/scripts/extract-diagnostics-catalog.sh @@ -0,0 +1,100 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: Apache-2.0 +# +# extract-diagnostics-catalog.sh — mine Mendix's consistency-diagnostics catalog +# (CE/CW/CI codes) from Mendix.Modeler.Texts.dll into structured, tiered data. +# +# The SCRIPT is ours; its OUTPUT is Mendix's proprietary message corpus — do NOT +# commit the generated catalog.{csv,json}. See PROPOSAL_check_diagnostics_catalog.md. +# +# Usage: scripts/extract-diagnostics-catalog.sh [VERSION] [LOCALE] [OUTDIR] +# Example: scripts/extract-diagnostics-catalog.sh 11.12.1 en-US /tmp/diagcat +# Requires: curl, tar, monodis (apt-get install -y mono-utils), python3 +set -euo pipefail + +VERSION="${1:-11.12.1}" +LOCALE="${2:-en-US}" +OUTDIR="${3:-./diag-catalog-$VERSION}" +DLL="modeler/Mendix.Modeler.Texts.dll" +URL="https://cdn.mendix.com/runtime/mxbuild-${VERSION}.tar.gz" + +command -v monodis >/dev/null || { echo "ERROR: monodis not found (apt-get install -y mono-utils)" >&2; exit 1; } +mkdir -p "$OUTDIR"; cd "$OUTDIR" + +echo "[1/3] streaming $DLL from $URL ..." +# tar --occurrence=1 exits after the one member, closing the pipe early; curl then +# gets a (harmless) SIGPIPE/write-failure. Tolerate it and verify by file existence. +set +o pipefail +curl -fsSL "$URL" 2>/dev/null | tar -xz --occurrence=1 "$DLL" || true +set -o pipefail +[ -f "$DLL" ] || { echo "ERROR: failed to extract $DLL from $URL" >&2; exit 1; } + +echo "[2/3] extracting embedded PO resources via monodis ..." +mkdir -p po && ( cd po && monodis --mresources "../$DLL" >/dev/null 2>&1 ) +PO="$(ls po/*problem-descriptions_${LOCALE}.po | grep -vE 'deprecated|test' | head -1)" +[ -n "$PO" ] || { echo "ERROR: primary $LOCALE PO not found" >&2; exit 1; } + +echo "[3/3] parsing + tiering $PO ..." +python3 - "$PO" "$VERSION" <<'PY' +import re, json, csv, sys, collections +po, version = sys.argv[1], sys.argv[2] +SEV = {"CE": "error", "CW": "warning", "CI": "info"} +# static-checkability tiers (heuristic: message text + source subsystem) +D = ['version control','marketplace','app store','environment','deployment','license', + 'certificate','migration','team server','commit','branch','merge','governance'] +C = ['data source','react client','not supported in','offline','native','reachable', + 'end event','unreachable','navigation','home page','loop'] +B = ['xpath','expression','type of','of type','cannot be converted','incompatible', + 'return type','data type','parameter','must return','regular expression','template'] +def tier(blob): + if any(k in blob for k in D): return 'D' + if any(k in blob for k in C): return 'C' + if any(k in blob for k in B): return 'B' + return 'A' + +code_re = re.compile(r'^(C[EWI])(\d+)__(.*)$') +rows, loc = [], [] +lines = open(po, encoding='utf-8').read().splitlines() +i = 0 +while i < len(lines): + ln = lines[i] + if ln.startswith('# LOCATION:'): + loc.append(re.sub(r'\s*\(in project.*', '', ln.split('LOCATION:', 1)[1]).strip()) + elif ln.startswith('# NOTE:') and 'MX_DOCS_CAT' in ln: + docs = ln.split('MX_DOCS_CAT', 1)[1].lstrip(':(colon) ').strip() + elif ln.startswith('msgid "'): + mid = ln[7:-1] + j = i + 1 + while j < len(lines) and not lines[j].startswith('msgstr'): j += 1 + msg = '' + if j < len(lines): + msg = lines[j][8:].rstrip('"') + k = j + 1 + while k < len(lines) and lines[k].startswith('"'): + msg += lines[k].strip().strip('"'); k += 1 + m = code_re.match(mid) + if m: + params = sorted(set(re.findall(r'\{([A-Z0-9_]+)\}', msg))) + rows.append(dict(code=m.group(1)+m.group(2), severity=SEV[m.group(1)], + slug=m.group(3), message=msg, params=params, + locations=sorted(set(loc)), + docs_cat=locals().get('docs'), + tier=tier((m.group(3) + ' ' + msg + ' ' + ' '.join(loc)).lower()))) + loc, docs = [], None + i += 1 + +json.dump({"version": version, "codes": rows}, open("catalog.json", "w"), indent=1) +with open("catalog.csv", "w", newline='') as f: + w = csv.writer(f); w.writerow(["code","severity","tier","message","params","source","docs_cat"]) + for r in rows: + w.writerow([r['code'], r['severity'], r['tier'], r['message'], + ';'.join(r['params']), ';'.join(r['locations']), r['docs_cat'] or '']) + +sev = collections.Counter(r['severity'] for r in rows) +tiers = collections.Counter(r['tier'] for r in rows) +print(f" codes: {len(rows)} ({dict(sev)})") +print(f" tiers: {dict(sorted(tiers.items()))} (A=structural B=expr/type C=cross-doc D=runtime)") +print(f" parameterised: {sum(1 for r in rows if r['params'])}/{len(rows)}") +print(f" wrote catalog.json + catalog.csv (LOCAL ONLY — Mendix's corpus, do not commit)") +PY +echo "Done: $OUTDIR/catalog.{json,csv}"