diff --git a/.claude/skills/fix-issue.md b/.claude/skills/fix-issue.md index ca0cac36b..6c5956530 100644 --- a/.claude/skills/fix-issue.md +++ b/.claude/skills/fix-issue.md @@ -156,6 +156,7 @@ to the symptom table below, so the next similar issue costs fewer reads. | A built-in widget action (`actionbutton`/`container`/`linkbutton` `Action: microflow M.Flow(Param: $x)`) **silently drops the argument** on the **default (modelsdk)** engine — `describe` shows `Action: microflow M.Flow` (no args), the flow runs with no parameter, and a required-param flow never fires so the button/row no-ops at runtime. `mxcli check`/`docker check`/`docker build` all pass (0 errors) — silent break. Legacy persists it (the workaround) | `clientActionToGen`'s `MicroflowClientAction` case built the `Forms$MicroflowSettings` from the microflow **name only** (`microflowSettingsToGen(x.MicroflowName)`), never serializing `x.ParameterMappings` — the executor builds them correctly into the model, the writer threw them away. (NOT the same as `custom-widgets.md`'s "action mapping too complex for auto-mapping" — that's about *pluggable-widget action-typed properties* in def.json generation, a different path.) | `mdl/backend/modelsdk/widget_write.go` (`microflowSettingsToGen`) | Give `microflowSettingsToGen` the mappings param; emit one `genPg.MicroflowParameterMapping` each — `SetParameterQualifiedName(".")` (BY_NAME) + `SetExpression(variable-or-expression)` — mirroring the legacy `writer_widgets_action.go` BSON. The client-action caller passes `x.ParameterMappings`; the microflow-datasource callers pass `nil`. Verified: default-engine describe now keeps `(Param: $x)`; a type-correct page = 0 mxbuild errors (both engines give the *same* CE0117 on a type-mismatched arg — that's a model error, not the writer). Tests: `TestClientActionToGen_MicroflowParameterMappings`; bug-test `mdl-examples/bug-tests/bug1-widget-action-param-mapping.mdl`. Nanoflow widget actions were a separate gap (the whole `NanoflowClientAction` case was missing) — now fixed in Bug 2 below. Bug 1 | | A microflow `declare $x Module.Entity [= $obj];` (object/entity-typed local variable) passes `mxcli check` but mxbuild rejects it with **CE0053** "Selected type is not allowed" (+ CE0038 "Value required", + CE7247 on a following `set`) — a silent trap. The declare-entity attribute qualifier is a red herring: the whole construct is invalid, bare or initialized | Mendix's Create Variable activity (what `declare` maps to) only holds **primitive** types; there is no object/list Create Variable form (same restriction as MDL040 for lists). Objects must come from a parameter, a `retrieve … limit 1`, a `create` object, or a loop iterator. The check was missing; the microflow parser records a bare `Module.X` declare type as the ambiguous `TypeEnumeration` (`EnumRef`, `ExplicitEnum=false`), an explicit `Enumeration(Module.X)` sets `ExplicitEnum=true` | `mdl/executor/validate_microflow.go` (`walkBody` `*ast.DeclareStmt` case, next to the MDL040 list check) | Add **MDL043**: flag `Type.Kind == TypeEntity \|\| (TypeEnumeration && EnumRef != nil && !ExplicitEnum)`; error points to parameter/retrieve/create/loop and notes "if it's an enum, write `Enumeration(...)`". Reliable connection-free (no backend) because bare object names resolve to a distinct AST shape from explicit enums. Also fix the skills that wrongly taught `declare $x Module.Entity;` / `declare $x as …` as valid (`write-microflows.md`, `cheatsheet-variables.md`, `cheatsheet-errors.md`, `check-syntax.md`, `patterns-crud.md`, `patterns-data-processing.md`, `business-events.md`, `resolve-forward-references.md`, `migrate-k2-nintex.md`, `README.md`). Test: `TestValidateMicroflow_DeclareObjectIsRejected`; bug-test `mdl-examples/bug-tests/declare-object-variable-rejected.fail.mdl`. mxbuild-confirmed (11.12.0) | | No way to change an existing **enumeration value's caption** in place — `alter enumeration` had only `ADD`/`RENAME`/`DROP VALUE` + `SET COMMENT`; `RENAME VALUE X TO Y` changes the *name*, not the caption. The only route was drop + recreate, which fails while the enum is referenced by an attribute | Missing grammar action + AST op + executor case — a full-stack gap, not a code bug | `mdl/grammar/domains/MDLDomainModel.g4` (`alterEnumerationAction`) → `mdl/ast/ast_enumeration.go` (`AlterEnumOp`) → `mdl/visitor/visitor_enumeration.go` (`ExitAlterEnumerationAction`) → `mdl/executor/cmd_enumerations.go` (`execAlterEnumeration`) | Add `MODIFY VALUE IDENTIFIER CAPTION STRING_LITERAL` (reuses existing `MODIFY`/`CAPTION` tokens, so no lexer change); add `AlterEnumModifyCaption` op reusing the `Caption` field; executor finds the value by name and replaces only the `en_US` translation (preserves the value's ID + other locales). Re-captions in place, so it works while referenced (mxbuild 0 errors) — no drop needed. Value names in `alter` must be plain identifiers (a reserved-word-named value like `Created` can't be targeted — pre-existing, shared by ADD/RENAME/DROP). Tests: `TestAlterEnumeration_ModifyValueCaption` (visitor) + `TestAlterEnumeration_ModifyValueCaption_Mock` (executor); example in `01-domain-model-examples.mdl` | +| Editing `themesource/**/main.scss` (or `theme/web/main.scss`) while `mxcli run --local` serves on `:8080` keeps showing **old styles** — reads exactly like a stale compiled-CSS cache. `rm -rf theme-cache/ .mendix-cache/ deployment/` "fixes" it only because a restart came with it | THREE distinct causes, none a CSS cache: (1) **no `--watch` = no watcher at all** — `mxbuild --serve` only rebuilds on a `/build` request (startup, or a watch tick), so a save changes nothing; (2) **stale process silently adopted** — a leftover serve/runtime on the ports answers the startup readiness probes (`waitReady`/`waitAdminReady` only check "port answers"), so a new run attaches to the OLD process and its own child is torn down by `defer`; a backgrounded `run --local` whose wrapping shell exited non-zero dies while its serve+runtime keep serving; (3) the theme source was **watched by nothing** — the `--watch` signal was model-only (`.mpr`+`mprcontents/`). The incremental theme step itself is FINE: one `/build` after an scss **content** edit does rewrite `theme-cache/web/theme.compiled.css` (verified), so there is no cache to clear | `cmd/mxcli/docker/runlocal.go` — `checkTargetPortsFree` (guard), `themeSourceMTime`/`sourceMTime` (watch signal), `watchAndApply` (generation log) | (2) Refuse to boot when `:8080/:8090/:6543` already answer, with an actionable message (never auto-kill — user's call). (3) Add `theme/`+`themesource/` scss/css/js to the `--watch` mtime-poll signal (`sourceMTime` = max(model, theme)); poll-based so it's container-safe (unlike the rollup chokidar/inotify web-client watcher). Log a build-generation counter (`build #2`) so "did it take?" is answerable. Docs: `docs-site/src/tools/run-local.md` + skill `run-local.md` — "SCSS needs a rebuild (`--watch` or clean restart), never a cache-clear; kill the old serve/runtime first". Tests `TestThemeSourceMTime_WatchesThemeAndThemesource`, `TestCheckTargetPortsFree` | --- diff --git a/.claude/skills/mendix/run-local.md b/.claude/skills/mendix/run-local.md index dd90761f5..34f57b5dd 100644 --- a/.claude/skills/mendix/run-local.md +++ b/.claude/skills/mendix/run-local.md @@ -68,8 +68,40 @@ mxcli run --local -p app.mpr --watch mxcli exec add-page.mdl -p app.mpr ``` -`--watch` observes the project directory (MPR v1 and v2 layouts), ignoring -`deployment/`, `.git`, and `node_modules`. +`--watch` observes two source trees and rebuilds when either changes: the **model +source** (`.mpr` + `mprcontents/`, v1 and v2 layouts) and the **theme source** +(`theme/` and `themesource//web/` — SCSS/CSS/JS). It ignores build output +(`deployment/`, `theme-cache/`, `.mendix-cache/`, `.mxcli/`). Both signals are mtime +polling, so they work on container filesystems where inotify is silent. Each apply is +logged with a build-generation counter (`build #2`, …) so you can confirm a change +landed. + +## Editing themes (SCSS) — rebuild, never clear caches + +A theme edit (`theme/web/main.scss`, module SCSS) needs a **rebuild**, not a +cache-clear. With `--watch`, just save the `.scss` — the theme source is watched and +the loop rebuilds and hot-applies. Without `--watch`, nothing is watched, so the save +does nothing until you restart `run --local` (or re-run with `--watch`). `mxbuild +--serve` recompiles the theme on its next build and correctly picks up SCSS content +changes, so **never** `rm -rf theme-cache/ .mendix-cache/ deployment/` — clearing +caches is a red herring. + +## "My edit didn't show up" — stale process, not stale cache + +`run --local` refuses to boot when its ports (8080/8090/6543) are already answering, +because a leftover `run --local` / `mxbuild --serve` / runtime would otherwise be +silently adopted and keep serving old output (it looks like a cache but is a stale +**process**). If a background `run --local` died while its serve+runtime kept serving, +recover with: + +```bash +pgrep -af 'mxbuild --serve|runtimelauncher|mxcli run' # find them +kill # stop each +curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080 # want 000 +``` + +Launch `run --local` as the **sole** command in its invocation (don't chain a trailing +`sleep`/`curl` whose non-zero exit can kill the backgrounded run); poll separately. ## Flags diff --git a/cmd/mxcli/docker/runlocal.go b/cmd/mxcli/docker/runlocal.go index d60e3c803..e7feb546d 100644 --- a/cmd/mxcli/docker/runlocal.go +++ b/cmd/mxcli/docker/runlocal.go @@ -187,6 +187,43 @@ func pingTCP(hostPort string, timeout time.Duration) error { return nil } +// checkTargetPortsFree refuses to boot when any of the loop's ports is already +// answering, which almost always means a previous `mxcli run --local` (or a +// stray mxbuild --serve / runtime) is still alive. Without this guard the boot +// "succeeds" against the stale process: the readiness probes only check that the +// port answers (StartServe.waitReady / runtime waitAdminReady), so mxcli adopts +// the old serve/runtime, the fresh JVM that failed to bind is torn down by +// defer, and the old process keeps serving stale output — reading exactly like a +// stale cache. Refusing with an actionable message turns that silent failure +// into a clear one. We only *detect* here (never kill): reaping someone else's +// process is the user's call. +func checkTargetPortsFree(o LocalRunOptions) error { + host := "127.0.0.1" + type p struct { + port int + role string + flag string + } + for _, c := range []p{ + {o.AppPort, "app", "--app-port"}, + {o.AdminPort, "admin API", "--admin-port"}, + {o.ServePort, "mxbuild serve", "--serve-port"}, + } { + hostPort := fmt.Sprintf("%s:%d", host, c.port) + if err := pingTCP(hostPort, 500*time.Millisecond); err == nil { + return fmt.Errorf("port %d (%s) is already in use — a previous 'mxcli run --local' "+ + "or a stray mxbuild --serve/runtime is likely still serving on it.\n"+ + " A stale process is silently adopted otherwise, so edits appear to do nothing (looks like a stale cache — it isn't).\n"+ + " Free the ports, then retry:\n"+ + " pgrep -af 'mxbuild --serve|runtimelauncher|mxcli run' # find them\n"+ + " kill # stop each; confirm with: curl -s -o /dev/null -w '%%{http_code}' http://%s:%d (want 000)\n"+ + " Or run on different ports with %s (and --admin-port/--serve-port).", + c.port, c.role, host, o.AppPort, c.flag) + } + } + return nil +} + // webClientSourceMTime returns the newest mtime of the browser-client *source* // under /web, excluding the rollup output (dist/) and the build log. // A page/widget/theme edit bumps it (and needs a client re-bundle); a @@ -238,6 +275,48 @@ func projectSourceMTime(projectPath string) time.Time { return newest } +// themeSourceMTime returns the newest mtime of the app's *theme source* — the +// SCSS/CSS/JS the designer edits under theme/ (app-level: main.scss, +// custom-variables.scss, …) and themesource//web/ (per-module). These +// live outside the .mpr/mprcontents tree, so projectSourceMTime never sees them; +// without this a theme edit triggers no rebuild even under --watch, and the app +// keeps serving the old styles (the "SCSS cache" symptom — really a missing +// watch). mxbuild --serve recompiles the theme on the next /build, so surfacing +// the edit as a change signal is all that's needed. Walk is mtime-polling (same +// as projectSourceMTime), so it is reliable on container filesystems where +// inotify is not — no watcher fd involved. +func themeSourceMTime(projectPath string) time.Time { + dir := filepath.Dir(projectPath) + var newest time.Time + for _, sub := range []string{"theme", "themesource"} { + root := filepath.Join(dir, sub) + _ = filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error { + if err != nil || d.IsDir() { + return nil + } + switch strings.ToLower(filepath.Ext(path)) { + case ".scss", ".css", ".js", ".json": + if info, err := d.Info(); err == nil && info.ModTime().After(newest) { + newest = info.ModTime() + } + } + return nil + }) + } + return newest +} + +// sourceMTime is the combined --watch change signal: the newer of the model +// source (projectSourceMTime) and the theme source (themeSourceMTime). Either a +// model edit or a theme edit re-triggers the build. +func sourceMTime(projectPath string) time.Time { + m := projectSourceMTime(projectPath) + if t := themeSourceMTime(projectPath); t.After(m) { + return t + } + return m +} + // RunLocal boots the warm local dev loop: resolve tooling, start mxbuild --serve // and a standalone runtime, do the first build+apply, then (with Watch) rebuild // and hot-apply on every project change until interrupted. @@ -245,6 +324,16 @@ func RunLocal(opts LocalRunOptions) error { opts.applyDefaults() w, stderr := opts.Stdout, opts.Stderr + // 0. Refuse fast if the loop's ports are already taken (a stale run/serve/ + // runtime). Skipped for SetupOnly, which never boots a server. This is the + // single-instance guard: without it a stale process is silently adopted and + // keeps serving old output. + if !opts.SetupOnly { + if err := checkTargetPortsFree(opts); err != nil { + return err + } + } + // 1. Detect the project's Mendix version. fmt.Fprintln(w, "Detecting project version...") reader, err := mpr.Open(opts.ProjectPath) @@ -526,8 +615,12 @@ func watchAndApply(opts LocalRunOptions, serve *ServeServer, rt *LocalRuntime, w signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM) defer signal.Stop(sigCh) - fmt.Fprintln(w, "Watching for changes (Ctrl-C to stop)...") - last := projectSourceMTime(opts.ProjectPath) + fmt.Fprintln(w, "Watching model + theme source for changes (serving build #1; Ctrl-C to stop)...") + last := sourceMTime(opts.ProjectPath) + // gen is the served build generation — a monotonic counter surfaced on every + // apply so "did my change take?" is answerable from the log without guessing. + // The initial boot build is generation 1. + gen := 1 ticker := time.NewTicker(opts.PollInterval) defer ticker.Stop() @@ -537,12 +630,13 @@ func watchAndApply(opts LocalRunOptions, serve *ServeServer, rt *LocalRuntime, w fmt.Fprintln(w, "\nShutting down...") return nil case <-ticker.C: - now := projectSourceMTime(opts.ProjectPath) + now := sourceMTime(opts.ProjectPath) if !now.After(last) { continue } last = now - fmt.Fprintln(w, "Change detected, rebuilding...") + gen++ + fmt.Fprintf(w, "Change detected, rebuilding (build #%d)...\n", gen) start := time.Now() // Capture the client-bundle generation and the web-source mtime before @@ -584,13 +678,13 @@ func watchAndApply(opts LocalRunOptions, serve *ServeServer, rt *LocalRuntime, w } client := "" if bundled { - client = ", client re-bundled" + client = fmt.Sprintf(", client re-bundled (gen %d)", watcher.Generation()) } - fmt.Fprintf(w, " applied via %s in %s%s -> %s\n", action, time.Since(start).Round(time.Millisecond), client, rt.AppURL()) + fmt.Fprintf(w, " build #%d applied via %s in %s%s -> %s\n", gen, action, time.Since(start).Round(time.Millisecond), client, rt.AppURL()) maybeScreenshot(opts, rt) // Refresh the baseline AFTER the apply so an edit made mid-build is // still caught on the next tick. - last = projectSourceMTime(opts.ProjectPath) + last = sourceMTime(opts.ProjectPath) } } } diff --git a/cmd/mxcli/docker/runlocal_test.go b/cmd/mxcli/docker/runlocal_test.go index 3d7035cf6..2f65ab599 100644 --- a/cmd/mxcli/docker/runlocal_test.go +++ b/cmd/mxcli/docker/runlocal_test.go @@ -3,9 +3,12 @@ package docker import ( + "fmt" "io" + "net" "os" "path/filepath" + "strings" "testing" "time" ) @@ -242,3 +245,80 @@ func TestPageLabel(t *testing.T) { t.Errorf("pageLabel(/p/x) = %q", pageLabel("/p/x")) } } + +func TestThemeSourceMTime_WatchesThemeAndThemesource(t *testing.T) { + dir := t.TempDir() + mpr := filepath.Join(dir, "App.mpr") + if err := os.WriteFile(mpr, []byte("x"), 0o644); err != nil { + t.Fatal(err) + } + // App-level theme and a per-module theme source. + themeMain := filepath.Join(dir, "theme", "web", "main.scss") + moduleScss := filepath.Join(dir, "themesource", "travel", "web", "main.scss") + for _, f := range []string{themeMain, moduleScss} { + _ = os.MkdirAll(filepath.Dir(f), 0o755) + if err := os.WriteFile(f, []byte("// scss"), 0o644); err != nil { + t.Fatal(err) + } + } + + base := themeSourceMTime(mpr) + if base.IsZero() { + t.Fatal("expected a non-zero theme source mtime") + } + + // Editing the app-level main.scss MUST advance the signal. + future := time.Now().Add(time.Hour) + _ = os.Chtimes(themeMain, future, future) + if !themeSourceMTime(mpr).After(base) { + t.Error("theme/web/main.scss change should advance the theme signal") + } + + // Editing a per-module theme source MUST advance the signal too. + base2 := themeSourceMTime(mpr) + future2 := time.Now().Add(2 * time.Hour) + _ = os.Chtimes(moduleScss, future2, future2) + if !themeSourceMTime(mpr).After(base2) { + t.Error("themesource//web/main.scss change should advance the theme signal") + } + + // sourceMTime combines model + theme: a theme edit advances the combined signal + // even when the model is untouched (the core of the Problem-3 fix). + combinedBase := sourceMTime(mpr) + future3 := time.Now().Add(3 * time.Hour) + _ = os.Chtimes(themeMain, future3, future3) + if !sourceMTime(mpr).After(combinedBase) { + t.Error("a theme-only edit should advance the combined watch signal") + } +} + +func TestCheckTargetPortsFree(t *testing.T) { + // A run whose ports are all free must pass. + opts := LocalRunOptions{} + opts.applyDefaults() + // Move ports to almost-certainly-free high values so a real serve/8080 on the + // box doesn't make the test flaky. + opts.AppPort, opts.AdminPort, opts.ServePort = 5, 6, 7 // privileged/unused; nothing listens + if err := checkTargetPortsFree(opts); err != nil { + t.Errorf("expected free ports to pass, got: %v", err) + } + + // Occupy a port, point AppPort at it, and expect a refusal that names it. + ln, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatalf("listen: %v", err) + } + defer ln.Close() + busy := ln.Addr().(*net.TCPAddr).Port + opts.AppPort = busy + err = checkTargetPortsFree(opts) + if err == nil { + t.Fatalf("expected refusal when port %d is occupied", busy) + } + if want := fmt.Sprintf("port %d", busy); !strings.Contains(err.Error(), want) { + t.Errorf("error should name the busy port %q; got: %v", want, err) + } + if !strings.Contains(err.Error(), "already in use") { + t.Errorf("error should explain the port is in use; got: %v", err) + } +} diff --git a/docs-site/src/tools/run-local.md b/docs-site/src/tools/run-local.md index 437b72f92..0fd95f999 100644 --- a/docs-site/src/tools/run-local.md +++ b/docs-site/src/tools/run-local.md @@ -78,12 +78,66 @@ so structural changes need a restart; behavioural changes do not. ## The change signal -`--watch` watches the model **source** — the `.mpr` file and the `mprcontents/` -document tree (v2) — not the whole project dir. This is deliberate: the serve/mxbuild +`--watch` watches two **source** trees and rebuilds when either changes: + +- the **model source** — the `.mpr` file and the `mprcontents/` document tree (v2); and +- the **theme source** — `theme/` (app-level `main.scss`, `custom-variables.scss`, …) + and `themesource//web/` (per-module SCSS/CSS/JS). + +It does **not** watch the whole project dir. This is deliberate: the serve/mxbuild build rewrites `deployment/`, `theme-cache/`, and `.mendix-cache/` on every run, and screenshots land in `.mxcli/`; watching only the source keeps that build-output churn -from re-triggering the loop. The intended cycle: an agent (or you) edits the model -with `mxcli exec`/MDL, and the running `run --local` picks it up and hot-applies it. +from re-triggering the loop. Both signals are **mtime polling** (default 1 s), so they +work on container filesystems where inotify does not fire — no watcher fd is involved. + +Each applied change is logged with a **build generation** counter (`build #2`, +`build #3`, …; the boot build is `#1`), so "did my change take?" is answerable from +the log instead of guessed. + +The intended cycle: an agent (or you) edits the model with `mxcli exec`/MDL — or edits +a theme `.scss` — and the running `run --local` picks it up and hot-applies it. + +## Editing themes (SCSS): rebuild, don't clear caches + +A theme edit (e.g. `theme/web/main.scss`) needs a **rebuild**, not a cache-clear. +`mxbuild --serve` recompiles the theme on its next `/build`, and that recompile +correctly picks up SCSS **content** changes — there is no incremental-theme cache to +clear (verified: one `/build` after an `main.scss` content edit changes +`theme-cache/web/theme.compiled.css`). So: + +- **With `--watch`** — just save the `.scss`; the theme source is watched and the loop + rebuilds and hot-applies automatically. +- **Without `--watch`** — nothing watches anything, so a save changes nothing in the + running app. Trigger a rebuild: restart `run --local`, or use `--watch`. + +Do **not** `rm -rf theme-cache/ .mendix-cache/ deployment/` — clearing caches is a red +herring. If a theme edit "won't show up", the cause is that no rebuild ran (Problem +above) or a **stale process is still serving** (below), never a stale compiled-CSS +cache. + +## "My edit didn't show up" — it's usually a stale process, not a cache + +`run --local` refuses to boot if its ports (`8080` app, `8090` admin, `6543` serve) +are already answering — because a previous `run --local` (or a stray `mxbuild --serve` +/ runtime) left alive would otherwise be **silently adopted**: the startup readiness +probes only check that the port answers, so a fresh run would attach to the old +process and keep serving old output. That reads exactly like a stale cache but is a +stale **process**. + +If you started `run --local` in the background and the wrapping shell exited non-zero +(e.g. a chained `sleep`/`curl` that failed), the `run --local` process can die while +its `mxbuild --serve` + runtime keep serving on `:8080`. Launch `run --local` as the +**sole** command in its own invocation — don't chain a `sleep`/status check after it in +the same shell — and poll separately. To recover from a stale process: + +```bash +pgrep -af 'mxbuild --serve|runtimelauncher|mxcli run' # find them +kill # stop each +curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080 # want 000 (port free) +``` + +Then start `run --local` again. Or run on different ports with `--app-port` / +`--admin-port` / `--serve-port`. ## Pages render in the browser