Re-runnable domain scripts: exec --continue-on-error + idempotent ADD/DROP ATTRIBUTE

.claude/skills/fix-issue.md

@@ -166,6 +166,7 @@ to the symptom table below, so the next similar issue costs fewer reads.
 | No way to change an existing **enumeration value's caption** in place — `alter enumeration` had only `ADD`/`RENAME`/`DROP VALUE` + `SET COMMENT`; `RENAME VALUE X TO Y` changes the *name*, not the caption. The only route was drop + recreate, which fails while the enum is referenced by an attribute | Missing grammar action + AST op + executor case — a full-stack gap, not a code bug | `mdl/grammar/domains/MDLDomainModel.g4` (`alterEnumerationAction`) → `mdl/ast/ast_enumeration.go` (`AlterEnumOp`) → `mdl/visitor/visitor_enumeration.go` (`ExitAlterEnumerationAction`) → `mdl/executor/cmd_enumerations.go` (`execAlterEnumeration`) | Add `MODIFY VALUE IDENTIFIER CAPTION STRING_LITERAL` (reuses existing `MODIFY`/`CAPTION` tokens, so no lexer change); add `AlterEnumModifyCaption` op reusing the `Caption` field; executor finds the value by name and replaces only the `en_US` translation (preserves the value's ID + other locales). Re-captions in place, so it works while referenced (mxbuild 0 errors) — no drop needed. Value names in `alter` must be plain identifiers (a reserved-word-named value like `Created` can't be targeted — pre-existing, shared by ADD/RENAME/DROP). Tests: `TestAlterEnumeration_ModifyValueCaption` (visitor) + `TestAlterEnumeration_ModifyValueCaption_Mock` (executor); example in `01-domain-model-examples.mdl` |
 | A page with a native **LISTVIEW** over a `database from` (XPath) datasource passes `mx check` but **crashes the browser client at runtime** and redirects to login: `TypeError: Cannot read properties of undefined (reading 'length')` at `processResult` → `retrieveByXPath`. Pluggable Gallery/DataGrid2 database sources are fine | The serialized `Forms$ListViewXPathSource` omitted the arrays the client reads `.length` of. **Codec (default):** `Forms$ListViewSearch` emitted without its `SearchRefs` list — the encoder drops an empty, never-`Set` PartList unless the `$Type` is in `RegisterTypeDefaults` (GridSortBar.SortItems was registered; ListViewSearch was not). **Legacy:** wrote a bogus `Forms$ListViewSort` + a `Paths` key (renamed `SearchRefs` in 7.11.0) and no `Forms$GridSortBar`. NOT a `SortItems` marker issue — an empty `[2]` compiles fine when search is off; the crash is the absent `SearchRefs`. Diagnose by building a Deploy target and reading the compiled `deployment/web/pages/<Page>.js` (the client model) + `mxcli bson dump --type page` on the source | `mdl/backend/modelsdk/widget_write.go` (`init` RegisterTypeDefaults) + `sdk/mpr/writer_widgets_display.go` (`serializeListViewDataSource`, `emptyListViewXPathSource`) | Codec: `RegisterTypeDefaults("Forms$ListViewSearch", {MandatoryLists: []string{"SearchRefs"}})` (emits empty marker-3 list). Legacy: emit `Forms$GridSortBar`/`SortItems` (mirror `SerializeCustomWidgetDataSource`) + `Forms$ListViewSearch`/`SearchRefs` + `ForceFullObjects`; drop the bogus `Sort`/`Paths`. Tests: `TestListViewSourceToGen_SearchRefsEmitted` (codec, encode-level), `TestSerializeListViewDataSource_Database` + `TestEmptyListViewXPathSource_Shape` (legacy). Repro `mdl-examples/bug-tests/listview-database-source-searchrefs.mdl` |
 | Editing `themesource/**/main.scss` (or `theme/web/main.scss`) while `mxcli run --local` serves on `:8080` keeps showing **old styles** — reads exactly like a stale compiled-CSS cache. `rm -rf theme-cache/ .mendix-cache/ deployment/` "fixes" it only because a restart came with it | THREE distinct causes, none a CSS cache: (1) **no `--watch` = no watcher at all** — `mxbuild --serve` only rebuilds on a `/build` request (startup, or a watch tick), so a save changes nothing; (2) **stale process silently adopted** — a leftover serve/runtime on the ports answers the startup readiness probes (`waitReady`/`waitAdminReady` only check "port answers"), so a new run attaches to the OLD process and its own child is torn down by `defer`; a backgrounded `run --local` whose wrapping shell exited non-zero dies while its serve+runtime keep serving; (3) the theme source was **watched by nothing** — the `--watch` signal was model-only (`.mpr`+`mprcontents/`). The incremental theme step itself is FINE: one `/build` after an scss **content** edit does rewrite `theme-cache/web/theme.compiled.css` (verified), so there is no cache to clear | `cmd/mxcli/docker/runlocal.go` — `checkTargetPortsFree` (guard), `themeSourceMTime`/`sourceMTime` (watch signal), `watchAndApply` (generation log) | (2) Refuse to boot when `:8080/:8090/:6543` already answer, with an actionable message (never auto-kill — user's call). (3) Add `theme/`+`themesource/` scss/css/js to the `--watch` mtime-poll signal (`sourceMTime` = max(model, theme)); poll-based so it's container-safe (unlike the rollup chokidar/inotify web-client watcher). Log a build-generation counter (`build #2`) so "did it take?" is answerable. Docs: `docs-site/src/tools/run-local.md` + skill `run-local.md` — "SCSS needs a rebuild (`--watch` or clean restart), never a cache-clear; kill the old serve/runtime first". Tests `TestThemeSourceMTime_WatchesThemeAndThemesource`, `TestCheckTargetPortsFree` |
+| Re-running a domain script that is 90% already-applied applies **none** of the remaining 10%: `alter entity … add attribute X` errors `attribute 'X' already exists` and, because `exec` halts on the first error, everything after it is skipped. No idempotent add/drop and no continue-past-errors | Two gaps: (1) `ADD ATTRIBUTE` / `DROP ATTRIBUTE` had no `IF NOT EXISTS` / `IF EXISTS` guard, so a re-apply was a hard error; (2) `ExecuteProgram` returns on the first statement error | grammar `mdl/grammar/domains/MDLDomainModel.g4` (`ifNotExists`/`ifExists`, `alterEntityAction`) + AST `mdl/ast/ast_entity.go` (`IfNotExists`/`IfExists`) + visitor `mdl/visitor/visitor_entity.go` (`ExitAlterEntityAction`) + executor `mdl/executor/cmd_entities.go` (add/drop guards) + `mdl/executor/executor.go` (`ExecuteProgramContinueOnError`) + `cmd/mxcli/cmd_exec.go` (`--continue-on-error`) | Add `IF NOT EXISTS`/`IF EXISTS` to the grammar (regen), carry the flag on the AST, and in the executor turn the already-exists / not-found error into a skip-with-notice when the guard is set. Separately add `mxcli exec --continue-on-error`: attempts every statement, prints each failure as `statement N: …`, exits non-zero if any failed (never masks a real error; `exit`/`quit` still stop the run). Bug-test `mdl-examples/bug-tests/f10-idempotent-alter-entity.mdl`. Findings #10 |
 | `alter entity M.E add attribute X: autonumber;` (no seed) or `... add attribute Created: AutoCreatedDate;` (renamed AutoX) passes `mxcli check` "Check passed!" but fails the build (CE7247) / silently discards the name — while the SAME attribute in `create entity` is correctly flagged (MDL023 / MDL022) | The per-attribute checks (MDL021/022/023) only ran on `CreateEntityStmt`; the ALTER ENTITY ADD ATTRIBUTE path had no validation at all, so an attribute added later escaped every rule | `mdl/executor/cmd_enumerations.go` (`ValidateAlterEntity`, `validateEntityAttribute`) wired from `cmd/mxcli/cmd_check.go` + `cmd/mxcli/lsp_diagnostics.go` | Extract the CREATE loop body into `validateEntityAttribute(attr, persistent, entityName)`; add `ValidateAlterEntity(stmt)` that runs it on `AlterEntityAddAttribute`. The entity kind isn't known from ALTER, so the persistent-only MDL020 is skipped; the kind-independent MDL021/022/023 all run. Bug-test `mdl-examples/bug-tests/f6-autonumber-seed-alter.fail.mdl`. Findings #6 (alter path) |
 | `create or modify entity M.E ( <subset> )` on an entity that already has more attributes **silently drops** every attribute not re-listed (36→2 attrs seen in practice), then widgets/microflows still bound to them fail the build with CE1613 — and the "already exists" error that leads users here recommends the destructive `create or modify` for a partial edit | `create or modify` rebuilds the entity from the statement alone and REPLACEs the stored one, so any omitted attribute is deleted with no warning; the `NewAlreadyExistsMsg` hint pointed at `create or modify` without distinguishing "replace whole" from "add one" | `mdl/executor/cmd_entities.go` (`droppedEntityMembers`, the warn block before `UpdateEntity`, and the `execCreateEntity` already-exists message) | Warn-only (non-blocking, the user asked to modify): before `UpdateEntity`, diff existing vs replacement members (`droppedEntityMembers` — named attrs case-insensitive + the four audit flags) and print what's dropped + point at `alter entity … add attribute` for incremental edits. Fix the already-exists message to recommend `alter entity` for a member change and reserve `create or modify` for a full replace. Bug-test `mdl-examples/bug-tests/f24-create-or-modify-dataloss.mdl`. Findings #24 |
 

cmd/mxcli/cmd_exec.go

@@ -17,14 +17,23 @@ var execCmd = &cobra.Command{
 	Short: "Execute an MDL script file",
 	Long: `Execute an MDL script file containing MDL commands.
 
+By default execution stops at the first error. With --continue-on-error, every
+statement is attempted; each failure is reported (prefixed with its statement
+number) and execution continues, exiting non-zero if any statement failed. This
+makes a partially-applied domain script re-runnable — the already-applied
+statements (e.g. "attribute already exists") error individually while the not-
+yet-applied ones still run — without a failure masking later work.
+
 Example:
   mxcli exec setup.mdl
   mxcli exec -p app.mpr script.mdl
+  mxcli exec -p app.mpr script.mdl --continue-on-error
 `,
 	Args: cobra.ExactArgs(1),
 	Run: func(cmd *cobra.Command, args []string) {
 		filePath := args[0]
 		projectPath, _ := cmd.Flags().GetString("project")
+		continueOnError, _ := cmd.Flags().GetBool("continue-on-error")
 
 		// Read the file
 		content, err := os.ReadFile(filePath)
@@ -58,6 +67,19 @@ Example:
 			os.Exit(1)
 		}
 
+		if continueOnError {
+			res, err := exec.ExecuteProgramContinueOnError(prog, os.Stderr)
+			if err != nil && !errors.Is(err, executor.ErrExit) {
+				fmt.Fprintf(os.Stderr, "Error: %v\n", err)
+				os.Exit(1)
+			}
+			fmt.Fprintf(os.Stderr, "%d statements: %d succeeded, %d failed\n", res.Total, res.Succeeded, res.Failed)
+			if res.Failed > 0 {
+				os.Exit(1)
+			}
+			return
+		}
+
 		if err := exec.ExecuteProgram(prog); err != nil {
 			if errors.Is(err, executor.ErrExit) {
 				return
@@ -67,3 +89,8 @@ Example:
 		}
 	},
 }
+
+func init() {
+	execCmd.Flags().Bool("continue-on-error", false,
+		"Run every statement, reporting each failure instead of halting at the first (exits non-zero if any failed) — makes a partially-applied script re-runnable")
+}

cmd/mxcli/syntax/features_domain_model.go

@@ -49,9 +49,10 @@ func init() {
 			"alter entity", "modify entity", "add attribute",
 			"drop attribute", "rename attribute", "add index",
 			"event handler", "documentation",
+			"if not exists", "if exists", "idempotent",
 		},
-		Syntax:  "ALTER ENTITY Module.Name ADD ATTRIBUTE AttrName: Type [constraints];\nALTER ENTITY Module.Name DROP ATTRIBUTE AttrName;\nALTER ENTITY Module.Name RENAME ATTRIBUTE OldName TO NewName;\nALTER ENTITY Module.Name MODIFY ATTRIBUTE AttrName SET DEFAULT val;\nALTER ENTITY Module.Name ADD INDEX (attr1, attr2);\nALTER ENTITY Module.Name SET DOCUMENTATION 'text';\nALTER ENTITY Module.Name ADD EVENT HANDLER ON BEFORE COMMIT CALL Module.MF RAISE ERROR;",
-		Example: "ALTER ENTITY Shop.Customer ADD ATTRIBUTE Phone: String(20);\nALTER ENTITY Shop.Customer DROP ATTRIBUTE OldField;\nALTER ENTITY Shop.Customer RENAME ATTRIBUTE Email TO EmailAddress;\nALTER ENTITY Shop.Customer ADD INDEX (EmailAddress);\nALTER ENTITY Shop.Customer\n  ADD EVENT HANDLER ON BEFORE COMMIT CALL Shop.Validate($currentObject) RAISE ERROR;",
+		Syntax:  "ALTER ENTITY Module.Name ADD ATTRIBUTE [IF NOT EXISTS] AttrName: Type [constraints];\nALTER ENTITY Module.Name DROP ATTRIBUTE [IF EXISTS] AttrName;\nALTER ENTITY Module.Name RENAME ATTRIBUTE OldName TO NewName;\nALTER ENTITY Module.Name MODIFY ATTRIBUTE AttrName SET DEFAULT val;\nALTER ENTITY Module.Name ADD INDEX (attr1, attr2);\nALTER ENTITY Module.Name SET DOCUMENTATION 'text';\nALTER ENTITY Module.Name ADD EVENT HANDLER ON BEFORE COMMIT CALL Module.MF RAISE ERROR;\n\nIF NOT EXISTS / IF EXISTS make the add/drop a no-op (skipped, not an error)\nwhen the attribute is already present / already gone — so a domain script\nre-runs cleanly. For a whole script, 'mxcli exec --continue-on-error' reports\neach failed statement and keeps going instead of halting at the first.",
+		Example: "ALTER ENTITY Shop.Customer ADD ATTRIBUTE Phone: String(20);\nALTER ENTITY Shop.Customer ADD ATTRIBUTE IF NOT EXISTS Phone: String(20);  -- re-runnable\nALTER ENTITY Shop.Customer DROP ATTRIBUTE IF EXISTS OldField;              -- re-runnable\nALTER ENTITY Shop.Customer RENAME ATTRIBUTE Email TO EmailAddress;\nALTER ENTITY Shop.Customer ADD INDEX (EmailAddress);\nALTER ENTITY Shop.Customer\n  ADD EVENT HANDLER ON BEFORE COMMIT CALL Shop.Validate($currentObject) RAISE ERROR;",
 		SeeAlso: []string{"domain-model.entity.create", "domain-model.entity.attributes"},
 	})
 

docs/01-project/MDL_QUICK_REFERENCE.md

@@ -95,6 +95,10 @@ Modifies an existing entity without full replacement.
 | Set position | `alter entity Module.Name set position (100, 200);` | Canvas position |
 | Add system attribute | `alter entity Module.Name add attribute owner: autoowner;` | Same syntax as regular attributes |
 | Drop system attribute | `alter entity Module.Name drop attribute owner;` | Drop by system attribute name |
+| Add attribute (idempotent) | `alter entity Module.Name add attribute if not exists AttrName: type;` | Skipped (not an error) if the attribute already exists — re-runnable |
+| Drop attribute (idempotent) | `alter entity Module.Name drop attribute if exists AttrName;` | Skipped (not an error) if the attribute is already gone — re-runnable |
+
+> **Re-running domain scripts.** `IF NOT EXISTS` / `IF EXISTS` make an individual add/drop a no-op when already applied. To re-run a whole script that is partly applied, use `mxcli exec script.mdl --continue-on-error`: every statement is attempted, each failure is reported with its statement number, and the command exits non-zero if any failed (a real error is still surfaced, never masked).
 
 **Example:**
 ```sql

mdl-examples/bug-tests/f10-idempotent-alter-entity.mdl

@@ -0,0 +1,12 @@
+-- Findings #10: domain-delta scripts weren't re-runnable — a second run of
+-- `alter entity ... add attribute X` errored ("already exists") and, because
+-- exec halts on the first error, applied none of the remaining statements.
+--
+-- IF NOT EXISTS / IF EXISTS make each add/drop a no-op (skipped, not an error)
+-- when already applied, so this whole file can be run repeatedly and converge.
+-- (For scripts using the plain add/drop forms, `mxcli exec --continue-on-error`
+-- reports each failed statement and keeps going instead of halting.)
+
+alter entity Sudoku.Game add attribute if not exists PuzzleNo: integer default 1;
+alter entity Sudoku.Game add attribute if not exists DifficultyLevel: integer default 1;
+alter entity Sudoku.Game drop attribute if exists LegacyFlag;

mdl/ast/ast_entity.go

@@ -109,6 +109,12 @@ type AlterEntityStmt struct {
 	Position           *Position        // For SET POSITION
 	EventHandler       *EventHandlerDef // For ADD/DROP EVENT HANDLER
 	BoolValue          bool             // For SET ALLOW_CREATE_CHANGE_LOCALLY
+	// Idempotency guards (findings #10). IfNotExists on ADD ATTRIBUTE skips the
+	// add (with a notice) when the attribute already exists; IfExists on DROP
+	// ATTRIBUTE skips the drop when it is already gone — so a domain script
+	// re-runs cleanly instead of erroring and halting.
+	IfNotExists bool // For ADD ATTRIBUTE IF NOT EXISTS
+	IfExists    bool // For DROP ATTRIBUTE IF EXISTS
 }
 
 func (s *AlterEntityStmt) isStatement() {}