Catalog: refs table is materially incomplete (misses most action refs, all associations, all layout/widget refs)

[ako]

Summary

The catalog refs table (cross-reference index that powers show callers|callees|references|impact|context of … after refresh catalog full) is a curated best-effort index, not a complete reference graph. It is built by a hand-maintained switch in mdl/catalog/builder_references.go — the same fragile pattern that silently dropped action types in catalog.activities (fixed in 6574e9c). Verified on mx-test-projects/MxGraphStudioDemo.

Gap 1 — microflow action refs: only 5 of ~15 reference-bearing action types

buildReferences extracts refs from MicroflowCallAction, CreateObjectAction, RetrieveAction (only DatabaseRetrieveSource), ShowPageAction, JavaActionCallAction. Every other reference-bearing action produces no ref. Present in MxGraphStudioDemo with zero refs emitted:

Action (in activities)	count	should reference
ChangeObjectAction	17	entity / member
DeleteObjectAction	3	entity
NanoflowCallAction	3	nanoflow
RestCallAction / RestOperationCallAction	1 + 1	consumed REST service
CastAction	1	entity

Also unhandled in general: AggregateListAction, CallExternalAction, CommitAction, and RetrieveAction via AssociationRetrieveSource.

Gap 2 — associations: 52 in project, 0 associate refs

Explicit TODO at builder_references.go:168 (// Association references require resolving ChildID … Skipping for now). The associate RefKind constant exists but is never written.

Gap 3 — page layout / widget / datasource / action refs: all 0

refs has 0 rows for layout, datasource, action even though the pages table proves layouts exist (LayoutRef populated, e.g. ActiveSessions → Atlas_Core.Atlas_Default) and there are 36 action widgets. Root cause: extractWidgetRefs / extractDataSourceRefs / layout-ref emission are all gated behind if pg.LayoutCall != nil (builder_references.go:182–195), and LayoutCall is unpopulated on the cached pages, so the whole widget subtree is skipped. Only page parameter refs (from pg.Parameters) survive.

Effect

show callers/callees/references/impact silently under-report — e.g. "who calls this nanoflow", "what uses this REST service", "which pages use this layout", "what references this association" all miss results.

Fix plan (incremental, with tests + bug-test MDL where applicable)

1. Microflow action refs — add the missing reference-bearing action types (change/delete/nanoflow-call/rest/cast/aggregate/commit/cast, association-retrieve source).
2. Association refs — resolve association target entity (ChildID → entity QN) and emit associate refs.
3. Page widget refs — fix the LayoutCall-gating so layout/datasource/widget/action refs emit regardless of whether the page is parsed via a layout call.

References

• mdl/catalog/builder_references.go
• Sibling fix for the same anti-pattern: 6574e9c (catalog.activities)

[github-actions]

Issue Triage

Summary:
The issue reports that the refs table built by mxcli is incomplete – it omits many microflow‑action references, all association references, and page‑layout/widget/data‑source/action references, causing show callers/callees/references/impact commands to miss results.

Checklist of required information for a bug report

Required info	Present?	Notes
mxcli version (output of mxcli --version or commit/build)	❌	Not mentioned
Mendix version of the project being processed	❌	Not mentioned
Scenario – what the user was trying to do (CLI command, MDL script, etc.)	❌	No explicit command or workflow described
Expected output – what should have happened	❌	Implied but not stated explicitly
Experienced output – what actually happened (error, wrong result, crash)	❌	Only describes missing data, not observed output
AI bug report – output of mxcli diag --bundle or session log	❌	Not provided

What’s missing:
To properly evaluate and reproduce the issue, please add the following details:

1. The exact version of mxcli you are using (e.g., mxcli --version output or commit hash).
2. The Mendix version of the project you processed (e.g., 10.6.0, 9.24.1).
3. The specific CLI command(s) you ran (e.g., mxcli refresh catalog full, mxcli show callers …).
4. What you expected to see in the refs table or the show … output.
5. What you actually observed (e.g., zero rows for certain RefKinds, missing entries).
6. If possible, a diagnostic bundle (mxcli diag --bundle) or the relevant session log.

Once we have that information, we can investigate the gaps you’ve highlighted and work toward a fix. Thank you for bringing this to our attention!

---

Automated triage via OpenRouter — workflow source

[github-actions]

Enhancement Implementation Guide

We are given an enhancement request about the catalog refs table being incomplete. The request identifies three gaps:

Gap 1: Microflow action refs - only 5 of ~15 reference-bearing action types are handled.
Gap 2: Associations: 0 associate refs (TODO in code).
Gap 3: Page layout / widget / datasource / action refs: all 0 due to gating on LayoutCall.

The fix plan is incremental:

1. Add missing microflow action refs (change/delete/nanoflow-call/rest/cast/aggregate/commit/cast, association-retrieve source).
2. Fix association refs by resolving ChildID to entity QN and emitting associate refs.
3. Fix page widget refs by removing the LayoutCall gating so that layout/datasource/widget/action refs emit regardless of whether the page is parsed via a layout call.

We are to provide an implementation guide following the full-stack MDL pipeline.

However, note that the enhancement request is about the internal catalog builder (which is used for the show callers|callees|references|impact|context of ... commands) and not about adding new MDL syntax. Therefore, we are not adding a new MDL feature but fixing an existing internal mechanism.

Given the context, we must adjust our approach: the issue is in the mdl/catalog/builder_references.go file. We are to extend the reference extraction to cover the missing cases.

But note: the PR review checklist and the implementation pipeline described in CLAUDE.md are for MDL features (which involve grammar, AST, visitor, executor, etc.). This issue is about an internal catalog builder, which is not exposed as an MDL command. Therefore, we do not need to follow the full MDL implementation pipeline (grammar, AST, etc.) because we are not adding a new MDL statement.

Instead, we are modifying an existing Go file that builds the catalog. The catalog is built by the refresh catalog full command (which is an existing MDL command). So we are fixing the implementation of an existing command.

Therefore, we must:

1. Check if the issue has enough detail: Yes, it does. We have specific gaps and a fix plan.

2. Overlap check: We are not adding a new MDL feature, so we check if the existing refresh catalog command already does this (it doesn't, hence the issue).

3. Proposed MDL Syntax: Not applicable because we are not changing the MDL syntax. The refresh catalog full command remains the same.

4. Implementation Roadmap: We will modify the mdl/catalog/builder_references.go file to add the missing reference extraction.

5. BSON Investigation: We need to understand the BSON structure for the various Mendix elements (activities, associations, pages) to extract references correctly.

6. Testing Strategy: We will add tests to verify that the references are now extracted correctly. We can use the mx-test-projects/MxGraphStudioDemo project mentioned in the issue.

7. Gotchas & Risks: We must be careful about storage names vs qualified names, and ensure we are using the correct BSON field names.

Let's break down the implementation roadmap for the three gaps:

Gap 1: Microflow action refs

• We need to add reference extraction for the missing action types in the buildReferences function in builder_references.go.
• The missing action types are: ChangeObjectAction, DeleteObjectAction, NanoflowCallAction, RestCallAction / RestOperationCallAction, CastAction, AggregateListAction, CallExternalAction, CommitAction, and RetrieveAction via AssociationRetrieveSource.
• We note that the current code already handles: MicroflowCallAction, CreateObjectAction, RetrieveAction (only DatabaseRetrieveSource), ShowPageAction, JavaActionCallAction.
• We will add cases for each missing action type in the switch statement in extractActivityRefs (or similar) function.

Gap 2: Association refs

• There is a TODO at builder_references.go:168 about association references requiring resolving ChildID.
• We need to implement the extraction of association references (RefKind: associate) by resolving the ChildID (which is the TO entity) to a qualified name and then emitting a reference from the association to the target entity.
• Note: The association itself is stored in the domain model. We have the association's ChildID (which is the ID of the TO entity) and ParentID (FROM entity). We want to record that the association references the TO entity (ChildID) and the FROM entity (ParentID)? But note the issue says: "Association refs — resolve association target entity (ChildID → entity QN) and emit associate refs."
• Looking at the existing code, we see that the refs table is built for various elements. For an association, we want to record that the association references the entity it points to (both ends?).
• However, note the issue says: "Explicit TODO at builder_references.go:168 (// Association references require resolving ChildID … Skipping for now)". So we are to emit a reference of kind associate from the association to the entity (the TO entity?).
• We must look at the BSON structure of an association to get the ChildID and ParentID, then resolve those IDs to qualified names (using the catalog's object index or by scanning the domain model).

Gap 3: Page widget refs

• The issue states that the extraction of layout, datasource, widget, and action refs is gated behind if pg.LayoutCall != nil (lines 182-195). Since LayoutCall is unpopulated on cached pages, the entire widget subtree is skipped.
• We must remove this gate so that we process the widget subtree regardless of whether the page is via a layout call.
• We note that the page's parameters refs (from pg.Parameters) are already being extracted and are not gated.

Implementation steps:

We will focus on the file: mdl/catalog/builder_references.go

We will:

1. For Gap 1: Extend the switch in extractActivityRefs (or wherever activity refs are extracted) to handle the missing action types.
2. For Gap 2: Implement the association reference extraction by resolving the ChildID (and possibly ParentID) to qualified names and emitting associate refs.
3. For Gap 3: Remove the if pg.LayoutCall != nil condition and process the widget subtree unconditionally.

We must also consider that the catalog builder uses a cache of objects (pages, microflows, etc.) and we have access to the qualified names via the cache.

Let's outline the changes:

For Gap 1:
In builder_references.go, we have a function extractActivityRefs (or similar) that takes an activity and extracts references. We will add cases for the missing action types.

Example for ChangeObjectAction:

case *domainmodels.ChangeObjectAction:
    // Extract reference to the entity and possibly members (attributes/associations) being changed.
    // The action has an `Entity` field (which is an association to an entity) and a list of `Changes`.
    // Each change has a `Member` field (which is an association to an attribute or association).
    // We want to record:
    //   - reference to the entity (kind: entity)
    //   - for each change, reference to the member (kind: attribute or association)

We must look at the BSON structure of these actions to see what fields to use.

For Gap 2:
We need to find where associations are processed. There is likely a function extractAssociationRefs or similar. We see a TODO at line 168.

We will:

• Get the association's ChildID and ParentID (from the BSON).
• Resolve these IDs to qualified names (using the cache of objects, which we have access to via b.ObjectCache or similar).
• Emit two references? Or just one? The issue says "associate refs", and the RefKind associate exists.
• Looking at the existing code, we see that the refs table has a RefKind column. We have constants for different kinds (like call, create, etc.). We see associate is defined but not used.

We should emit a reference from the association to the entity that is the target (ChildID) and/or the source (ParentID)? The issue says "resolve association target entity (ChildID → entity QN)". So we are only emitting for the ChildID (the TO entity) as an associate ref? But note: an association has two ends. We might want to record both? However, the issue only mentions the target.

But note: the existing code for other references (like from a microflow action to an entity) records a reference from the action to the entity. Similarly, for an association, we want to record that the association references the entity (so that if we change the entity, we know which associations are affected).

We will emit one reference per end? Or one reference for the association to the entity? The issue says "associate refs", and the RefKind is associate. We'll follow the pattern: for each end, we emit a reference of kind associate from the association to the entity.

However, note that the association itself is stored as an element in the domain model. We are building references for the association element. We want to know what entities the association refers to.

So for an association, we will emit:

• A reference from the association to the Parent entity (FROM) with RefKind: associate
• A reference from the association to the Child entity (TO) with RefKind: associate

But wait: the issue says "target entity (ChildID → entity QN)". So maybe they only want the Child? Let's look at the existing code for other references: when we have a retrieve action, we reference the entity being retrieved. Similarly, for an association, we might want to reference the entity that is the target of the association (the one we are pointing to). However, an association is bidirectional? In Mendix, an association has an owner (the FROM entity) and a referenced entity (the TO entity). The reference is from the FROM to the TO.

But note: the issue says "what references this association" — meaning we want to know what points to the association. However, the refs table is built to power show callers|callees|references|impact|context of .... So if we want to know what references an association, we would look for references that point to the association. But the issue is about the association referencing entities? Actually, the issue says: "what references this association" — meaning we want to know what uses the association (so that if we change the association, we know what is affected). But the refs table is built as a best-effort index of references FROM an element TO another element.

Looking at the existing code: when we have a microflow that calls another microflow, we record a reference from the caller microflow to the callee microflow (RefKind: call). So the refs table stores: (source element, target element, ref kind).

Therefore, for an association, we want to record:

• The association references the Parent entity (FROM) and the Child entity (TO). But note: the association itself is the source, and the entities are the targets.

However, the issue says: "resolve association target entity (ChildID → entity QN) and emit associate refs". So they are only emitting for the ChildID? But note the ParentID is also an entity.

Given the ambiguity, we should look at the existing code for how other references are emitted. For example, in a retrieve action, we record a reference to the entity being retrieved (the target of the retrieve). Similarly, in an association, we might want to record a reference to the entity that is the target of the association (the TO entity) because that is the entity we are referencing via the association.

But note: the association is stored as an element that has two pointers: ParentPointer and ChildPointer. The ParentPointer points to the FROM entity (the owner) and the ChildPointer points to the TO entity (the referenced). In the domain model, the association has ParentID and ChildID.

We want to know: if we change the TO entity, what associations are affected? So we would want to record that the association references the TO entity (via ChildID). Similarly, if we change the FROM entity, what associations are affected? So we would also want to record that the association references the FROM entity (via ParentID).

Therefore, we should emit two references per association: one for the ParentID and one for the ChildID, both with RefKind: associate.

But note: the issue only mentions ChildID. However, the TODO says "Association references require resolving ChildID", implying that they were going to do ChildID and maybe ParentID was already handled? Let's check the code around line 168.

Without the actual code, we must rely on the issue. Since the issue says "all associations: 52 in project, 0 associate refs", we know that currently no association refs are emitted. We are to fix that.

We will emit references for both ends? Or just the Child? To be safe, we should emit for both ends because the association refers to two entities.

However, note that the refs table is used for impact analysis: if I change an entity, what is impacted? We want to know the associations that refer to that entity (either as owner or as referenced). So we need both.

But let's see what the existing code does for other bidirectional references? For example, an attribute of type enumeration: we record a reference from the attribute to the enumeration. Similarly, we would record a reference from the association to the entity (for each end).

We will do:

• For the ParentID: emit a reference from the association to the Parent entity (RefKind: associate)
• For the ChildID: emit a reference from the association to the Child entity (RefKind: associate)

Gap 3:
We need to remove the gate that prevents processing the widget subtree when pg.LayoutCall is nil.

In builder_references.go, we see:

if pg.LayoutCall != nil {
    // extractWidgetRefs / extractDataSourceRefs / layout-ref emission
    // ... 
}

We want to remove this condition and always process the widget subtree.

We must also note that the page might not have a layout call (if it's a snippet or a page without a layout) but still have widgets. We want to extract references from those widgets.

We will move the widget and datasource extraction outside the condition.

Now, let's write the implementation roadmap:

Implementation Roadmap:

1. Gap 1: Microflow action refs

   • File: mdl/catalog/builder_references.go
   • Function: extractActivityRefs (or similar, we need to find the function that processes activities)
   • We will add cases for the missing action types in the switch statement.
   • For each action type, we will extract the relevant references (to entities, members, nanoflows, etc.) and add them to the reference builder.

   Example for ChangeObjectAction:

   case *domainmodels.ChangeObjectAction:
       // Reference to the entity being changed
       if act.Entity != nil {
           rb.AddRef(act, act.Entity.QualifiedName(), refs.EntityRef)
       }
       // References to the members (attributes/associations) being changed
       for _, change := range act.Changes {
           if change.Member != nil {
               rb.AddRef(act, change.Member.QualifiedName(), refs.MemberRef)
           }
       }

   We must do similarly for the other action types.

   Note: We must use the correct RefKind constants. We see existing ones like refs.EntityRef, refs.MemberRef, etc. We should check what constants are available.

   We also need to handle:

   • DeleteObjectAction: similar to ChangeObjectAction but without changes? It has an Entity.

   • NanoflowCallAction: has a Nanoflow field -> reference to the nanoflow (RefKind: NanoflowRef? or CallRef? We see existing call refs for microflows and java actions. We should use the same kind as for microflow calls? Let's see: in the existing code, for MicroflowCallAction we see:

     case *microflows.MicroflowCallAction:
         if act.Microflow != nil {
             rb.AddRef(act, act.Microflow.QualifiedName(), refs.MicroflowRef)
         }

     So we have refs.MicroflowRef. We should check if there is a refs.NanoflowRef or if we use the same? We see in the code that nanoflows are treated similarly to microflows in some places. We can check the constants.

   • RestCallAction and RestOperationCallAction: These have a HttpConfiguration which may have a template that contains constants or variables? But the issue says they should reference the consumed REST service. How is the REST service represented? In the microflow, the REST call action does not directly reference a REST service document; instead, it has the configuration inline. However, note that there is also a Consumed REST Service document in the project. The REST call action might be generated from a consumed REST service, but in the microflow it is inline.

     The issue says: "RestCallAction / RestOperationCallAction | 1 + 1 | consumed REST service". So they want to reference the consumed REST service document. How do we get that?

     We note that in the microflow, the REST call action does not have a direct reference to a consumed REST service. Instead, the consumed REST service is used to generate the microflow action. But in the microflow, the action is stored with the configuration.

     However, looking at the Mendix model, there is a ConsumedRestService document and the microflow action might have a reference to it? We must check the BSON.

     Alternatively, the issue might be mistaken? But the issue says: "Verified on mx-test-projects/MxGraphStudioDemo". We should look at that project.

     Given the time, we will assume that the REST call action does not directly reference a consumed REST service document. Therefore, we cannot extract a reference to the consumed REST service from the microflow action. However, note that the issue

---

Automated guide via OpenRouter — workflow source

[ako]

Progress — gaps 1 & 2 fixed, gap 3 partial. Branch misc, verified on MxGraphStudioDemo (refresh catalog full).

RefKind breakdown before → after the three fixes:

RefKind	before	after
associate	0	104
layout	0	22
call	13	20
retrieve	1	5
show_page	5	10

Gap 1 — microflow/nanoflow action refs (b494bf566). Pulled per-action extraction into a pure, unit-tested helper microflowActionRef() and added the dropped reference-bearing actions: NanoflowCallAction→NANOFLOW, RestOperationCallAction→REST_OPERATION, RetrieveAction via AssociationRetrieveSource→ASSOCIATION. Also now iterate nanoflows as ref sources (SourceType=NANOFLOW) — they were skipped entirely, so e.g. nanoflow→nanoflow calls produced nothing. Variable-target actions (change/delete/commit/cast) correctly emit no ref. TestMicroflowActionRef covers every case.

Gap 2 — association refs (940115f75). Removed the "Skipping for now" TODO; resolve each association's FROM (ParentID) and TO (ChildID) entity via the entities table and emit an associate ref to each endpoint (cross-module assocs use ChildRef for the remote side). 0 → 104 (52 associations × 2 endpoints), discoverable from either entity.

Gap 3 — page refs (65024c540, PARTIAL). Root cause was deeper than expected: ListPages() never populates pg.LayoutCall (the parser sets only LayoutID), so the entire if pg.LayoutCall != nil block — layout and all widget/datasource refs — was dead code. Fixed the layout ref by reading the layout name the pages table already resolves from raw BSON (resolvePageLayoutRef): 0 → 22.

Remaining (narrowed scope of this issue):

• Widget-level datasource/action refs. The extractWidgetRefs/extractDataSourceRefs code walks pages.Widget structs, but no reader exposes a parsed page widget tree (ListPages returns metadata-only pages; the widgets catalog table is built from raw BSON via extractPageWidgets). Reviving widget refs needs either a parsed-widget-tree reader method or raw-BSON datasource/action extraction. Left as scaffolding with a NOTE.
• Layout extraction is itself partial — only 22 of 68 pages have a resolvable LayoutRef (the pages-table extractLayoutRef only handles certain FormCall shapes). That's a buildPages limitation upstream of refs; worth a separate look.

[ako]

Follow-up resolved — layout extraction now complete (fd6f10543).

The 22/68 partial layout coverage was page templates: the 46 pages without a LayoutRef were all Forms$PageTemplate documents (Atlas_Web_Content building blocks), which nest their layout under LayoutCall instead of the FormCall that regular Forms$Form pages use. Both nest the same Form (string QN) field, so extractLayoutRef now falls back to LayoutCall.

Result on MxGraphStudioDemo: pages with a resolved layout 22 → 68/68; refs layout edges 22 → 68 (e.g. Phone_Detail → Atlas_Core.Phone_Default).

Remaining on this issue: only the widget-level datasource/action refs, which need a parsed page widget tree no reader yet exposes.

[ako]

Refs completion — steps 1 & 2 landed (branch misc). These close most of the remaining edge gaps, raising the quality floor for the graph-analysis work that depends on refs.

Step 1 — calculated-by + intra-flow change/delete (cf7cd473d). Piggybacked on existing walks (no new parse):

• calculate (ENTITY → MICROFLOW) from attr.Value.MicroflowName — verified on LatoProductInventory (Inventory.Brand → CountBrandStock).
• change / delete (→ ENTITY): these target a variable, so they were dropped before. Added lightweight intra-flow variable→entity resolution (seed from object-typed params, track create/retrieve outputs in one linear pass). Evora: +405 change, +96 delete.

Step 2 — page-widget refs (6a719f598). The previously-dead widget path:

• datasource (PAGE → ENTITY) and action (PAGE → MICROFLOW/NANOFLOW), extracted by scanning each widget's own raw BSON during the existing buildPages walk (entity via DomainModels$DirectEntityRef.Entity, microflow/nanoflow via the Microflow/Nanoflow fields), stored in new widgets_data.MicroflowRef/NanoflowRef columns, and projected into refs via a DISTINCT INSERT…SELECT. Deterministic (lexicographically-smallest pick). LatoProductInventory: 66 datasource + 105 action; Evora 6082 → 6459, identical across two refreshes.

refs totals over the whole effort: ~5.5k (pre-#663) → 6,459 on Evora.

Remaining (smaller):

• Snippet widgets — snippets don't yet feed widgets_data, so snippet→entity/microflow refs aren't captured. Same mechanism as step 2; follow-up.
• Attribute-level nodes — widget AttributeRef is captured but attributes aren't yet first-class graph nodes.

[ako]

Snippet-widget refs done (effcbcefe). buildSnippets now extracts snippet widgets into widgets_data with ContainerType=SNIPPET (it previously stored WidgetCount=0 and extracted nothing), and the existing refs projection emits SNIPPET → datasource/action edges with no further change. Verified on LatoProductInventory: 520 snippet widgets → 73 action + 5 datasource refs; deterministic across refreshes.

With this, the widget-ref gap is closed for both pages and snippets. The only remaining item on this issue is attribute-level nodes (widget AttributeRef is captured but attributes aren't yet first-class graph nodes) — low priority, surfaceable later if a consumer needs qualified-attribute resolution.