Skip to content

fix(storage): persist MeshNode.SyncBehavior in Postgres (decouple survives restart) - #119

Merged
rbuergi merged 1 commit into
mainfrom
pg-persist-syncbehavior
Jun 30, 2026
Merged

rbuergi merged 1 commit into
mainfrom
pg-persist-syncbehavior

Conversation

@rbuergi

@rbuergi rbuergi commented Jun 29, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The static-repo "Not synced" decouple claim (MeshNode.SyncBehavior) had no Postgres column. PostgreSqlStorageAdapter persists only the typed content JSONB plus a fixed set of metadata columns (state, version, …) — SyncBehavior was dropped on write and the row→node mapper never read it, so every reload from PG defaulted to Include.

Consequence: a partition an admin decoupled via Admin → Partition Sync (or per-node Stop Sync) silently re-coupled on the next restart/redeploy, and the next static-repo import re-clobbered admin-managed content. This is the root cause behind the client-i Provider/Anthropic API key reverting to the shared Azure Foundry default (2026-06-29): the key was entered + the partition decoupled, but the decouple evaporated on the next deploy and the importer re-seeded our key.

The decouple feature appeared to work because SyncBehavior survives in-memory (the live MeshNodeStreamCache keeps the full node) — it just never made it to storage. Sqlite was unaffected: it serializes the whole node as JSON, so the claim round-trips there. The bug is Postgres-only.

Fix

Persist SyncBehavior as a mesh_nodes column (it mirrors state — a node-level metadata enum), the only table whose nodes participate in static-repo sync:

  • Schema — add sync_behavior SMALLINT NOT NULL DEFAULT 0 to mesh_nodes in every provisioning path of PostgreSqlSchemaInitializer.
  • Migration V39 — ALTER TABLE … ADD COLUMN IF NOT EXISTS across every existing schema (public + auth + all partitions), idempotent, one dynamic-SQL pass.
  • Adapter — write it in the UPSERT (mesh_nodes only — satellite tables don't carry it), surface it on the single-node / prefix reads (satellite-safe 0 AS sync_behavior so UNION branches stay aligned), and read it via a defensive mapper (PgMeshNodeReader.ReadSyncBehavior — column absent ⇒ Include) shared by the storage adapter and the cross-schema query provider. The defensive read means no existing SELECT that omits the column can break.

The authoritative decouple read (StaticRepoImporter.ReadClaimedRoots → GetMeshNodeStream → the adapter's exact read) now returns the persisted claim, so a decoupled partition root durably skips import across restarts.

Repro test

SyncBehaviorPersistenceTests (Testcontainers PG): round-trips ExcludeThisAndChildren through PostgreSqlStorageAdapter.Write/Read. Fails before this change (mapper returns Include), passes after. Plus a default-Include case.

Verification

  • MeshWeaver.Hosting.PostgreSql.Test: 509 passed, 1 skipped, 0 failed (the change touches core read/write SELECTs, so the whole PG suite is the regression gate).
  • Migration + PG projects build clean (0 warnings).

Scope notes

  • Whole-partition decouple (the documented Admin → Partition Sync flow) is fully covered: the partition-root claim is read authoritatively and skips the entire subtree.
  • The mesh_node_history trigger and auth-mirror trigger are intentionally left untouched (they insert explicit column lists; history/audit doesn't need the claim).
  • Also includes the /provider-keys skill (src/MeshWeaver.AI/Data/Skill/) documenting AI provider key administration — the decouple step it prescribes is now durable.

🤖 Generated with Claude Code

…vives restart)

The static-repo "Not synced" decouple claim (MeshNode.SyncBehavior) had no
Postgres column. PostgreSqlStorageAdapter persisted only the typed content plus
a fixed metadata column set, so SyncBehavior was dropped on write and the
row->node mapper defaulted every reload to Include. A partition an admin
decoupled therefore silently re-coupled on the next restart, and the next
static-repo import re-clobbered admin-managed content — e.g. a customer's
Provider/Anthropic API key reverting to the shipped Azure Foundry default
(atioz, 2026-06-29). Sqlite was unaffected: it serializes the whole node as JSON.

- Add sync_behavior SMALLINT column to mesh_nodes (every provisioning path in
  the schema initializer) + V39 migration (ADD COLUMN IF NOT EXISTS across every
  existing schema: public, auth, and all partitions).
- Write it in the UPSERT (mesh_nodes only — satellite tables don't carry it),
  surface it on the single-node/prefix reads (satellite-safe `0 AS sync_behavior`
  in the UNION branches), and read it via a defensive mapper (column absent ->
  Include) shared by the storage adapter and the cross-schema query provider.
- Repro: SyncBehaviorPersistenceTests round-trips ExcludeThisAndChildren through
  the PG adapter (fails before this change, which mapped it back to Include).
- Docs: add the /provider-keys skill for administering AI provider keys.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 1)

1 068 tests  +10   1 066 ✅ +10   4m 57s ⏱️ +21s
   12 suites ± 0       2 💤 ± 0 
   12 files   ± 0       0 ❌ ± 0 

Results for commit c16f077. ± Comparison against base commit 9e8ee46.

@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 2)

 12 files  ±  0   12 suites  ±0   5m 41s ⏱️ -13s
734 tests  - 115  733 ✅  - 115  1 💤 ±0  0 ❌ ±0 
736 runs   - 140  735 ✅  - 140  1 💤 ±0  0 ❌ ±0 

Results for commit c16f077. ± Comparison against base commit 9e8ee46.

This pull request removes 115 tests.
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.AgenticAI.md")
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.ExecuteScript.md")
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.ExecutiveAssistan"···)
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.McpAuthentication"···)
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.ModelProviderSetu"···)
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.PlatformProviderS"···)
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.ProviderConfigura"···)
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.Tools.MeshPlugin."···)
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.AI.md")
MeshWeaver.Persistence.Test.DocumentationCodeBlockCompilationTest ‑ ExecutedCsharpBlocks_MustCompile(embeddedResourceName: "MeshWeaver.Documentation.Data.Architecture.ActionB"···)
…

@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 3)

1 173 tests  ±0   1 173 ✅ ±0   5m 22s ⏱️ +13s
   11 suites ±0       0 💤 ±0 
   11 files   ±0       0 ❌ ±0 

Results for commit c16f077. ± Comparison against base commit 9e8ee46.

@github-actions

Copy link
Copy Markdown
Contributor

Test Results (shard 0)

1 576 tests   - 654   1 576 ✅  - 649   3m 13s ⏱️ - 5m 6s
   11 suites  -   1       0 💤  -   5 
   11 files    -   1       0 ❌ ±  0 

Results for commit c16f077. ± Comparison against base commit 9e8ee46.

This pull request removes 654 tests.
MeshWeaver.AI.Test.ActivityLogStreamTest ‑ Progress_Messages_Stream_Gradually_Not_Just_At_The_End
MeshWeaver.AI.Test.ActivityLogStreamTest ‑ Script_Failure_Flips_ActivityLog_Status_To_Failed
MeshWeaver.AI.Test.ActivityLogStreamTest ‑ Script_Log_Messages_Land_On_ActivityLog_Node
MeshWeaver.AI.Test.AgentChatClientDeadlockTest ‑ GetOrderedAgentsAsync_WithContextPath_ConcurrentCallers_DoNotDeadlock
MeshWeaver.AI.Test.AgentChatClientDeadlockTest ‑ GetOrderedAgentsAsync_WithContextPath_SingleCaller_ResolvesQuickly
MeshWeaver.AI.Test.AgentChatClientDeadlockTest ‑ GetOrderedAgentsAsync_WithMarkdownContext_DoesNotDeadlock
MeshWeaver.AI.Test.AgentChatClientTest ‑ AgentChatClient_Initialize_SurfacesPartitionAgents_NotNodeTypeNamespaceAgents
MeshWeaver.AI.Test.AgentChatClientTest ‑ AgentChatClient_Initialize_SurfacesPlatformAgentsForAnyContext
MeshWeaver.AI.Test.AgentChatClientUnitTest ‑ FindCyclicDelegations_Chain_ReturnsEmpty
MeshWeaver.AI.Test.AgentChatClientUnitTest ‑ FindCyclicDelegations_DelegationsWithNoMatchingTarget_ReturnsEmpty
…

@github-actions

Copy link
Copy Markdown
Contributor

Test Results

   46 files   -   1     46 suites   - 1   19m 14s ⏱️ - 4m 45s
4 551 tests  - 759  4 548 ✅  - 754  3 💤  - 5  0 ❌ ±0 
4 553 runs   - 784  4 550 ✅  - 779  3 💤  - 5  0 ❌ ±0 

Results for commit c16f077. ± Comparison against base commit 9e8ee46.

This pull request removes 769 and adds 10 tests. Note that renamed tests count towards both.
MeshWeaver.AI.Test.ActivityLogStreamTest ‑ Progress_Messages_Stream_Gradually_Not_Just_At_The_End
MeshWeaver.AI.Test.ActivityLogStreamTest ‑ Script_Failure_Flips_ActivityLog_Status_To_Failed
MeshWeaver.AI.Test.ActivityLogStreamTest ‑ Script_Log_Messages_Land_On_ActivityLog_Node
MeshWeaver.AI.Test.AgentChatClientDeadlockTest ‑ GetOrderedAgentsAsync_WithContextPath_ConcurrentCallers_DoNotDeadlock
MeshWeaver.AI.Test.AgentChatClientDeadlockTest ‑ GetOrderedAgentsAsync_WithContextPath_SingleCaller_ResolvesQuickly
MeshWeaver.AI.Test.AgentChatClientDeadlockTest ‑ GetOrderedAgentsAsync_WithMarkdownContext_DoesNotDeadlock
MeshWeaver.AI.Test.AgentChatClientTest ‑ AgentChatClient_Initialize_SurfacesPartitionAgents_NotNodeTypeNamespaceAgents
MeshWeaver.AI.Test.AgentChatClientTest ‑ AgentChatClient_Initialize_SurfacesPlatformAgentsForAnyContext
MeshWeaver.AI.Test.AgentChatClientUnitTest ‑ FindCyclicDelegations_Chain_ReturnsEmpty
MeshWeaver.AI.Test.AgentChatClientUnitTest ‑ FindCyclicDelegations_DelegationsWithNoMatchingTarget_ReturnsEmpty
…
MeshWeaver.Hosting.Monolith.Test.AgentSyncedQueryFromHostedHubTest ‑ HostedSubHub_GetQuery_AgentDropdownNamesAreNonEmpty
MeshWeaver.Hosting.Monolith.Test.AgentSyncedQueryFromHostedHubTest ‑ HostedSubHub_GetQuery_ReturnsAgentsAndModels
MeshWeaver.Hosting.Monolith.Test.CodeEditRecompileTest ‑ IsDirty_FlipsTrueOnSourceEdit_FalseAfterCompile
MeshWeaver.Hosting.Monolith.Test.CodeEditRecompileTest ‑ NodeType_RequestedReleasePath_PinsToHistoricalRelease
MeshWeaver.Hosting.Monolith.Test.CodeEditRecompileTest ‑ PressingCompileButton_SetsRequestedReleaseAt_AndProducesNewRelease
MeshWeaver.Hosting.Monolith.Test.CompileActivityNoPhantomPathTest ‑ CompiledNodeType_LastCompilationActivityPath_PointsAtAnExistingActivityNode
MeshWeaver.Hosting.Monolith.Test.CompileActivityNoPhantomPathTest ‑ SubscribingAbsentCompileActivity_InTightLoop_StaysResponsive_NoStorm
MeshWeaver.Hosting.Monolith.Test.KernelScriptMemoryLeakTest ‑ ScriptState_IsCollected_AfterMeshAndServiceProviderDisposal
MeshWeaver.Hosting.PostgreSql.Test.SyncBehaviorPersistenceTests ‑ DecoupleClaim_SurvivesReloadFromStorage
MeshWeaver.Hosting.PostgreSql.Test.SyncBehaviorPersistenceTests ‑ UnclaimedNode_DefaultsToInclude

@rbuergi
rbuergi merged commit 327ad27 into main Jun 30, 2026
11 checks passed
@rbuergi
rbuergi deleted the pg-persist-syncbehavior branch June 30, 2026 04:52
rbuergi added a commit that referenced this pull request Jun 30, 2026
#119 added src/MeshWeaver.AI/Data/Skill/provider-keys.md (an 8th built-in skill)
but left the two exact-set assertions in SkillNodeTypeTest +
SkillHarnessImportSourceTest at the prior 7, turning main red. Add provider-keys
(sorts after model) so the assertions match the shipped Data/Skill/*.md set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rbuergi added a commit that referenced this pull request Jun 30, 2026
SkillNodeTypeTest.BuiltInSkillProvider_ShipsStandardSkills_AsPickSkillNodes and
SkillHarnessImportSourceTest.SkillStaticRepoSource_Emits_BuiltInSkills_Under-
SkillPartition assert the exact set of shipped built-in skills with an ordered
.Equal(...). #119 added a new built-in skill (src/MeshWeaver.AI/Data/Skill/
provider-keys.md) but didn't update these two expected lists, so both fail
DETERMINISTICALLY on main (reproduced locally: actual has 8 skills incl.
"provider-keys", expected 7). The shipped behaviour is correct — the tests were
stale. Append "provider-keys" (alphabetical) to both expected lists.

Verified: both tests pass (2/2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
rbuergi added a commit that referenced this pull request Jun 30, 2026
…e reduce (#112)

* fix(sync): JsonElement value-equality in EntityUpdate dedup + MeshNode reduce

Two JsonElement-as-object defects in the sync core:

FIX 1 — JsonSynchronizationStream.ToEntityUpdates (both overloads):
DistinctBy(x => new { x.Id, x.Collection }) never deduped. x.Id is the result
of DecodePointerSegment → JsonSerializer.Deserialize<object> → a boxed
JsonElement, whose struct (_parent/_idx cursor) has no value equality, so the
anonymous key compared every id as distinct and duplicate per-entity
EntityUpdates leaked into the stream (contradicting the documented "one distinct
update per affected entity"). Key on je.GetRawText() (stable; ToString() stays
stable for string/guid ids).

FIX 2 — MeshDataSource.ReduceToMeshNode patch path:
`change.Value as MeshNode` returned null when change.Value is a JsonElement
(the cross-hub / mirror patch path), pushing a null-content ChangeItem into the
mirror. Deserialize the JsonElement to MeshNode the same way the sibling
PatchMeshNode does (threaded the hub's JsonSerializerOptions into the reducer).

Repros: EntityUpdateDedupTest (count 2 → 1), ReduceToMeshNodeJsonElementTest
(null → node). Full Data.Test (254) + Graph.Test (436) green under
DOTNET_PROCESSOR_COUNT=2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(ai): include /provider-keys in the built-in skill-set assertions

#119 added src/MeshWeaver.AI/Data/Skill/provider-keys.md (an 8th built-in skill)
but left the two exact-set assertions in SkillNodeTypeTest +
SkillHarnessImportSourceTest at the prior 7, turning main red. Add provider-keys
(sorts after model) so the assertions match the shipped Data/Skill/*.md set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant