Repository navigation
fix(#354): boot self-heal asserts served AI catalog partitions are search-indexed - #524
Merged
Merged
Conversation
…arch-indexed The config-alias/allow-list drift that made the global AI catalogs (Provider/Agent/Skill/Harness) render empty was fixed by #407. This closes the live RESIDUAL: nothing at boot asserted that a served catalog partition actually landed in the cross-schema search index (public.searchable_schemas). Because the static-repo import's own queries are all partition-PINNED, they never trigger a SyncSearchableSchemasAsync, so a freshly-provisioned catalog schema stays absent from searchable_schemas until the first unpinned fan-out query happens to run one. Meanwhile every index query (search nodeType:ModelProvider, children listing, the catalog areas) returns empty although exact-path get works — "materialized but unindexed". Fix (additive, idempotent, reactive): - StaticRepoImporter.ImportAll takes the served catalog set and, after the import, runs AssertCatalogPartitionsIndexed: for each catalog actually backed by a registered source this run, FORCE one searchable_schemas rebuild (bypassing the query-hot-path throttle — a one-time boot action, never per-query) so a just-provisioned schema is indexed immediately, then read the registry back and surface a startup-failure notification (existing NotifyStartupFailure path) for any served catalog still absent. In-memory backends and non-served catalogs are left untouched. PG round-trips run on the IIoPool; no Observable.FromAsync, no await. - ICrossSchemaQueryProvider gains SyncSearchableSchemasAsync(bool force, ct); the PG + Snowflake impls bypass the TTL when force=true (single-flight is still honoured). Existing callers are unchanged. - Portal boot passes AiContentSources.ContentPartitions as the assertion set. Test (PG Testcontainers): CatalogPartitionIndexSelfHealTests pins that a served catalog is registered in searchable_schemas + a cross-schema search finds its seeded nodes, and a revert-proven case where a dropped searchable_schemas row is re-registered by the next boot's self-heal even inside the throttle window (the force=true bypass is what makes it deterministic). Ran locally, 2/2 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Contributor
Contributor
Contributor
Contributor
Contributor
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the live residual of #354 (the config drift was already fixed by #407; this adds the boot-time self-heal/assert that served AI catalog partitions are provisioned + indexed).
The residual
On affected instances the global AI catalog partitions (
Provider/Agent/Skill/Harness) are materialized but missing from the search index: exact-pathget @Provider/Anthropicworks, but every index-based access (search nodeType:ModelProvider,@Provider/*children listing, the AI-menu catalog areas) returns empty. #407 fixed the config-alias/allow-list drift that stopped the catalogs being imported. What remained: nothing at boot asserted that a served catalog actually landed inpublic.searchable_schemas.Root cause of the residual: the static-repo import's own queries are all partition-pinned, so they never trigger a
SyncSearchableSchemasAsync. A freshly-provisioned catalog schema therefore stays absent fromsearchable_schemasuntil the first unpinned fan-out query happens to run one — until then the cross-schema UNION skips it and index queries return 0 forever.Fix (additive, idempotent, reactive)
StaticRepoImporter.ImportAllnow takes the served catalog set and, after the import, runsAssertCatalogPartitionsIndexed: for each catalog partition actually backed by a registered source this run (candidate ∩ current sources — an in-memory-served or non-served catalog is left alone), it forces onesearchable_schemasrebuild (bypassing the query-hot-path throttle — this is a one-time boot action, not a per-query rebuild), reads the registry back, and surfaces a startup-failure notification (existingNotifyStartupFailurepath) for any served catalog still absent. So an already-broken instance self-repairs on restart and the failure is visible. No-op on in-memory backends (noICrossSchemaQueryProvider). PG round-trips run on theIIoPool; noObservable.FromAsync, noawait.ICrossSchemaQueryProvidergainsSyncSearchableSchemasAsync(bool force, ct); the PostgreSQL + Snowflake implementations bypass the TTL whenforce=true(single-flight mutual-exclusion is still honoured). Existing callers are unchanged (the old(ct)overload delegates withforce:false).AiContentSources.ContentPartitionsas the assertion set.Test
test/MeshWeaver.Hosting.PostgreSql.Test/CatalogPartitionIndexSelfHealTests(PG Testcontainers,pgvector/pgvector:pg17):ImportAll, the served catalog is registered insearchable_schemasand a cross-schema search finds its seededModelProvider/Skillnodes;searchable_schemasrow that later goes missing (the "materialized but unindexed" drift) is re-registered by the next boot's self-heal even inside the 30 s throttle window — theforce=truebypass is exactly what makes that deterministic (a plain throttled sync would skip and leave it gone).Ran locally via Testcontainers — 2/2 green. Release
-warnaserror -p:CIRun=truebuild of every touched project is clean (the only diagnostics are the pre-existing repo-wideNU1902AngleSharp advisory).🤖 Generated with Claude Code