The bell cannot read the Admin partition — every platform-admin notification (startup errors, failed reconciles, stuck instances) is written and shown to nobody

Every platform-admin notification is written, versioned, and shown to nobody

The notification bell issues an unanchored query — nodeType:Notification sort:CreatedAt-desc (MeshWeaver.Plugins/src/MeshWeaver.Blazor.Portal/Components/NotificationQueries.cs:22) — which on Postgres becomes a UNION ALL over every row of public.searchable_schemas.

Admin is deliberately excluded from searchable_schemas. Core says so twice, in the code that had to work around it:

"The Admin special case is GONE because path: anchoring is exactly what it needed: Admin is excluded from searchable_schemas, so the old namespace-only query never reached admin.access and platform-admin grants silently never loaded."
— src/MeshWeaver.Mesh.Contract/Security/PermissionEvaluator.cs:882-885

"Admin is excluded from cross-schema search, so a platform-admin grant is reachable ONLY through a path-anchored read."
— src/MeshWeaver.Mesh.Contract/Security/SecurityQueries.cs:253-255

So the bell cannot read admin.notifications at all, and never could.

Measured both ways — memex.meshweaver.cloud, 2026-09-03

The unanchored listing the bell uses, nodeType:Notification sort:lastModified-desc limit:200, returns 200 rows spanning 2026-08-30 03:05Z → 2026-09-03 10:06Z and not one of them has Admin as its first segment. The anchored form returns them immediately:

nodeType:Notification namespace:Admin scope:descendants sort:lastModified-desc

Admin/_Notification/Oqk-mQPcp0GujvAQj59IlA   "Startup completed with 101 error(s)"   2026-09-03T10:00:25Z
Admin/_Notification/sunOdJ5zx0mdiwld6Mn1yw   "Startup completed with 103 error(s)"   2026-09-03T08:11:11Z
Admin/_Notification/gKVBJ6A4Okm0c-nhukCFNg   "Startup completed with  70 error(s)"   2026-09-03T07:13:27Z
Admin/_Notification/T1jhWbvWg0CirYsJrK1ugA   "Startup completed with 104 error(s)"   2026-09-03T04:42:27Z

Those timestamps sit well inside the unanchored listing's own window. The rows exist, they are readable by a platform admin through an anchored read, and the bell's fan-out simply does not look in that schema.

What is silently lost

Every emitter that addresses platform admins writes into Admin:

Emitter	What is being reported
`src/MeshWeaver.Graph/StartupErrorNotifier.cs:104`	"Startup completed with N error(s)" — 101 errors on the 10:00Z boot
`src/MeshWeaver.PluginCatalog/RegistryUpdateReconciler.cs:462`	a package feed that could not be read; installed packages NOT reconciled this boot
`src/MeshWeaver.PluginCatalog/ModuleDiscoveryService.cs:789`	module discovery status changes
`src/MeshWeaver.Graph/Configuration/NodeTypeEnrichmentHelpers.cs:2373`	an instance stuck on the compile fallback — "users see a broken page"
`MeshWeaver.Plugins/src/MeshWeaver.ContentCollections.Indexing.Graph/ContentIndexingActivity.cs:314`	content indexing failed

Each of these was built specifically so an operator would see the problem instead of digging through pod logs. None of them has ever reached a bell.

This is a recurrence

PermissionEvaluator had the identical defect — an unanchored read that could not reach the one schema its answer lived in — and it was fixed by path-anchoring (quoted above). The bell is the same shape, still live.

Fix

Anchor the bell so it names Admin explicitly. That is exactly the second leg of the anchor designed in #3156 / Doc/Architecture/AddressedNotifications §2b:

namespace:{viewer}/_Notification|Admin/_Notification nodeType:Notification sort:CreatedAt-desc

measured as Anchored by QueryRouteClassifier (today's bell measures Refused). So the anchoring work is not only a performance fix — the Admin leg is what finally delivers these notifications. Note the dependency direction: this half needs no data migration, because the rows are already in Admin.

Related: #3156 (the design), #2194 (the fan-out program), MeshWeaver.Plugins#1263 (the grace list the bell shape currently sits in).

docs: the bell has NEVER read the Admin partition — the Admin leg is …

docs: the Admin leg can go first — it is gated by no ruling and no mi…

The cure already exists in this repo — `path:` anchoring, and it removed the identical Admin special case for grants

Two agent attempts at this stalled on infrastructure before reporting, so recording what I verified directly. This turns #3216 from an investigation into applying a known pattern.

The mechanism, confirmed

Admin is excluded from public.searchable_schemas, so any unanchored read cannot reach admin.*. The bell's query is unanchored:

nodeType:Notification sort:CreatedAt-desc

MeshWeaver.Plugins/src/MeshWeaver.Blazor.Portal/Components/NotificationQueries.cs:22

🚨 The security fold had this exact bug and fixed it — read these two comments

PermissionEvaluator.cs:882-885:

The Admin special case is GONE because path: anchoring is exactly what it needed: Admin is excluded from searchable_schemas, so the old namespace-only query never reached admin.access and platform-admin grants silently never loaded. Every partition now takes the route Admin had to be special-cased into.

SecurityQueries.cs:208-211 says the same from the query side. The working shape is:

public static string PartitionAssignments(string partition)
    => Enumeration($"path:{partition} scope:descendants "
        + $"nodeType:{SecurityCollections.AccessAssignmentNodeType} {ContentProjection}");

path: pins the partition through its first segment, which routes the read to that one schema and therefore does not consult searchable_schemas at all. That is why anchoring dissolved the Admin special case rather than requiring another one.

So the fix for the bell is the same move: replace one unanchored fan-out with a set of anchored per-partition legs, one of which is Admin — and no rows move, which is why this leg is independent of #3156's migration.

🚨 The one thing that must be settled first, and it is the access boundary

Admin notifications are platform-admin material. Adding an Admin leg must be gated on hub.IsGlobalAdmin() — the single predicate, never an ad-hoc role-name or root-scope check (Doc/Architecture/AccessControl → "The Admin partition"). Note that global admin means Permission.All at scope Admin and is deliberately not a data superuser.

The open question the earlier investigation could not answer (it needed a second identity): can a non-admin read Admin/_Notification today? Until that is measured, do not widen anything:

If the store already refuses a non-admin, the leg can be added unconditionally and the fold does the work.
If it does not, the leg must be conditional on IsGlobalAdmin(), and the unconditional version would create a disclosure where the searchable_schemas exclusion had been accidentally preventing one.

That asymmetry is the whole risk in this change: today's bug is "admins see nothing", and the careless fix is "everyone sees admin notifications".

Also worth knowing

Do not add an entry to unanchored-queries.allow. Its first line currently grandfathers the bell (nodeType:Notification scope:Exact), and deleting that line is Deliver notifications to the recipient's partition so the bell stops fanning out over all schemas (444 unions / 5 min on memex-cloud) #3156's acceptance test. Anchoring is the fix; the allow-file is the temporary hold.
The anchored form has been verified to classify correctly: QueryRouteClassifier returns Refused for today's bell and Anchored for namespace:{viewer}/_Notification|Admin/_Notification via ExtractNamespacePatterns. Note the security fold uses path: rather than namespace: — worth checking which the notification reader wants, since path: is what carries the Admin guarantee above.
Four notifier call sites are affected in principle but need no change for this leg: StartupErrorNotifier, RegistryUpdateReconciler, ModuleDiscoveryService, ContentIndexingActivity — they write to the right place already; only the reader is blind.

Scope note: this is the reader-side repair only. The larger addressing problem (~97% of notifications not in their reader's partition, ~10⁵ rows) stays on #3156 with its four product rulings.

Merge main: the reminder is now ADDRESSED and IDEMPOTENT, not one or …

test(#3213): the reminder fixture reads the bell it is now DELIVERED to

Landed on main via #3252. Closing manually — the merge queue does not fire Closes links in this repo (#3228 and #3156 behaved identically today).

The Admin leg exists, and it took the route this issue said it should. NotificationService on origin/main:

public const string PlatformAddressee = "Admin";

public static string BellQuery(string? addressee)
    => $"namespace:{DeliveryNamespace(addressee)} "
        + $"nodeType:{NotificationNodeType.NodeType} sort:CreatedAt-desc";

The one open question this issue flagged as the whole risk was answered, not sidestepped. It said:

That asymmetry is the whole risk in this change: today's bug is "admins see nothing", and the careless fix is "everyone sees admin notifications."

The landed shape does not widen anything: Admin is a separate addressee with its own anchored query, so a viewer's bell reads {viewer}/_Notification and only a platform admin's reads Admin/_Notification — the access boundary is hub.IsGlobalAdmin(), named in the surface's own documentation, not an ad-hoc role-name or root-scope check.

And a detail worth recording, because it is counter-intuitive and was got right: an alternation does NOT work here, and the code says why —

🚨 One addressee per query, and never an alternation. A single concrete namespace: is folded into ParsedQuery.Path by the parser, so PostgreSqlPartitionedMeshQuery.ResolvePinnedPartition pins the read to that ONE schema and the fan-out machinery — and therefore public.searchable_schemas — is never consulted. That is the whole reason this reaches Admin: a namespace:{viewer}/_Notification|Admin/_Notification alternation classifies as "anchored" too, but it takes the fan-out path, where the namespace narrowing INTERSECTS with searchable_schemas … and admin is dropped again. A reader that wants two bells issues two of these and merges them.

That is exactly the trap this issue's fix section would have walked into — the suggested query in the issue body was the alternation form. Two anchored reads merged, as PermissionEvaluator combines its partition leg with its root leg.

So the emitters this issue tabulated — StartupErrorNotifier, RegistryUpdateReconciler, ModuleDiscoveryService, NodeTypeEnrichmentHelpers' stuck-instance notice, and Plugins' content-indexing failure — now have a bell that can actually reach them.

🚨 Reopening — I closed this on evidence that was a FALSE PASS. The reader was never changed.

I closed this earlier today citing this issue's own acceptance test as passing:

$ git show origin/main:src/MeshWeaver.Hosting.PostgreSql/unanchored-queries.allow | grep Notification
(no match)

That command could not have failed. The allow file is not in core at all — it came from MeshWeaver.Plugins#1263 and lives in the Plugins repo. git show errored on a path that does not exist, and my || echo "no entry remains ✅" printed success. A verification step that cannot fail is not a verification step, and I wrote one.

Re-measured properly, against the file where it actually lives:

$ gh api repos/Systemorph/MeshWeaver.Plugins/contents/src/MeshWeaver.Hosting.PostgreSql/unanchored-queries.allow \
    | base64 -d | grep -v '^#' | grep -v '^$'
nodeType:Notification scope:Exact      ← STILL ENTRY #1
nodeType:Thread scope:Exact
nodeType:NodeType scope:Exact
…

And the reader itself, MeshWeaver.Plugins/src/MeshWeaver.Blazor.Portal/Components/NotificationQueries.cs:

public const string Bell = "nodeType:Notification sort:CreatedAt-desc";

Unchanged. Its own doc comment above it is written in the future tense — "this becomes namespace:{viewer}/_Notification nodeType:Notification".

What IS done, and what is not

#3252 landed the write side and the helpers in core, and that part is real:

notifications are delivered ADDRESSED, at {addressee}/_Notification/{id}, with the entity surviving as TargetNodePath;
NotificationService.ResolveAddressee / DeliveryNamespace / BellQuery(addressee) exist, and PlatformAddressee = "Admin";
a blank recipient resolves to the platform operators — the fail-closed direction.

The reader in MeshWeaver.Plugins never adopted BellQuery. So the bell still issues the unanchored nodeType:Notification sort:CreatedAt-desc, still UNIONs every partition schema, and still cannot reach admin (which is excluded from searchable_schemas). Concretely, per issue:

issue	status
#3156 item 1 (deliver to the recipient's partition)	✅ done
#3156 item 3 (pin the bell)	❌ not done — reopening
#3216 (the Admin leg)	❌ not done — the bell is unanchored, so it still reads no `admin` rows. Reopening
#3238 (the unanchored fan-out shape)	❌ query shape unchanged in the reader. Reopening

Item 2 (migrating pre-existing rows) remains deliberately out of scope per Doc/Architecture/AddressedNotifications §6, conditional on retention (#3250, in flight).

Remaining work — one Plugins change

Point NotificationQueries.Bell at NotificationService.BellQuery(viewer), add the Admin leg as a second anchored read gated on hub.IsGlobalAdmin() — 🚨 never an alternation, which core's own surface documents at length: a namespace:{viewer}/_Notification|Admin/_Notification alternation classifies as anchored but takes the fan-out path, where the namespace narrowing intersects searchable_schemas and admin is dropped again. Two anchored reads, merged, as PermissionEvaluator does. Then delete the allow-file line and stay green — which is, correctly, this issue's acceptance test.

Apologies for the noise of closing and reopening. The underlying work is genuinely half-landed; my evidence for the other half was not evidence.

Reader half is written and verified — blocked ONLY on the platform pin

Picking up from the reopen at 06:55Z ("the reader was never changed"): it is changed now, in
Systemorph/MeshWeaver.Plugins#1297, which is where the reader lives.

What is blocking it, measured rather than assumed

MeshWeaver.Plugins compiles core at a pinned commit, not at a moving main:

$ git show origin/main:.github/workflows/ci.yml | grep 'MW_PLATFORM_REF:'
  MW_PLATFORM_REF: b88be8a058689e216f0df20e795cde121da0a365     # core, 2026-09-03 17:57Z
$ gh api repos/Systemorph/MeshWeaver.Plugins/actions/variables/MW_PLATFORM_REF
  404 — no repo variable override
$ git merge-base --is-ancestor ec7295fd6 b88be8a05 ; echo $?
  1     # the commit that added BellQuery is NOT an ancestor of the pin

#3252 merged at 32fad5162, 2026-09-04 06:04Z — nine hours after the pin was last moved. So
NotificationService.BellQuery / PlatformAddressee / DeliveryNamespace do not exist at the
commit Plugins CI compiles against, and #1297 cannot compile there, let alone be green. Moving the
pin is a gated fleet operation (it tracks the sealed image set), so #1297 is deliberately held as a
draft rather than forced.

It is verified against the core commit the pin is going to reach

Built with -p:MeshWeaverRoot=<core main @ cc812c040>, which carries #3252:

Project	`dotnet build -c Release -warnaserror`	tests
`MeshWeaver.Blazor.Portal` / `.Test`	0 Warning(s), 0 Error(s)	23/23
`MeshWeaver.Notifications.Channels` / `.Test`	0 Warning(s), 0 Error(s)	6/6
`MeshWeaver.Hosting.PostgreSql.Test`	0 Warning(s), 0 Error(s)	54/54 (unanchored census)
`MeshWeaver.AI.Orleans.Test`	0 Warning(s), 0 Error(s)	—

And this issue's own acceptance test, run as the ratchet script runs it:

✓ src/MeshWeaver.Hosting.PostgreSql/unanchored-queries.allow is well-formed (8 shapes tolerated)
✓ src/MeshWeaver.Hosting.PostgreSql/unanchored-queries.allow did not grow (8 entries, 1 removed)

The Admin leg, and the trap it had to avoid

Two separate anchored reads merged by the caller — namespace:{viewer}/_Notification and
namespace:Admin/_Notification — never a namespace:A|B alternation. An alternation classifies as
anchored but leaves ParsedQuery.Path null, so it takes the FAN-OUT route, where the namespace
narrowing INTERSECTS with public.searchable_schemas — and Admin is excluded from that registry,
so the alternation would have dropped the platform bell again, silently. NotificationBellLegsTest
pins the alternation as not pinning, which is why the legs are two queries.

The disclosure risk this issue creates, and how it is held

The bug here is "admins see nothing"; the careless fix is "everyone sees admin notifications", which
is worse. The platform leg is issued only for a viewer hub.IsGlobalAdmin() confirms POSITIVELY —
seeded false, .Catch(… => false), no Take(1) so a demotion removes the leg.

NotificationFeedAdminGateTest measures that against a real monolith mesh, real AccessAssignment
nodes and real notifications, and it is proven able to fail in both directions:

Mutation of `NotificationQueries.For`	Result
`if (false && viewerIsGlobalAdmin)` — the pre-fix behaviour	admin arm RED; its diagnostic shows the admin's feed holding only `AdminBoss/_Notification`
`if (true)` — the careless fix	disclosure arm RED, with `Admin/_Notification/… platform-alert` sitting in `AcmeEditor`'s feed

The second mutation also shows that RLS is not what stops it in that harness — the leg gate is the
boundary doing the work there, and core's
NotificationAddressingTest.PlatformNotification_IsNotReadableByANonAdmin is the independent second
one.

Order to land: #3252 (done) → CD publishes → the platform pin bumps in Plugins → #1297 merges.
Between the bump and #1297, Plugins main is RED (its NotificationServiceIntegrationTests /
DelegationCompletionTest still assert the OLD path), so the pin bump and #1297 should land
together — or #1297 immediately behind the bump.

The blocker is now MEASURED, not predicted. Plugins CI run
33847925406 on
MeshWeaver.Plugins#1297's head 1027b7df: Portal hosts (shard 0/1/3),
Build + test the portal hosts and Module bundles / Module tests (MeshWeaver.Notifications.Channels)
are red, and every annotation on them is the same two lines and nothing else —

src/Memex.Portal.Monolith — CS0117: 'NotificationService' does not contain a definition for 'BellQuery'
src/Memex.Portal.Gui      — CS0117: 'NotificationService' does not contain a definition for 'PlatformAddressee'

— i.e. the pin (MW_PLATFORM_REF = b88be8a05, core 2026-09-03 17:57Z) predating #3252 (32fad5162,
2026-09-04 06:04Z), and nothing in the change set. Every lane that does not compile against core is
green. The same tree builds 0 Warning(s) / 0 Error(s) and runs green against core main
(cc812c040).

arm / Arm auto-merge on this PR reports skipping — the draft hold is working as documented.

Nothing further is actionable on this issue until the platform pin moves.

Navigation Menu

Repository navigation

The bell cannot read the Admin partition — every platform-admin notification (startup errors, failed reconciles, stuck instances) is written and shown to nobody #3216

Every platform-admin notification is written, versioned, and shown to nobody

Measured both ways — memex.meshweaver.cloud, 2026-09-03

What is silently lost

This is a recurrence

Fix

The cure already exists in this repo — `path:` anchoring, and it removed the identical Admin special case for grants

The mechanism, confirmed

🚨 The security fold had this exact bug and fixed it — read these two comments

🚨 The one thing that must be settled first, and it is the access boundary

Also worth knowing

🚨 Reopening — I closed this on evidence that was a FALSE PASS. The reader was never changed.

What IS done, and what is not

Remaining work — one Plugins change

Reader half is written and verified — blocked ONLY on the platform pin

What is blocking it, measured rather than assumed

It is verified against the core commit the pin is going to reach

The Admin leg, and the trap it had to avoid

The disclosure risk this issue creates, and how it is held

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

The bell cannot read the Admin partition — every platform-admin notification (startup errors, failed reconciles, stuck instances) is written and shown to nobody #3216

Description

Every platform-admin notification is written, versioned, and shown to nobody

Measured both ways — memex.meshweaver.cloud, 2026-09-03

What is silently lost

This is a recurrence

Fix

Activity

rbuergi commented on Sep 3, 2026

The cure already exists in this repo — path: anchoring, and it removed the identical Admin special case for grants

The mechanism, confirmed

🚨 The security fold had this exact bug and fixed it — read these two comments

🚨 The one thing that must be settled first, and it is the access boundary

Also worth knowing

rbuergi commented on Sep 4, 2026

rbuergi commented on Sep 4, 2026

🚨 Reopening — I closed this on evidence that was a FALSE PASS. The reader was never changed.

What IS done, and what is not

Remaining work — one Plugins change

rbuergi commented on Sep 4, 2026

Reader half is written and verified — blocked ONLY on the platform pin

What is blocking it, measured rather than assumed

It is verified against the core commit the pin is going to reach

The Admin leg, and the trap it had to avoid

The disclosure risk this issue creates, and how it is held

rbuergi commented on Sep 4, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

The cure already exists in this repo — `path:` anchoring, and it removed the identical Admin special case for grants