Repository navigation
Description
What is failing
A GetDataResponse posted by the content-collections config handler was delivered to the root mesh hub (mesh/EfNT…) instead of the requesting actor. The root hub is the mesh's router and must not be an end of a work delivery; work landing on its action block competes with routing and can starve real SubscribeRequest traffic (the 2026-06-11 prod wedge).
Probable cause — medium-high confidence.
The report is a target-role ORIGIN line at ContentCollectionsExtensions.HandleCollectionConfigRequest (ContentCollectionsExtensions.cs:344): the handler is the answering half — it posts its response back to wherever the request came from (ResponseFor(delivery) / the request's sender). So the offending side is the requester that issued the collection-config/GetData request with the router (mesh/{id}) as sender — almost certainly a mesh-singleton holding the DI-injected IMessageHub, which in the root container is the router. This matches the known residue documented in Router Traffic Detection: "three content-collection reads" kept posting as mesh/{id} off a proven-router hub field after #4477 hopped only the lifecycle verb.
The fix is therefore not at the named call site (nothing to hop on a reply). It is to hop the requesting call site onto hub.ReadIssuingHub() (portal/reads-{meshId}) — the correct seam for a bounded one-shot read. Do not use StreamSubscribingHub() here, and never hop a subscription onto ReadIssuingHub() (it registers no handlers by design).
Impact
One occurrence on one pod, deduped per (role, type) per hub, so this is the first sighting of this shape — not an outage. But the class it belongs to is the known precursor to a portal-wide wedge: router-block work starving routing. Worth closing while it is one line rather than a burst.
Where to look
- The reply side (named by the report, innocent):
MeshWeaver.ContentCollections.ContentCollectionsExtensions.HandleCollectionConfigRequest(ContentCollectionsExtensions.cs:344). - The request side (the actual offender, to be identified): find the caller issuing
CollectionConfigRequest/GetDatafor collections off the DI-injectedIMessageHub— checkMeshOperationsand the content-collection readers noted in Router Traffic Detection as unhopped sibling exchanges. - Reference shape for the fix:
MeshServicehops bothIssuingHub+NodeOperationTarget.
Related prior art in the same family: #4463, #4477, #4614/#4615/#4617, #4697. Not a duplicate — this is a new (role, call-site) pair, and unlike #4697 the requester-side offender is not yet identified.
Evidence
| Fingerprint | 8cdc3487631aca71 |
| Category | MeshWeaver.Messaging.MessageHub |
| Severity | Error |
| Top frame | MeshWeaver.ContentCollections.ContentCollectionsExtensions.HandleCollectionConfigRequest (ContentCollectionsExtensions.cs:344) |
| Namespace | memex-cloud |
| Pods | memex-portal-deployment-76cf776847-9p7z4 |
| Occurrences | 1 |
| First seen | 2026-09-23 20:56:19Z |
| Last seen | 2026-09-23 20:56:19Z |
| Routing | not determined — no configured route matches the category MeshWeaver.Messaging.MessageHub. This repository is the configured fallback, not a finding about who owns the fault; the category names the LOGGER, which may not be the subject. |
Recent log lines
2026-09-23 20:56:19Z memex-portal-deployment-76cf776847-9p7z4 fail: MeshWeaver.Messaging.MessageHub[0]
ROUTER_TRAFFIC ORIGIN: GetDataResponse was POSTED with the mesh hub as target (sender: Posts, target: mesh/EfNTPfFIy0WLbHUEEmk3jg). The mesh hub is the ROUTER and must not be an end of a work delivery. If the role above includes 'sender', this post LEFT the router: hop it onto the seam that matches what this delivery IS — MeshExtensions.NodeOperationIssuingHub() for a node LIFECYCLE write, MeshExtensions.ReadIssuingHub() for a bounded one-shot READ, MeshExtensions.StreamSubscribingHub() for a remote stream SUBSCRIPTION. The three are NOT interchangeable and the wrong one fails SILENTLY: ReadIssuingHub() registers no handlers by design, so a subscription hopped onto it stops reporting AND stops receiving data (#4614). If the role above includes 'target', ask where that target CAME FROM, because the two cases have opposite fixes: if this call site CHOSE the router as the destination, the call site is the offender — address the owning node instead (MeshExtensions.NodeOperationTarget()); if the target was read off an incoming request or subscription (request.Subscriber, ResponseFor(delivery) — SubscribeAck, DataChangedEvent, StreamErrorEvent and StreamEndedEvent all are), then this call site is the innocent answering half and the hub to move is the one that SUBSCRIBED or REQUESTED (#4697). 'sender AND target' means both apply. Reported once per role+type for this hub. Call site:
at MeshWeaver.ContentCollections.ContentCollectionsExtensions.HandleCollectionConfigRequest (ContentCollectionsExtensions.cs:344)
at MeshWeaver.Messaging.MessageHubConfiguration+<>c__DisplayClass65_0`1.<WithHandler>b__0 (MessageHubConfiguration.cs:312)
at System.Reactive.Linq.ObservableImpl.SelectMany`2+ObservableSelector+_.OnNext
at System.Reactive.Linq.ObservableImpl.SelectMany`2+ObservableSelector+_+InnerObserver.OnNext
at System.Reactive.BasicProducer`1.SubscribeRaw
at System.Reactive.Linq.Ob…[truncated]
Opened automatically from Admin/_LogIncident/8cdc3487631aca71. Recurrences are folded into this issue rather than opening new ones.
Activity
rbuergi commentedon Sep 24, 2026
Duplicate of #5620 — same log site
MeshWeaver.Messaging.MessageHub: ROUTER_TRAFFIC[ ORIGIN]: {type} … with the mesh hub as sender/target; consolidated.Read before closing: The ORIGIN half of the SAME exchange #5620 reports: 2026-09-23 20:56:19Z, pod 76cf776847-9p7z4, router mesh/EfNTPfFIy0WLbHUEEmk3jg ↔ Posts, one millisecond apart.
The fix, verification and any recurrence are tracked on #5620.