fix: a teardown report and a stalled delete name what they tell you to find (#3712, #1198)

src/MeshWeaver.Documentation/Data/Architecture.md

@@ -357,6 +357,7 @@ Each theme starts with its introductory page, followed by related architecture t
 - [Departed Platform Assemblies](DepartedPlatformAssemblies) — an assembly that leaves the platform for a module breaks every OTHER module that binds it, at LOAD time and invisibly to every compile gate; why "those are the platform" is one answer per host
 - [Detached Response Continuations](DetachedResponseContinuations) — why a `hub.Observe(...)` continuation runs on the RESPONDING hub's action block, what that cost on the mesh's one node-CRUD hub, and the six invariants that make the hop an opt-in rather than the default
 - [Reading a Disposal Stall Verdict](DisposalStallVerdicts) — what each field of the disposal snapshot actually measures, the three that were read as evidence while measuring nothing, and the verdict hole that sent 47 reports to children that were not the problem
+- [A Failure Report Answers Its Own Instruction](AFailureReportAnswersItsOwnInstruction) — a report that says "find why this hub disposed" while holding the answer, and an outstanding-work field that rendered "not measured" identically to "none"
 - [Ambient Test-Host Hangs](AmbientTestHostHangs) — what decides whether a killed test host can be diagnosed at all, and the readings of it already falsified
 - [In-Mesh Tests and the Seal](InMeshTestsAndTheSeal) — a Tests area no required context executes is a latent trunk red the seal detonates fleet-wide; how to measure a gate before requiring it
 - [Cancel and Join Are Two Questions](CancelAndJoinSequencing) — a deadline that asks work to stop and a deadline that waits for it to have stopped must not share one clock

src/MeshWeaver.Documentation/Data/Architecture/AFailureReportAnswersItsOwnInstruction.md

@@ -0,0 +1,200 @@
+---
+NodeType: Markdown
+Name: "A Failure Report Answers Its Own Instruction"
+Abstract: "Two live incidents in which a failure report told the reader to go and find a fact the reporting code already held: the deferred-delivery discard that said 'find why this hub disposed' without naming the teardown, and the commit-stage delete timeout that printed an outstanding-work field as '-' while the drain was stuck on the rest of the subtree. The rule, the two shapes it covers, and why a field that renders 'not measured' identically to 'none' is worse than no field."
+Icon: "<svg viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg'><rect width='24' height='24' rx='4' fill='#b23c17'/><path d='M12 7v6' fill='none' stroke='white' stroke-width='2' stroke-linecap='round'/><circle cx='12' cy='17' r='1.3' fill='white'/></svg>"
+Thumbnail: "images/DataMesh.svg"
+Authors:
+  - "Roland Buergi"
+Tags:
+  - "Architecture"
+  - "Disposal"
+  - "Diagnostics"
+  - "Operations"
+---
+
+# A Failure Report Answers Its Own Instruction
+
+> **The rule: if a report ends by telling the reader to go and find something, the reporting code
+> must first check whether it is already holding it — and if a report carries a field for
+> outstanding work, every producer of that report fills it, because an unfilled one does not
+> abstain, it asserts the opposite.**
+
+Both halves were measured on live portals in September 2026, one day apart, in two subsystems that
+share no code. They are the same defect.
+
+Related: [Reading a Disposal Stall Verdict](../DisposalStallVerdicts) — what a snapshot field actually
+measures; [Bounds Must Be Ordered](../BoundsMustBeOrdered) — why the innermost, most specific answer
+has to win the race; [Controls That Cannot Fail](../ControlsThatCannotFail) — one word covering several
+states.
+
+---
+
+## Shape 1 — the report tells you to find what it already knows
+
+`MessageService.Dispose()` discards deliveries a hub accepted and parked behind its initialization
+gates, and files each one at Error (event **7301**). The line ended:
+
+```text
+[DISPOSE-DISCARD] Hub RiskTransfer is disposing with SubscribeRequest (id=…, from cache/…)
+still deferred; initialization gates closed at deferral: [DataContextInit,MeshNodeInit] — the
+message is NOT processed; the sender is answered ShuttingDown. RunLevel=ShutDown. Accepted work
+must be drained before a hub goes down; find why this hub disposed before its deferred work
+could run.
+```
+
+*Find why this hub disposed.* The hub knows. `MessageHub` records `disposeRequestedBy`,
+`disposeReason` and `cascadeOwner` one frame earlier, in `HandleDispose` — added precisely so
+[#3510](https://github.com/Systemorph/MeshWeaver/issues/3510)'s
+*"`[QUIESCE-START]` on a root should name who asked"* could be satisfied. But `[QUIESCE-START]` is
+`Information`, and the red-log pipeline files `Error`s. **So the one line that becomes an issue was
+the one line without the attribution.**
+
+Measured on `Admin/_LogIncident/d2249f800ffc2577`: **364 occurrences, 2026-09-08 → 2026-09-14,
+13 pods**. Every captured sample names the message, its sender and the gates it sat behind. Not one
+says which teardown threw it away — so the reader cannot tell an operator recycle from a NodeType
+rebind from an owner's cascade, which are three different investigations.
+
+The fix is not a new measurement. It is passing a value that already exists across ten lines of the
+same method:
+
+```text
+… RunLevel=ShutDown. This teardown was requested by a cascade from its owner mesh; why: the
+owner's own teardown — … . Accepted work must be drained before a hub goes down — that
+attribution is who to ask why this hub went down with work still parked behind its gates.
+```
+
+### The second reader is the one who cannot see the log at all
+
+The stranded sender is in another process as often as not. It gets a NACK, and that NACK said only
+that the hub had gone away. The same clause therefore goes into the NACK text — the same reasoning
+that removed the generic disposal sentence from the sibling path in
+[Retiring an Activation](../RetiringAnActivation).
+
+### Attribution must VARY, or it is decoration
+
+`DisposalAttribution` renders four distinguishable answers, and the absence of a routed request is
+itself one of them:
+
+| what happened | what the report says |
+|---|---|
+| a routed `DisposeRequest` that stated a reason | `requested by <sender>; why: <the reason>` |
+| a routed `DisposeRequest` posted by the hub to ITSELF (a rebind, a self-heal) | `requested by itself — a self-posted DisposeRequest (<address>), i.e. a rebind or self-heal recycle; why: …` |
+| a routed `DisposeRequest` with no reason, or a blank one | `requested by <sender>; why: reason not stated by the caller` |
+| **an owner's cascade** — this hub is going down because its owner is | `requested by a cascade from its owner <owner>; why: the owner's own teardown — <the ORIGINATING cause, propagated unchanged down the chain>` |
+| no routed request and no cascade (host teardown, a `using`) | `requested by a direct Dispose() (no routed DisposeRequest)` |
+
+Three of those distinctions are load-bearing and none of them can be inferred from the others:
+
+- **The cascade form is not the direct-dispose form.** A cascaded child IS disposed by a direct
+  `Dispose()` call from `HostedHubsCollection`, so before `cascadeOwner` existed every child in a
+  wave reported `a direct Dispose()` — indistinguishable from host teardown, and reading the child
+  told you nothing about the root. That was #3510's wedge one level down. The cascade carries the
+  **originating** cause, not merely the immediate parent, so a leaf names the event that started it
+  however deep the tree is.
+- **The last row rules the message path out**, which is exactly what the production incident needed
+  and could not get.
+- **A blank reason is an unstated one.** `Reason` is free text from the poster; `null` was once the
+  only value treated as "not stated", so an empty string rendered a literal `why: ` with nothing
+  after it — the defect again in a new costume. It is normalised at the single capture point, so
+  every reader inherits it rather than each having to remember.
+
+### One cause, claimed once
+
+Two writers can record the cause — the `DisposeRequest` handler on the action block, and the owning
+collection's cascade note on whatever thread is disposing the owner — so "first cause wins" has to
+be a **claim**, not a pair of reads. It is one `Interlocked.CompareExchange`, and the handler
+additionally declines to claim at all when the teardown has already begun: `Dispose()` sets its
+shutting-down flag and only then posts its first `ShutdownRequest`, so a routed request arriving in
+that window still sees `RunLevel=Started`, is admitted, and its turn runs *after* the teardown
+started. Without the check it would overwrite a true `a direct Dispose()` reading with its own.
+
+**A report that names the WRONG cause is worse than one that names none** — it is this same defect,
+pointing somewhere else.
+
+**Corollary, and it is the reason this section names a poster:** the rendering is only as good as
+what posters supply. `MeshOperations.Recycle` — the operations / MCP `recycle` verb, the one
+teardown that comes from OUTSIDE the framework's own lifecycle and therefore the one whose cause a
+log cannot reconstruct — was the single production poster of a `DisposeRequest` with no `Reason`. It
+now states one.
+
+---
+
+## Shape 2 — a field that renders "not measured" the same as "none"
+
+The recursive delete reports every stage failure through one line:
+
+```text
+[DeleteNode] timeout path=… stage=<stage> partial-deleted=<n> unanswered=<set>
+```
+
+`unanswered=` exists so an operator learns WHICH node stopped the operation. The pre-flight stage
+fills it, and its occurrences read like this:
+
+```text
+stage=pre-validate-descendants partial-deleted=0
+unanswered=sglauser/AgenticBusiness/01-MeetYourCoworker/AskAdvisor, …
+```
+
+The commit stage never set it. So a commit-stage timeout rendered:
+
+```text
+[DeleteNode] timeout path=Hosting/TriageStatus stage=commit partial-deleted=3 unanswered=-
+System.TimeoutException: [DeleteNode:commit] the bottom-up delete of 'Hosting/TriageStatus' made
+no progress for 30s — 3 path(s) removed from storage so far
+```
+
+(memex-cloud, 2026-09-14T08:45:38Z.)
+
+`-` is the same rendering the line uses for *"there is nothing outstanding"*. The field did not
+abstain — **it asserted the opposite of the truth**: three paths were removed, the drain was stuck
+on the rest of the subtree, and the report said it owed nothing. And the sentence beside it offers
+only a count, which identifies what SUCCEEDED and therefore identifies nothing about the failure.
+
+Both halves of the answer are live in that closure: the plan (`collected.ToDelete`) and the progress
+(`SnapshotProgress()`). The difference is the answer, and it costs one set operation on a path that
+has already failed:
+
+```text
+[DeleteNode:commit] the bottom-up delete of 'Hosting/TriageStatus' made no progress for 30s —
+3 of 9 planned path(s) removed from storage so far; still owed by the plan:
+Hosting/TriageStatus, Hosting/TriageStatus/…
+```
+
+Two details that are not incidental:
+
+- **"still owed by the PLAN", not "remaining".** A drain can remove MORE paths than it planned — a
+  child created between the plan snapshot and the drain — which is the whole subject of
+  [#3392](https://github.com/Systemorph/MeshWeaver/issues/3392). The set is what the plan still
+  owes; it is never a claim about what else storage may hold.
+- **The count stays.** Reporting real progress rather than `0` is the half of
+  [#1198](https://github.com/Systemorph/MeshWeaver/issues/1198) that landed earlier. Names are
+  ADDED; a rewrite that swapped one fact for another would read as a fix and be a regression.
+
+---
+
+## What this rule is not
+
+It is **not** a licence to downgrade or silence a report. Both changes here leave every level,
+every event id and every existing fact exactly where they were; they add the fact the reader was
+told to go and find. The two nearby changes that DID reclassify a line
+([#4178](https://github.com/Systemorph/MeshWeaver/issues/4178), a hub discarding its OWN deferred
+delivery;
+[#3647](https://github.com/Systemorph/MeshWeaver/issues/3647), a queued turn the pump is about to
+take) each moved a level only after establishing, by measurement, that the line asserted stranded
+work with no waiter — and each kept the line, at Debug, naming the same facts. A classification that
+stops reporting is a silenced fault, not a classification.
+
+---
+
+## Checklist when you write or review a failure report
+
+1. Does it end with an instruction (*find why…*, *check which…*)? If so, is the answer reachable
+   from where the line is written? If it is, put it in the line.
+2. Does it carry a field for outstanding / unanswered / pending work? Then **every** producer of
+   that line fills it, and "unset" must not render as "empty".
+3. Does the report identify what FAILED, or only count what SUCCEEDED?
+4. Does the value VARY with the condition? Write the negative case down — the absence of a cause is
+   itself an answer and has to be spelled as one.
+5. Does the party who cannot read this log — a caller in another process — get the same fact in its
+   NACK or response?

src/MeshWeaver.Documentation/Data/WhatsNew/2026-09-14-a-teardown-and-a-stalled-delete-now-name-what-they-tell-you-to-find.md

@@ -0,0 +1,57 @@
+---
+Name: A teardown report and a stalled delete now name what they tell you to look for
+Category: Fix
+Description: Two failure reports ended by sending the reader off to find a fact the reporting code was already holding — a hub discarding accepted work would not say which teardown discarded it, and a delete that stalled halfway through a subtree printed the field meant for outstanding work as "none". Both now name it, and so does the answer sent to the caller waiting on the other side.
+Icon: Search
+Order: -20260914
+---
+
+# A teardown report and a stalled delete now name what they tell you to look for
+
+Two unrelated parts of the platform had the same habit: a failure report that ends with an
+instruction — *find why this happened* — while the code writing it already knows the answer.
+
+## "Find why this hub disposed"
+
+When a hub is torn down while it is still starting up, any request it had accepted and parked is
+thrown away, and the sender is told to ask again. That is by design. It is also reported, because
+work the platform accepted and did not finish is worth a look.
+
+The report named the message, the sender, and the gate the message was waiting behind. Then it ended:
+*"find why this hub disposed before its deferred work could run."*
+
+The hub knew. Every teardown already records who asked for it and why — a rebind, an operator
+recycle, an owner going down and taking its children with it — and it was already printing that on
+a *different* line, at a level the automatic issue-filing never reads. So the one report that
+becomes a ticket was the one report without the answer. Over six days it was filed **364 times
+across thirteen servers**, and not one copy said which of those three very different situations it
+was.
+
+It now says. So does the reply sent to whoever was waiting — often a different process entirely,
+which can see none of this server's log at all. And when there was no request to attribute, it says
+*that*, because "nobody asked over the wire; something disposed it directly" rules out a whole class
+of cause and is an answer in its own right.
+
+One related gap closed with it: the operator **Recycle** action was the single place in the platform
+that tore a hub down without stating a reason, so everything it recycled reported "reason not stated
+by the caller". It now says what it is and who asked.
+
+## "3 paths removed" — and nothing about the rest
+
+Deleting a folder removes its contents from the bottom up. If the store stops answering partway
+through, the operation gives up rather than hang, and reports what happened.
+
+That report carries a field for *what is still outstanding* — the earlier stage of the same
+operation fills it in, listing exactly which items never answered. The final stage never did. So the
+field printed `-`, which is also how the report renders **"nothing is outstanding"**. A delete that
+was stuck on most of a subtree was reporting that it owed nothing, beside a count of the three items
+it had already managed. A count of what succeeded identifies nothing about what failed.
+
+It now lists the paths the delete still owes — the ones an operator actually has to go and look at —
+and keeps the progress count beside them.
+
+---
+
+Neither change quiets anything. Every report is still filed, at the same level, with everything it
+said before; they simply stop sending the reader to look for something that was in the room the
+whole time.