Repository navigation
docs(ReadingCiSignals): classify a red shard by its exit marker — HOST_CRASHED in the log is script echo - #2739
Merged
Merged
Conversation
…— TESTFAIL vs MASKED — and `HOST_CRASHED` in the log is script echo Measured 2026-08-30 while triaging two main reds that were reported as "2x then 4x HOST_CRASHED". The markers said one TESTFAIL (a SilentReadNackTest flake; the host completed normally) and one genuine MASKED. The HOST_CRASHED hits were the summarize step echoing its own explanatory sentence, so a grep -c over the log counts echoes, not crashes. Adds the marker table (TESTFAIL / MASKED / TIMEOUT / SIGNAL), the echo trap, and the attribute-by-reachability rule that exonerated #2726 (Monolith.Test references no Orleans project) and #2721 in the same triage. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EPCWramjFYquX5SaMdeJKW
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the “Reading CI Signals” documentation to improve CI triage accuracy by teaching readers to classify red shards using the authoritative exit marker (rather than log text that may be misleading due to script echo).
Changes:
- Added a new section explaining how to interpret shard exit-marker classifications (
TESTFAIL,MASKED,TIMEOUT,SIGNAL) and how they should drive attribution. - Documented the
HOST_CRASHED“echo trap” where log hits can come from summarizer script output rather than real crashes. - Added guidance to attribute failures by dependency reachability (reference graph) before adjacency.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ary outranks the marker Correction from meshweaver-61, verified in run 33298742141 shard 1: the assembly printed `MeshWeaver.Hosting.Monolith.Test Total: 331, Errors: 0, Failed: 2, ... Time: 461.193s`, so the host COMPLETED — the MASKED classification was the harness inferring a crash from `rc != trxfails`, an assumption that xUnit v3 exits with the failure count (2 failures exited 1). Neither of the two reds was a crash. The section now says the marker is where to LOOK, names MASKED as derived, and makes the execution summary the decisive evidence; a real crash is the ABSENCE of that line. Classifier fix tracked in #2738. Also: xUnit capitalization in prose per WritingTests.md (Copilot). The marker quote keeps the harness's own lowercase spelling — it is a verbatim transcript. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EPCWramjFYquX5SaMdeJKW
Contributor
Test Results (shard 3)701 tests 509 ✅ 4m 38s ⏱️ Results for commit e0591b5. |
Contributor
Test Results (shard 5)1 533 tests 1 533 ✅ 5m 31s ⏱️ Results for commit e0591b5. |
Contributor
Contributor
Test Results (shard 4) 10 files 10 suites 4m 48s ⏱️ Results for commit e0591b5. |
Contributor
Contributor
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Conserving a triage finding from today, per the new AGENTS.md rule (#2732).
What was measured
Two main reds this morning were reported as
2x then 4x HOST_CRASHED— a trunk emergency that would have parked a release wave. The exit markers said otherwise:5e73bfcaa(#2721)exit=1 TESTFAIL (1 failing test(s) … the host completed normally)SilentReadNackTest, the #2673/#1599 family) — no crash81f8869d5(#2638)exit=1 MASKED (trx records 2 failing test(s), which does not explain exit=1 — host crashed after streaming results)UserHomeLegacyPathResolutionTest)The
HOST_CRASHEDhits came from the Summarize test failures step echoing its own explanatory sentence ("…is written INTO the trx as a<project>.HOST_CRASHEDfailure byrecord-host-crash.py…"), sogrep -c HOST_CRASHEDscales with the number of steps that echoed it, not with crashes.What the page gains
TESTFAILvsMASKEDvsTIMEOUT/SIGNAL, and why they attribute differently;MeshWeaver.Hosting.Monolith.Test.csprojreferences no Orleans project, which exonerated fix(#2638): hold the silo stop until accepted routing work has landed #2726 in one grep; ci(plugins): nothing a node repo builds goes to NuGet — retire the NuGet-floor pack lane and the tool's pack verb #2721's own issue body had already exonerated it.Doc-only. No What's New entry — agent/maintainer-facing CI guidance, not a user-visible change (deliberate skip).
🤖 Generated with Claude Code
https://claude.ai/code/session_01EPCWramjFYquX5SaMdeJKW