Re-print errored assemblies in dotnet test end-of-run recap#9545
Merged
Conversation
dotnet/sdk#51952: an assembly that handshakes but then exits non-zero without any failed test (crash, Environment.FailFast, a hang-dump kill, an option rejected after the handshake, ...) currently only prints its process output inline when it completes. In a large multi-assembly 'dotnet test' run that output is scrolled far above the summary and is very hard to find; only the one-line per-assembly result survives in the summary. Mirror the existing handshake-failure recap: record each errored assembly (assembly + tfm + arch + exit code + stdout + stderr) and re-print it in an 'Errored assemblies:' section after the summary. Assemblies that exit non-zero because a test failed are excluded (already reported per-test); the recorded set mirrors the summary's 'error: N' count. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an end-of-run “Errored assemblies:” recap to the Terminal test reporter so that process-level failures (non-zero exit without failed tests) are discoverable in dotnet test multi-assembly runs, similar to the existing handshake-failure recap.
Changes:
- Record assemblies that exit non-zero with
FailedTests == 0(including captured stdout/stderr) duringAssemblyRunCompleted. - Append a new end-of-run recap section (“Errored assemblies:”) after the summary (and after the handshake-failure recap), and reset this state at
TestExecutionCompletedfor Hot Reload scenarios. - Add unit tests covering both inclusion (errored w/ no failed tests) and exclusion (non-zero exit due to failed tests), plus localized resource updates for the new header.
Show a summary per file
| File | Description |
|---|---|
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/OutputDevice/Terminal/TerminalTestReporterTests.cs | Adds unit tests validating the new errored-assembly recap behavior and its intended scope. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalTestReporter.Summary.cs | Appends the new errored-assembly recap after the existing handshake-failure recap. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalTestReporter.Lifecycle.cs | Records eligible errored assemblies (non-zero exit, no failed tests) and clears recap state at end of execution. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalTestReporter.ErroredAssemblies.cs | New implementation for tracking and rendering the “Errored assemblies:” recap. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalResources.resx | Adds the new ErroredAssembliesHeader resource string. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/TerminalResources.cs | Exposes TerminalResources.ErroredAssembliesHeader accessor. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.cs.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.de.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.es.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.fr.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.it.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.ja.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.ko.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.pl.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.pt-BR.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.ru.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.tr.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.zh-Hans.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/Terminal/xlf/TerminalResources.zh-Hant.xlf | Adds localized XLF entry for ErroredAssembliesHeader. |
Review details
- Files reviewed: 19/19 changed files
- Comments generated: 0
- Review effort level: Low
…test Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
0101
approved these changes
Jul 2, 2026
github-actions Bot
added a commit
that referenced
this pull request
Jul 4, 2026
- Add --report-azdo-groups/--report-azdo-annotations toggles (#9542) to Platform changelog - Add re-print errored assemblies in end-of-run recap (#9545) to Platform changelog - Add server-initiated session cancellation (#9549) to Platform changelog - Add fix for CloneWithUpdatedSource mutating this (#9581) to MSTest changelog Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Evangelink
added a commit
that referenced
this pull request
Jul 5, 2026
This was referenced Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to the handshake-failure visibility work for
dotnet test(dotnet/sdk#51608), addressing the residual gap reported in dotnet/sdk#51952.An assembly that does handshake but then exits with a non-zero code without any failed test — a crash,
Environment.FailFast, a hang-dump kill, or an option rejected after the handshake (the original report wasUnknown option '--hangdump') — currently only has its process output (exit code + stdout/stderr) printed inline byAssemblyRunCompleted. In a large multi-assemblydotnet testrun that inline output is scrolled far above the summary and is extremely hard to find; only the one-line per-assembly result survives in the end-of-run summary, so you know which assembly errored but not why.Handshake failures already get an end-of-run recap (
Handshake failures:viaAppendHandshakeFailureRecap); this PR gives errored-but-handshaked assemblies the same treatment.Change
assembly + tfm + arch + exit code + stdout + stderr) inTerminalTestReporter.ErroredAssemblies.cs.Errored assemblies:section after the summary (AppendErroredAssemblyRecap), right after the handshake-failure recap.error: Ncount (failedAssembliesWithoutFailedTests).TestExecutionCompletedalongside_handshakeFailures(HotReload correctness).ErroredAssembliesHeaderresource string (+ xlf viaUpdateXlf).Orchestrator-only: the in-process host reaches neither the exit-code overload of
AssemblyRunCompletednor the recap, so its output is unchanged.Tests
Added to
TerminalTestReporterTests:TestExecutionCompleted_WhenAssemblyErroredWithoutFailedTests_ReprintsErroredAssemblyRecap— asserts the recap header, the assembly identity, exit code, and captured stdout/stderr are surfaced at the end of the run.TestExecutionCompleted_WhenAssemblyExitedNonZeroWithFailedTests_DoesNotReprintErroredAssemblyRecap— guards the scope (normal test failures are not recapped).All 115
TerminalTestReporterTestspass locally (net9.0).Fixes dotnet/sdk#51952