Add Microsoft.Testing.Extensions.GitHubActionsReport (groups, annotations, step summary, slow-test notices) by @azat-msft in #9541 (backport to rel/4.3)#9580
Closed
nohwnd-bot wants to merge 8 commits into
Conversation
…y log groups Implements the first requirement of microsoft#9142: a new GitHub Actions report extension that emits ::group:: / ::endgroup:: workflow commands per test assembly so the runner UI collapses each assembly's output. Auto-activates when GITHUB_ACTIONS=true; controllable via --report-gh-groups on/off and --report-gh. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Emit a ::error workflow command for each failing test via a new GitHubActionsAnnotationReporter (IDataConsumer), mirroring the AzureDevOpsReport structure. Resolves the source file/line from the exception stack trace relative to GITHUB_WORKSPACE (or the .git root), skipping MSTest assertion-implementation frames, and falls back to a title-only annotation when no location is available. Adds the --report-gh-annotations on|off knob (default on) and a shared GitHubActionsFeature activation helper. Includes unit tests for the annotation formatting/escaping and option validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add GitHubActionsSummaryReporter that collects per-test outcomes and, at session end, appends a markdown roll-up (totals table, failures list, slowest tests) to the file named by GITHUB_STEP_SUMMARY, mirroring the AzureDevOpsSummaryReporter structure. Adds the --report-gh-step-summary on|off knob (default on) plus unit tests for the markdown shape. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add GitHubActionsSlowTestReporter that tracks in-progress tests and, on a 1s scan loop with exponential backoff, emits a ::notice workflow command for any test still running past a threshold (default 60s), mirroring the AzureDevOpsSlowTestReporter. Adds the --report-gh-slow-test-notices on|off knob (default on) and a --report-gh-slow-test-threshold seconds option, plus deterministic ScanOnce-driven unit tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Update PACKAGE.md to describe failure annotations, the job summary, and slow-test notices alongside log groups, with a table of all --report-gh options. Minor test cleanup. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In CI the terminal output device uses SimpleAnsi mode and emits a color
reset ('ESC[m') without a trailing newline after the colored failed-test
block. The failure annotation, emitted immediately after via the same
output device, inherited that dangling reset as a line prefix ('ESC[m::error
...'). GitHub only recognizes a workflow command when the line begins with
'::', so the annotation was silently dropped (groups and notices are emitted
at clean line boundaries and were unaffected).
Emit the annotation on a fresh line so '::error' always starts at column 0.
Verified end-to-end via the gh-report-validation harness: the failure
annotation now surfaces in the GitHub Actions Annotations panel and on the
file's diff gutter.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #9541 to rel/4.3
/cc @Evangelink @azat-msft