You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ci: run one general macOS job per pull request push - #7578
macOS runners are the scarcest: with several open PRs, 41 macOS jobs were queued against about 5 running, and macOS jobs waited up to 78 minutes to start (median 9). Each PR push queued three general macOS jobs (the .NET pipeline, AOT (macos-latest) and Engine tests (macos-latest)) plus the Apple smoke test. Most of that work repeats what Ubuntu and Windows already run.
Change
New PlatformNeutral category. It tags the 19 suites whose results don't depend on the OS: analyzers, code fixers, source-generator and public-API snapshots, templates, and the assertion and mock libraries. On pull requests, the macOS .NET pipeline skips them and runs the Reflection-mode engine tests itself.
aot.yml drops macOS from pull requests. The aot and engine-tests matrices run on Ubuntu and Windows only. Pushes to main and manual runs keep macOS, so UploadToNuGetModule still waits for macOS AOT and engine tests before it publishes.
Branch protection no longer requires AOT (macos-latest) or Engine tests (macos-latest). That was already applied through the API, since this PR could not merge otherwise. modularpipeline (macos-latest) is still required. The issue-pr-loop reference is updated to match.
On macOS, a PR push now queues one general job plus the Apple smoke test, instead of three plus the Apple smoke test.
macOS keeps the OS-sensitive suites on PRs: unit tests, RPC, Playwright, ASP.NET Core, OpenTelemetry, and the Reflection-mode engine tests.
Validation
TUnit.Pipeline builds with 0 warnings.
actionlint 1.7.7 is clean on aot.yml and dotnet.yml.
This PR's own checks exercise the change: the macOS .NET job should list no PlatformNeutral modules and should include RunEngineTestsModule, and aot.yml should start no macOS jobs.
Summary by CodeRabbit
CI Improvements
Pull requests now run platform-neutral tests, Engine tests, and AOT checks on Ubuntu and Windows, while macOS runs Reflection-mode Engine tests.
macOS Engine tests and AOT checks continue to run for non-pull-request events.
macOS runners are the scarcest: with several open PRs, 41 macOS jobs queued against
about five running, and waits reached 78 minutes. Each PR push queued the dotnet.yml
macOS pipeline plus the macOS AOT and Engine tests jobs, and most of that work repeats
what Ubuntu and Windows already run.
- Tag the analyzer, code-fixer, source-generator, public-API, template, assertion and
mock suites PlatformNeutral. The macOS pipeline skips them on pull requests and runs
the Reflection-mode engine tests itself.
- aot.yml leaves macOS out of pull requests. Pushes to main and manual runs keep it, so
UploadToNuGetModule still waits for the macOS AOT and engine tests before publishing.
- Branch protection no longer requires AOT (macos-latest) or Engine tests (macos-latest);
the issue-pr-loop reference now says so.
Fixes#7577
Review was skipped as selected files did not have any reviewable changes.
Run configuration
Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: CHILL
Plan: Advanced
Run ID: befb33c8-36a0-415a-88ed-74038acf69fa
Commits
Reviewing files that changed from the base of the PR and between b7885c7 and f0b5330.
You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.
Use the checkbox below for a quick retry:
Trigger review
No actionable comments were generated in the recent review.
Walkthrough
The pipeline adds a PlatformNeutral category to test modules and updates CI job selection. Pull requests run AOT and engine-test jobs on Ubuntu and Windows. The macOS .NET pull-request job skips platform-neutral suites and runs Reflection-mode engine tests. Other events retain macOS AOT and engine-test jobs.
Changes: CI test selection
Layer / File(s)
Summary
Define and assign the test category tools/TUnit.Pipeline/PipelineCategories.cs, tools/TUnit.Pipeline/Modules/*
Adds the PlatformNeutral category and assigns it to analyzer, assertion, mock, source-generator, template, and public API test modules.
Select jobs by event and platform .github/workflows/aot.yml, .github/workflows/dotnet.yml, .agents/skills/issue-pr-loop/references/repository.md, .claude/docs/project-structure.md
Pull requests run AOT and engine-test jobs on Ubuntu and Windows. The macOS .NET pull-request job skips AOT, packaging, NuGet package, and PlatformNeutral suites, while retaining Reflection-mode engine tests. The documentation describes the category and CI checks.
thomhurst/TUnit#7444 — This change updates the merge-gate reference to describe the required Engine tests and AOT platforms.
thomhurst/TUnit#7575 — The affected analyzer test module is now skipped in macOS pull-request runs, but the reported timeout behavior is not changed.
thomhurst/TUnit#7574 — The affected analyzer test module is now skipped in macOS pull-request runs, but the reported stall and hang-dump behavior are not changed.
Possibly related PRs
thomhurst/TUnit#7364 — This change modifies the EngineTests job split and the corresponding .NET workflow exclusions.
thomhurst/TUnit#7357 — This change uses the existing packaging and NuGet package category exclusions in the macOS pull-request branch.
thomhurst/TUnit#7519 — This change updates the runner matrix in aot.yml while retaining macOS AOT coverage on main pushes.
Suggested reviewers:quattordici
Merge Risk: Minimal · previous review b7885
The PR’s intended CI reduction is implemented without an established current-head coverage or publishing failure.
Pre-merge checks: 5
Check name
Status
Explanation
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Title check
The title clearly and concisely describes the main change: reducing pull-request macOS CI to one general job per push.
Linked Issues check
Issue [#7577] requires fewer macOS jobs on pull requests. .github/workflows/aot.yml selects Ubuntu and Windows for both AOT and Engine tests on pull requests, and retains macOS for pushes and manual…
Out of Scope Changes check
The changed workflow conditions, category constant, module annotations, and contributor documentation directly implement issue [#7577]. The merge-gate documentation change supports the related check-n…
Docstring Coverage
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Full details: Linked Issues check
Explanation
Issue [#7577] requires fewer macOS jobs on pull requests. .github/workflows/aot.yml selects Ubuntu and Windows for both AOT and Engine tests on pull requests, and retains macOS for pushes and manual runs. .github/workflows/dotnet.yml keeps the macOS job, skips Aot, Packaging, NuGetPackageTests, and PlatformNeutral, and therefore runs Reflection-mode EngineTests. The new PlatformNeutral category and the module annotations cover the platform-neutral suites named by [#7577]. The Apple smoke workflow is not changed by this pull request. No additional coding requirement exists in the linked issue.
Full details: Out of Scope Changes check
Explanation
The changed workflow conditions, category constant, module annotations, and contributor documentation directly implement issue [#7577]. The merge-gate documentation change supports the related check-name and coverage changes. No unrelated code or workflow behavior is demonstrated in the reviewed changes.
Full details: Docstring Coverage
Explanation
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 20 files. (4 skipped: 4 unsupported.)
Finishing Touches
Generate docstrings
Commit to this branch
Create a new PR
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
A rabbit checks the workflow row, Ubuntu and Windows start to go. Mac runs the tests it still must run, While neutral suites are skipped there. The category marks each module neat, And carrots celebrate the feat.
Recent review info
Run configuration
Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: CHILL
Plan: Advanced
Run ID: d9535ada-438d-4d6f-bc2b-63f1e3cf50a7
Commits
Reviewing files that changed from the base of the PR and between 98b0697 and b7885c7.
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.
Looks good. The change does what the description says, and I found no blocking issues.
Checked
PlatformNeutral is defined in PipelineCategories.cs and applied consistently through [ModuleCategory], with the using ModularPipelines.Attributes added where it was missing.
In dotnet.yml, macOS on pull requests ignores Aot Packaging NuGetPackageTests PlatformNeutral. EngineTests is not ignored there, so RunEngineTestsModule runs on macOS. Pushes to main keep the old ignore list, and aot.yml covers macOS engine and AOT tests.
The fromJSON(... && '[...]' || '[...]') matrix expression in aot.yml is a valid GitHub Actions pattern. The array strings are non-empty and therefore truthy, so the &&/|| ternary works.
Suggestions (non-blocking)
Required checks. Skipped matrix legs never report, so the removal of AOT (macos-latest) and Engine tests (macos-latest) from branch protection is needed. You've already said it's done. Keep the repo.md note in sync if the required checks change again.
Guarding the category list. The list of PlatformNeutral modules is maintained by hand, and a new suite will silently run on all three operating systems until someone tags it. A small pipeline test or doc note saying "tag OS-independent suites with PlatformNeutral" would keep the savings from eroding.
Readability. The nested &&/|| expression in ignore-categories is hard to read. Consider computing it in a prior step, or in a matrix.include field, so the policy sits in one named place.
Net effect: macOS jobs per PR push go from 3 to 1, with the Apple smoke test on top. That is a sensible trade.
Required checks: done before this PR opened. Branch protection now requires modularpipeline (ubuntu-latest), modularpipeline (macos-latest), and AOT and Engine tests for Ubuntu and Windows. The issue-pr-loop reference already matches.
Guarding the category list: added in b7885c7. .claude/docs/project-structure.md now says to tag OS-independent pipeline test modules with [ModuleCategory(PipelineCategories.PlatformNeutral)], next to the existing pointer to tools/TUnit.Pipeline. If a module is left untagged, it only costs runner time, not coverage, so I've kept this to a documentation note rather than adding a test.
Readability: keeping the expression. The policy depends on both the OS and github.event_name, so moving it to matrix.include or an earlier step still needs the same condition, just further from the comment that explains it. As it stands, the three branches sit directly under that comment, one per line.
This is a sensible CI-cost reduction, and I found no blocking issues.
Checked
The PlatformNeutral category is applied consistently, and the ModularPipelines.Attributes usings are added where they were missing.
The ignore-categories expression parses as intended: Ubuntu keeps Aot EngineTests. macOS on a pull request ignores Aot Packaging NuGetPackageTests PlatformNeutral and so runs RunEngineTestsModule, which is tagged EngineTests. Everything else, including macOS on push, is unchanged.
The fromJSON matrix in aot.yml leaves main pushes and manual runs on all three OSes, so UploadToNuGetModule still waits for macOS.
Suggestions (non-blocking)
The ignore-categories ternary chain is hard to read, and it gets harder with every new platform rule. Consider a small matrix.include or a step-level env var (for example IGNORE_CATEGORIES) computed once. You could do the same for the platform list in aot.yml, where the identical fromJSON(...) string appears twice.
The category only helps if new modules are tagged, and nothing enforces that. A pipeline-level sanity test (or a note in the module template) would stop untagged neutral suites from drifting back onto macOS. The docs line in project-structure.md is a good start.
Branch protection was changed outside the repo. Record that change, for example in the issue, so the config can be recreated.
Overall this looks good to merge once the PR's own checks confirm macOS lists no PlatformNeutral modules and that aot.yml starts no macOS jobs.
Thanks. Suggestion 3 is done: the branch-protection change, with the exact before and after required checks, is recorded on #7577 (#7577). Suggestions 1 and 2 are covered in my previous reply: the expression stays, and the tagging note is in b7885c7. I'll confirm the macOS job contents once this PR's checks finish.
No blocking issues found. The change does what the description says.
Expression logic: the ignore-categories expression is correct. && binds tighter than ||, so the cases resolve as Ubuntu, then macOS on PRs, then everything else. On PRs macOS runs EngineTests and skips PlatformNeutral. Windows and non-PR macOS keep their existing behavior. The >- fold yields a single-line expression.
Matrix: the fromJSON conditional matrix is a standard pattern. Main pushes and manual runs keep macOS, so the publish gate is unchanged.
Tagging: the PlatformNeutral tags are applied consistently, and the doc note in project-structure.md tells people to tag new modules.
Suggestions, not blockers:
Category drift. Nothing stops a new OS-sensitive module from being tagged PlatformNeutral, or a neutral one from being forgotten. A small pipeline test that lists the tagged modules would catch accidental changes to the set.
Coverage trade-off. Dropping macOS AOT on PRs means a macOS-only AOT regression only shows up after merge to main. That is an acceptable cost for the runner savings, but a nightly or label-triggered macOS AOT run on PRs would shorten the feedback loop.
Duplicated list. The two inline platform JSON arrays in aot.yml could come from one shared env or a setup job, so they can't drift apart.
Overall the change is sound and reduces macOS queue pressure.
Category drift: not adding a test. A test listing the tagged modules would only repeat the attributes. It wouldn't catch a wrong tag, just ask the author to update the list as well. Whether a suite depends on the OS is a review judgment, and .claude/docs/project-structure.md now says when to tag one.
macOS AOT feedback: every push to main still runs aot.yml on macOS for that exact commit, and UploadToNuGetModule won't publish until it passes. So a macOS-only AOT regression shows up on the merge commit, not days later. For a risky change, aot.yml can still be started by hand (workflow_dispatch) on the PR branch to get macOS before merging. A nightly run would duplicate that coverage.
Duplicated platform list: keeping both lists in the workflow. strategy.matrix can't read the env context, so a shared value would need a setup job, which is another runner and another hop before every AOT job starts. That's a lot of cost to deduplicate a two-line literal that sits next to its explanation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GitHub Actions
Error
Fixes #7577.
Problem
macOS runners are the scarcest: with several open PRs, 41 macOS jobs were queued against about 5 running, and macOS jobs waited up to 78 minutes to start (median 9). Each PR push queued three general macOS jobs (the
.NETpipeline,AOT (macos-latest)andEngine tests (macos-latest)) plus the Apple smoke test. Most of that work repeats what Ubuntu and Windows already run.Change
PlatformNeutralcategory. It tags the 19 suites whose results don't depend on the OS: analyzers, code fixers, source-generator and public-API snapshots, templates, and the assertion and mock libraries. On pull requests, the macOS.NETpipeline skips them and runs the Reflection-mode engine tests itself.aot.ymldrops macOS from pull requests. Theaotandengine-testsmatrices run on Ubuntu and Windows only. Pushes tomainand manual runs keep macOS, soUploadToNuGetModulestill waits for macOS AOT and engine tests before it publishes.AOT (macos-latest)orEngine tests (macos-latest). That was already applied through the API, since this PR could not merge otherwise.modularpipeline (macos-latest)is still required. The issue-pr-loop reference is updated to match.On macOS, a PR push now queues one general job plus the Apple smoke test, instead of three plus the Apple smoke test.
macOS keeps the OS-sensitive suites on PRs: unit tests, RPC, Playwright, ASP.NET Core, OpenTelemetry, and the Reflection-mode engine tests.
Validation
TUnit.Pipelinebuilds with 0 warnings.actionlint1.7.7 is clean onaot.ymlanddotnet.yml..NETjob should list noPlatformNeutralmodules and should includeRunEngineTestsModule, andaot.ymlshould start no macOS jobs.Summary by CodeRabbit