Repository navigation
[ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7642
Description
Triggering signals
- (issues #7627, #7630, #7636, #7637, #7639, #7640, #7641 — rubric finding: seven prior feedback-run attempts to land this fix via
create_pull_requestfailed at git push; the same edit is being carried forward as a direct commit on this run) - Rubric finding: of 59 ci-scan runs since window-start (2026-06-08T13:53:53Z), the recurring skip reason is
stale build window (>14d)/no follow-up build yet, defer to next run/no failed build in 7d. Runs hitting these conditions have consumed 2.2–2.4M+ effective tokens before concluding with noop — approximately 10× the ~250K ET of a correctly-halting Step 1 exit. - Rubric finding (tally correctness): the
skipped-with-reasoncolumn reads0on skip-only runs (runs 58 and 59); Hard Rule 10 mandates the literal| 0 | 0 | 0 | 1 |row so the tally is honest.
Proposed edits
.github/workflows/ci-scan.agent.md(Hard Rules section, after rule 9): Add Hard Rule 10 that elevates the no-scannable-build exit to the same hard constraint level as the issue-cap rule — names exact forbidden operations (AzDO timeline fetch, log download, Helix query) and gives the literal tally row| 0 | 0 | 0 | 1 |..github/workflows/ci-scan.agent.md(Step 1 trailing sentence): Replace the inline restatement of the three skip reasons with a single reference to Hard Rule 10 so the constraint is stated once, authoritatively.
Expected behavior change
On any run where Step 1 yields a selection-time skip (no follow-up build yet, defer to next run, stale build window (>14d), or no failed build in 7d), the scanner will append the reason to the coverage file, print | 0 | 0 | 0 | 1 |, call noop, and stop immediately — without fetching any AzDO timeline, downloading any task log, or querying any Helix work item. This eliminates the observed ~10× token variance between correct low-ET runs (~250K ET) and high-ET runs (2.2–2.4M+ ET) on identical pipeline state, and ensures the tally row correctly shows skipped-with-reason=1 on skip-only runs.
Note
🔒 Integrity filter blocked 3 items
The following items were blocked because they don't meet the GitHub integrity level.
- AccessViolationException when running Microsoft.ML.Scenarios.ScenariosTests.TestRegressionScenario #6960
list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved". - [DatabaseLoader] Create higher level convenient methods for DatabaseLoader #4182
list_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved". - #7610
list_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | noneGenerated by CI Failure Scanner - Feedback (machinelearning) · ● 4.2M · ◷
Note
This was originally intended as a pull request, but the git push operation failed.
Workflow Run: View run details and download patch artifact
The patch file is available in the agent artifact in the workflow run linked above.
To create a pull request with the changes:
# Download the artifact from the workflow run
gh run download 28001364209 -n agent -D /tmp/agent-28001364209
# Create a new branch
git checkout -b ci-scan-feedback/hard-rule-10-early-exit-2026-06-23-95e476217153e017
# Apply the patch (--3way handles cross-repo patches where files may already exist)
git am --3way /tmp/agent-28001364209/aw-ci-scan-feedback-hard-rule-10-early-exit-2026-06-23.patch
# Push the branch to origin
git push origin ci-scan-feedback/hard-rule-10-early-exit-2026-06-23-95e476217153e017
# Create the pull request
gh pr create --title '[ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build' --base main --head ci-scan-feedback/hard-rule-10-early-exit-2026-06-23-95e476217153e017 --repo dotnet/machinelearningShow patch preview (56 of 56 lines)
From 163c6c39021ab08ace2f51a42bc0c29a2f54e1fa Mon Sep 17 00:00:00 2001
From: "github-actions[bot]" <github-actions[bot]@users.noreply.github.com>
Date: Tue, 23 Jun 2026 04:11:33 +0000
Subject: [PATCH] ci-scan: add Hard Rule 10 to force early exit on no scannable
build
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
6 of 59 ci-scan runs consumed 2.2M-2.4M+ effective tokens before
concluding with a selection-time skip reason that should have stopped
the run at Step 1 in ~250K ET. The existing 'and stop' sentence was
not preventing further fetching of timelines, logs, and Helix data.
Add Hard Rule 10 which elevates the no-scannable-build exit to the same
level as the issue-cap and label rules: append the skip reason, print
| 0 | 0 | 0 | 1 |, call noop, and stop — without fetching any AzDO
timeline, downloading any log, or querying any Helix work item.
Update Step 1's trailing sentence to reference Hard Rule 10 directly
instead of restating the skip-reason list inline.
Signal: issues #7627, #7630, #7636, #7637, #7639, #7640, #7641
(seven prior failed PR attempts); runs consuming 2.2M+ ET on a
skip-only path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
.github/workflows/ci-scan.agent.md | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/ci-scan.agent.md b/.github/workflows/ci-scan.agent.md
index 0937c5f..8bbfdb5 100644
--- a/.github/workflows/ci-scan.agent.md
+++ b/.github/workflows/ci-scan.agent.md
@@ -78,6 +78,7 @@ These invariants are not delegated to the shared file. Honor them even if a shar
7. **All state under `/tmp/gh-aw/agent/`;** each bash call is a fresh subshell.
8. **AzDO REST is anonymous;** stay on `https://dev.azure.com/dnceng-public/public/_apis/build/...`. Follow every rule in [Environment constraints](shared/ci-scan.instructions.md#environment-constraints) (pre-bind URLs, `%24top`, no redirection).
9. **Sanitize every embedded log excer
... (truncated)
Activity
[ci-scan] Add Hard Rule 10 for early exit on no scannable build (#7654)