Skip to content

chore(e2e): run e2e suites when reusable-prioritize.yml changes - #9201

Merged
rh-hemartin merged 2 commits into
fullsend-ai:mainfrom
tpouyer:issue-AISDLC-659
Nov 9, 2026
Merged

rh-hemartin merged 2 commits into
fullsend-ai:mainfrom
tpouyer:issue-AISDLC-659

Conversation

@tpouyer

@tpouyer tpouyer commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Run the behaviour and playback e2e suites when .github/workflows/reusable-prioritize.yml changes, as they already do for reusable-dispatch.yml. The e2e workflow's relevance filters decide when those suites run; they listed reusable-dispatch.yml but not reusable-prioritize.yml, so a change to the prioritize reusable workflow — which the behaviour scenarios exercise via prioritize.yml — skipped both suites.

Related Issue

Resolves AISDLC-659

Changes

  • .github/workflows/e2e.yml — adds .github/workflows/reusable-prioritize.yml to the on.push.paths trigger and to the behaviour and playback jobs' file-relevance grep filters, beside reusable-dispatch.yml.
  • scripts/check-live-test-relevance-test.sh — adds the same path to the script's positive list, so the script now checks that this path stays relevant under both filters; the two grep filters in e2e.yml must stay identical, as the SYNC-WITH comments in both files require.
  • internal/scaffold/e2e_relevance_test.go (new) — three tests: both relevance filters in e2e.yml match reusable-prioritize.yml, on.push.paths lists it, and the regression script's positive list includes it.

The first commit adds the tests; the code commit makes the change (2 files, +4/−2). No doc updates are owed (no document enumerates the filter list); reusable-poll-jira.yml stays out deliberately — no behaviour feature exercises it.

Protected paths

  • .github/workflows/e2e.yml — protected CI workflow; its relevance filters are exactly where the missing suite trigger lives (on.push.paths plus both job greps).
  • scripts/check-live-test-relevance-test.sh — protected regression script; its positive path list gains reusable-prioritize.yml, so the script now checks that this path stays relevant under both filters.

Testing

The three new tests fail on the base branch — its e2e.yml and script list contain no reusable-prioritize entry — and pass with this change:

  • TestE2ERelevanceFiltersRunSuitesForReusablePrioritize
  • TestE2EPushPathsIncludeReusablePrioritize
  • TestLiveTestRelevanceScriptListsReusablePrioritize

Run: go test -race ./internal/scaffold/ -run 'TestE2E|TestLiveTestRelevance' and bash scripts/check-live-test-relevance-test.sh; pre-commit hooks and go vet pass.

  • make lint passes (stage changes first, then run)
  • Tests added/updated for new or modified logic

Checklist

  • PR title follows Conventional Commits (correct type, ! for breaking changes)
  • Commits are signed off (DCO)
  • I wrote this contribution myself and can explain all changes in it

Receipt

  • Maintainer review: change correct and minimal; verdict comment because two protected paths need human sign-off.

Acceptance criteria

# criterion tests status
1 Given a PR that changes only .github/workflows/reusable-prioritize.yml, When the e2e workflow's relevance step runs, Then the behaviour and playback suites run. TestE2ERelevanceFiltersRunSuitesForReusablePrioritize, TestE2EPushPathsIncludeReusablePrioritize satisfied
2 Given the change, When scripts/check-live-test-relevance-test.sh runs, Then it passes with reusable-prioritize.yml in its list. TestLiveTestRelevanceScriptListsReusablePrioritize satisfied
3 Given the change, When it is opened, Then it touches only .github/workflows/e2e.yml and scripts/check-live-test-relevance-test.sh. — checked on this PR: the code commit touches only those two files; the new test file is in its own commit

Acceptance tests for https://redhat.atlassian.net/browse/AISDLC-659 (2 of 3
criteria; the rest declared for a person).
ac_1: TestE2ERelevanceFiltersRunSuitesForReusablePrioritize,
TestE2EPushPathsIncludeReusablePrioritize
ac_2: TestLiveTestRelevanceScriptListsReusablePrioritize
ac_3: not testable — No test can decide the file list of a PR before it
exists; a person checks it on the PR: the implementation commit must touch
only .github/workflows/e2e.yml and scripts/check-live-test-relevance-test.sh
(this story's frozen acceptance tests add
internal/scaffold/e2e_relevance_test.go beside them).
Red run on the base: 3 of these fail
(TestE2ERelevanceFiltersRunSuitesForReusablePrioritize,
TestE2EPushPathsIncludeReusablePrioritize,
TestLiveTestRelevanceScriptListsReusablePrioritize); the log is kept in the
run's evidence.

Signed-off-by: Tim Pouyer <tim.pouyer@gmail.com>
Fixes https://redhat.atlassian.net/browse/AISDLC-659

Acceptance tests: see the test(e2e) commit in this branch. Test rounds used:
1/100.

PR review: round 1, verdict approve (3 nitpick(s), 5 advisory(ies), 0
unresolved blocking).

Signed-off-by: Tim Pouyer <tim.pouyer@gmail.com>
@tpouyer
tpouyer requested a review from a team as a code owner November 9, 2026 00:16
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

E2E tests did not run

E2E tests run automatically for pull requests authored by users with write access or higher.

For other contributors, someone with write access must add the ok-to-test label after the latest push.

Reason: unauthorized

See E2E testing guide for details.

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Run e2e suites when reusable-prioritize.yml changes

🐞 Bug fix 🧪 Tests ⚙️ Configuration changes 🕐 10-20 Minutes

Grey Divider

AI Description

• Include reusable-prioritize.yml in the e2e push trigger and both suite relevance filters.
• Extend regression coverage so future filter changes cannot silently skip behaviour or playback.
Diagram

graph TD
  A["Prioritize change"] --> B["Push path trigger"] --> C["E2E workflow"] --> D{"Relevance filters"} --> E["Behaviour suite"]
  A --> F["PR or merge event"] --> C
  D --> G["Playback suite"]
Loading
High-Level Assessment

The targeted path additions fit the existing workflow and regression-script design. Centralizing the duplicated filters would be a broader refactor with little benefit for this fix.

Files changed (3) +116 / -2

Bug fix (1) +3 / -2
e2e.ymlRecognize reusable-prioritize changes across e2e triggers +3/-2

Recognize reusable-prioritize changes across e2e triggers

• Adds reusable-prioritize.yml to the main-branch push paths and to both the behaviour and playback changed-file filters. This prevents changes to that workflow from skipping the suites.

.github/workflows/e2e.yml

Tests (2) +113 / -0
e2e_relevance_test.goTest prioritize workflow trigger and filter coverage +112/-0

Test prioritize workflow trigger and filter coverage

• Adds tests that require the path in both identical relevance filters, the push path list, and the regression script’s positive list.

internal/scaffold/e2e_relevance_test.go

check-live-test-relevance-test.shGuard prioritize workflow relevance in script checks +1/-0

Guard prioritize workflow relevance in script checks

• Adds reusable-prioritize.yml to the positive single-file change cases checked against both suite filters.

scripts/check-live-test-relevance-test.sh

@qodo-code-review

qodo-code-review Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Prioritize workflow changes go untested ✗ Dismissed
Description
The new relevance entries select the behaviour and playback suites for reusable-prioritize.yml,
but neither suite invokes that reusable workflow. The behaviour shim calls reusable-dispatch.yml,
which implements prioritize inline, while the separate prioritize.yml caller is manual and
references @main, so a PR changing only the standalone workflow can receive green suite results
without exercising its change.

Grey Divider

Context sources
❌ Tickets: failed to load — Ticket collection could not establish complete evidence
✅ Compliance rules (platform): 96 rules
✅ Skills: 5 invoked
  writing-user-docs
  writing-adrs
  check-patch-coverage
  cutting-releases
  adr-corner
✅ Cross-repo context — repo relationships
✅ REVIEW.md
Review mode: Auto: ⚖️ Balanced: Touches protected CI workflow and regression script; warrants standard.

Grey Divider

Tip of the day
💡 Did you know, you can see how risky a PR is at a glance with blast radius labels

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread .github/workflows/e2e.yml Resolved
@fullsend-ai-review

fullsend-ai-review Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 12:18 AM UTC · Completed 12:25 AM UTC

Commit: d9cfb4c · View workflow run →

Runtime: pi · Model: sonnet → claude-sonnet-5-5 · Effort: high · Cost: $1.80

@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Nov 9, 2026
@fullsend-ai-review

Copy link
Copy Markdown

Risk Assessment: moderate (2/5)

Details

Touches CI workflow files and protected paths with moderate author contention and coupling history, but diff size is small, test ratio is high (67%), and recent churn and reverts are low.

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Medium

  • [protected-path] .github/workflows/e2e.yml — The PR modifies protected paths .github/workflows/e2e.yml and scripts/check-live-test-relevance-test.sh. The description links the tracking ticket (AISDLC-659) and explains the rationale, but human approval is always required for protected-path changes regardless of context.

Low

  • [over-engineering] internal/scaffold/e2e_relevance_test.go:91 — TestLiveTestRelevanceScriptListsReusablePrioritize locates the shell script's positive path list by string-matching for path in \\ ... ; do, which breaks if the script is reformatted, and its second loop repeats the filter assertions already made by TestE2ERelevanceFiltersRunSuitesForReusablePrioritize. Optional: simplify or drop it.

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.

Comment thread internal/scaffold/e2e_relevance_test.go Resolved
@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Nov 9, 2026
@codecov

codecov Bot commented Nov 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@tpouyer

tpouyer commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Intentional: Same finding as the thread on .github/workflows/e2e.yml:164 (qodo finding 0d7b85fc): the standalone reusable-prioritize.yml is suite-relevant by design — the thin per-repo prioritize.yml caller bypasses the dispatch shim and calls it directly (docs/contributing/workflow-contracts.md), and ADR 62 requires it stay in sync with the inlined prioritize job in reusable-dispatch.yml that the behaviour scenarios' /fs-prioritize routing exercises. Validating the standalone workflow at PR revision would contradict ADR 62's @main caller design and is outside AISDLC-659's scope.

@tpouyer

tpouyer commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Per decisions.jsonl, the person's final-review approval (question:final-review:854f653e7c): I approve the modification of the two protected paths

@rh-hemartin
rh-hemartin added this pull request to the merge queue Nov 9, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 9, 2026
@rh-hemartin
rh-hemartin added this pull request to the merge queue Nov 9, 2026
Merged via the queue into fullsend-ai:main with commit 8931df0 Nov 9, 2026
105 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-manual-review Review requires human judgment risk/moderate PR risk: moderate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants