Skip to content

test(#5494): fold fork-PR issue-harness negative into fork-label positive - #5501

Merged
ifireball merged 1 commit into
mainfrom
agent/5494-fold-fork-issue-negative
Jul 23, 2026
Merged

ifireball merged 1 commit into
mainfrom
agent/5494-fold-fork-issue-negative

Conversation

@fullsend-ai-coder

Copy link
Copy Markdown
Contributor

Summary

Fold the standalone "Fork PR does not trigger issue-only harness" negative scenario into the existing "Fork PR label dispatches harness" positive scenario, eliminating a ~90s settle wait while maintaining the same test coverage.

Related Issue

Part of #5490.

Changes

  • Register a second harness (fork-issue-ping) with an issue-only trigger (event.entity.kind == "work_item") in the fork-label positive scenario
  • Assert the issue-only harness did not run after the fork harness positive assertions complete
  • Remove the standalone fork↛issue negative scenario that independently opened a fork PR and waited 90s

Testing

  • make lint — pre-commit could not run in sandbox (network); post-script runs authoritative check
  • make go-test on ./pkg/behaviourtest/... — all tests pass
  • go vet ./pkg/behaviourtest/... — clean
  • Fork dispatch feature stays green (requires live e2e run)

Checklist

  • PR title follows Conventional Commits (correct type, ! for breaking changes)
  • Commits are signed off (DCO) — autonomous agent, exempt
  • I wrote this contribution myself and can explain all changes in it

Closes #5494

Post-script verification

  • Branch is not main/master (agent/5494-fold-fork-issue-negative)
  • Secret scan passed (gitleaks — c9c7540c2995d9a294ede00c2444d2eb600e4852..HEAD)
  • PR body secret scan passed (gitleaks — no-git)
  • Pre-commit hooks passed (authoritative run on runner)
  • Tests ran inside sandbox

…tive

Register an issue-only harness ("fork-issue-ping") in the fork-label
positive scenario and assert it did not run, replacing the standalone
"Fork PR does not trigger issue-only harness" scenario. This removes
one ~90s settle wait by piggybacking the negative assertion on the
existing fork PR + label dispatch round-trip.

Note: pre-commit could not run in sandbox (network access denied).
The post-script runs an authoritative pre-commit check.

Unrelated test failures in internal/binary, internal/cli, and
internal/fetch are pre-existing (sandbox network restrictions).
All behaviourtest package tests pass.

Closes #5494
@fullsend-ai-coder
fullsend-ai-coder Bot requested a review from a team July 23, 2026 04:57
@fullsend-ai-coder fullsend-ai-coder Bot added the ready-for-review Triggers review agent dispatch label Jul 23, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Jul 23, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 4:59 AM UTC · Completed 5:08 AM UTC
Commit: ce866d2 · View workflow run →

@github-actions

Copy link
Copy Markdown

Site preview

Preview: https://85f9a80b-site.fullsend-ai.workers.dev

Commit: ce866d23a14590731a023e3e7eca1600ec99fe79

@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@fullsend-ai-review

Copy link
Copy Markdown

Looks good to me


Labels: PR refactors e2e fork-dispatch behaviour tests (component/e2e + component/dispatch) as a maintenance chore (type/chore).

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge component/e2e End-to-end tests component/dispatch Workflow dispatch and triggers type/chore Maintenance and housekeeping tasks labels Jul 23, 2026
@ifireball ifireball added the ok-to-test Allow e2e CI to run after maintainer review (must be re-applied after each push) label Jul 23, 2026
@ifireball
ifireball added this pull request to the merge queue Jul 23, 2026
Merged via the queue into main with commit 14fdc08 Jul 23, 2026
45 of 49 checks passed
@ifireball
ifireball deleted the agent/5494-fold-fork-issue-negative branch July 23, 2026 06:35
@fullsend-ai-retro

fullsend-ai-retro Bot commented Jul 23, 2026 •

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 6:37 AM UTC · Completed 6:51 AM UTC
Commit: ce866d2 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #5501 — test(#5494): fold fork-PR issue-harness negative into fork-label positive

Workflow went well — no proposals filed.

This retro covers a clean, zero-rework agent workflow on PR #5501, a code-agent-authored e2e test consolidation chore (part of epic #5490).

Timeline

Time (UTC) Event
04:34 ifireball creates #5494, triggers /fs-triage
04:37–04:40 Triage agent completes (~5 min), labels ready-to-code + triaged
04:45 ifireball triggers /fs-code with explicit scoping instructions
04:45–04:57 Code agent run 29980341964 — 12 min total, ~9 min agent time. First-attempt success.
04:57 PR #5501 created (1 file, +13/−16 lines)
04:58–05:08 Review agent run 29980884048 — 10 min, $2.23. Dispatched 3/5 sub-agents (correctness, style, intent-coherence; skipped security + docs). Zero actionable findings. Approved, labeled ready-for-merge.
05:28 ifireball adds ok-to-test
06:22 ifireball approves (no comments)
06:35 PR merged via merge queue

Assessment

What went well:

  • Code agent succeeded first-try with a well-scoped, correct change. It even performed a proactive safety check (grepping for DispatchAgent usage to confirm multi-harness registration safety).
  • Review agent made smart triage decisions — skipping irrelevant sub-agents and correctly approving a clean change.
  • The correctness sub-agent identified that the refactored test is actually stronger than the original (verifies entity-kind filtering when the label matches, not just when a fork PR is opened with a different label).
  • Total agent pipeline time: ~28 minutes (triage + code + review). No rework cycles.

Why no proposals:

  • I considered filing an autonomy-readiness tracking issue (human review added zero findings, ~75 min latency). However, #5156 correctly identifies that zero-comment human approvals are not meaningful evidence of agent superiority — the absence of human comments cannot be used to infer depth of human review. This PR (trivially correct, single-file test refactoring) is exactly the pattern Autonomy-readiness methodology produces false evidence when human review has no comments #5156 flags as producing false autonomy evidence.
  • Existing issue #4532 already tracks a similar data point (test-only PR where agent review covered human review).
  • No review quality gaps, no rework, no token waste, and no pipeline inefficiencies were identified.

This branch was successfully deployed

1 active deployment
site-preview — ce866d23 Deployed Jul 23, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/dispatch Workflow dispatch and triggers component/e2e End-to-end tests ok-to-test Allow e2e CI to run after maintainer review (must be re-applied after each push) ready-for-merge All reviewers approved — ready to merge ready-for-review Triggers review agent dispatch type/chore Maintenance and housekeeping tasks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(behaviour): fold fork-PR↛issue-harness negative into fork-label positive

1 participant