Repository navigation
refactor(#5438)!: per-scenario World in context + in-process repo lease pool - #5444
Conversation
|
🤖 Finished Review · ✅ Success · Started 7:48 AM UTC · Completed 8:07 AM UTC |
Site previewPreview: https://c408bc38-site.fullsend-ai.workers.dev Commit: |
ReviewFindingsMedium
Low
Previous runReviewFindingsMedium
Previous run (2)ReviewFindingsMedium
Low
Previous run (3)ReviewFindingsMedium
Low
Previous run (4)ReviewFindingsMedium
Low
Previous run (5)Looks good to me Previous run (6)ReviewFindingsMedium
Low
Previous run (7)ReviewFindingsHigh
Low
Previous runReviewFindingsMedium
Low
Labels: All changes are in e2e/behaviour/ and pkg/behaviourtest/ — e2e testing infrastructure Previous run (8)ReviewFindingsHigh
Low
Previous run (9)ReviewFindingsMedium
Low
Labels: All changes are in e2e/behaviour/ and pkg/behaviourtest/ — e2e testing infrastructure |
|
/fs-fix All CI checks (unit, e2e, behaviour) are green. Please address the open review findings:
Optional (low priority, fix if straightforward):
|
|
🤖 Finished Fix · ✅ Success · Started 10:27 AM UTC · Completed 10:33 AM UTC |
🔧 Fix agent — iteration 1 (human-triggered)Addressed 3 of 4 review findings. (1) Hardened Release() with select/default panic on double-release. (2) Documented InitScenario signature change in behaviour-testing.md. (3) Surfaced Acquire errors in concurrent test. Deferred concept documentation per human instruction. Fixed (3):
Disagreed (1):
Tests: passed Updated by fullsend fix agent |
…InitScenario change - Replace unconditional channel send in Release() with select/default that panics on double-release (buffer full), preventing silent deadlock. - Add TestRepoPool_DoubleReleasePanics to cover the new panic behavior. - Surface Acquire errors in TestRepoPool_ConcurrentAcquireRelease via t.Errorf instead of silently returning. - Document the InitScenario(sc, template, pool) signature change and per-scenario World-via-context pattern in behaviour-testing.md for external consumers pinning to this module. Addresses review feedback on #5444
|
/fs-review Re-review after fix commit
Please confirm medium findings are resolved; remaining deferred item is low-priority concept docs for #5441. |
|
🤖 Finished Review · ✅ Success · Started 10:40 AM UTC · Completed 10:55 AM UTC |
|
/fs-fix Re-review on
Low finding about documenting |
|
🤖 Finished Fix · ✅ Success · Started 10:57 AM UTC · Completed 11:03 AM UTC |
🔧 Fix agent — iteration 2 (human-triggered)Addressed all review findings. Created commit with BREAKING CHANGE: trailer documenting suite.InitScenario and steps.Register API changes. Added steps.Register documentation to behaviour-testing.md API changes section. PR title update to add ! suffix was blocked by sandbox policy — post-script should set title to: refactor(#5438)!: per-scenario World in context + in-process repo lease pool Fixed (3):
Tests: passed Updated by fullsend fix agent |
|
/fs-review Re-review after fix commit
Please confirm the high finding is resolved. Only low/info findings should remain if any. |
|
🤖 Finished Review · ✅ Success · Started 11:09 AM UTC · Completed 11:29 AM UTC |
Superseded by updated review
🔧 Fix agent — iteration 8 (human-triggered)Fixed the potential deadlock in RepoPool.Release by unlocking the mutex before the channel send, exactly matching the pattern requested. Added RepoPool mention to behaviour-drivers.md. Left the optional nil-check suggestion as-is since beforeScenario guarantees a non-nil World in context. Fixed (2):
Disagreed (1):
Tests: passed Updated by fullsend fix agent |
|
/fs-review Head
Do not re-bump timeouts. Behaviour may sit near the 30m job limit on |
|
🤖 Review · ❌ Terminated · Started 8:47 PM UTC · Ended 9:01 PM UTC |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
🤖 Finished Review · ✅ Success · Started 8:47 PM UTC · Completed 9:01 PM UTC |
|
/fs-fix Codecov patch is failing (46.98% vs 80% target): 44 missing lines, all in
# Integration-only; covered by behaviour/e2e CI jobs (not unit coverage upload).
- "pkg/behaviourtest/steps/dispatch.go"
- "pkg/behaviourtest/suite/init.go"Extend that ignore list to the other steps files whose only patch changes are the FromContext migration (same integration-only class):
Prefer a single glob if clean, e.g. Do not bump |
|
🤖 Finished Fix · ✅ Success · Started 10:12 PM UTC · Completed 10:18 PM UTC |
Add five more pkg/behaviourtest/steps/ files to the .codecov.yml ignore list. These files are exercised by the behaviour suite (not the unit-coverage upload) and were causing a Codecov patch failure. A glob was considered but rejected because cleanup.go in the same directory has meaningful unit tests that should remain visible. Addresses review feedback on #5444
🔧 Fix agent — iteration 9 (human-triggered)Extended .codecov.yml ignore list with 5 integration-only step files to fix Codecov patch coverage failure. Listed files explicitly (not a glob) because cleanup.go in the same directory has meaningful unit tests. Fixed (1):
Tests: passed Updated by fullsend fix agent |
|
/fs-review Head
Timeout stays at 30. Expect only low/info / intentional breaking-api notes. |
|
🤖 Finished Review · ✅ Success · Started 10:27 PM UTC · Completed 10:40 PM UTC |
|
🤖 Finished Retro · ✅ Success · Started 4:00 AM UTC · Completed 4:22 AM UTC |
Retro: PR #5444 — per-scenario World in context + in-process repo lease poolTimeline
Key metrics
Review quality gapThe review agent approved at 13:17 UTC while human reviewer @waynesun09 found 2 HIGH + 5 MEDIUM issues on the same commit. The bot missed:
The human's multi-model squad (Claude, Grok, Gemini, Codex) provided diversity of perspective that the bot's single-vendor sub-agents did not. Evidence for existing issues
What went well
Proposals filedSee below for 3 improvement proposals targeting Proposals filed
|
Merge origin/main to pick up BT runtime-shortening changes (#5444, #5491, #5492, #5494, #5495). The merge completed without conflicts. Fix TestMintAgentToken_RepoResolutionError and TestResolveMintRepos_NeitherSet to explicitly clear REPO_FULL_NAME and MINT_REPOS via t.Setenv — these tests assumed the env vars were unset but failed when REPO_FULL_NAME was set in the test environment. Addresses review feedback on #5428
Summary
Per-scenario World via
context.Context: Each godog scenario now gets its ownWorldinstance, cloned from a suite-level template in theBeforehook and stored in the godog context. Step definitions extract their World viaworld.FromContext(ctx)instead of closing over a shared pointer. This prevents concurrent scenario hooks from clobbering each other's fields.In-process repo lease pool: Added
RepoPool— a buffered-channel pool pre-filled with 12 logical names (test-repo-01...test-repo-12). Scenarios acquire a name inBeforeand release it inAfter. AtConcurrency: 1the leased names are plumbing only; the functional test repo remainstest-repo.No behavior change:
GODOG_CONCURRENCYstays at 1, admin e2e still uses singulartest-repo, and all existing unit tests pass. This is infrastructure plumbing for the parallel execution work in ci(e2e): enable GODOG_CONCURRENCY for behaviour suite (default 12) #5441.Related Issue
Part of #3454. Prerequisite for #5439, #5440, #5441.
Changes
New files
pkg/behaviourtest/world/context.go—WithWorld/FromContexthelpers for storing World in contextpkg/behaviourtest/world/pool.go—RepoPooltype withAcquire/Release/Sizepkg/behaviourtest/world/pool_test.go— unit tests for pool (distinct names, blocking, release, concurrency)pkg/behaviourtest/world/context_test.go— unit tests for context helpers (round-trip, independence)Modified files
pkg/behaviourtest/world/world.go— addedLeasedRepoNamefield andClone()methodpkg/behaviourtest/world/world_test.go— added Clone tests (driver copy, scenario zeroing, independent mutation)pkg/behaviourtest/suite/init.go—InitScenarionow takestemplate+pool, clones World per-scenario, acquires/releases pool namespkg/behaviourtest/steps/registry.go—Registerno longer takes*world.Worldpkg/behaviourtest/steps/{triage,dispatch,dispatch_count,fork,dummy_agent}.go— step closures now acceptcontext.Contextand extract World viaFromContexte2e/behaviour/suite_test.go— createsRepoPool(12), passes template + pool toInitScenarioTesting
go test ./pkg/behaviourtest/...(world, suite, steps packages)go build ./...go vetcleanChecklist
Concurrency: 1preservedtest-repoconstant unchangedCloses #5438
Post-script verification
agent/5438-world-per-scenario-lease-pool)675086c5e606fe1ea9c09c22e1aa617aa17afeae..HEAD)