Problem
A batch that failed between becoming dependency-eligible (created) and being announced to the speculate topic was orphaned in created forever. Because created is dependency-eligible, every later batch took the orphan as a dependency. The queue wedged completely, while speculation kept re-planning and rebuilding.
Observed on a 100-PR demo run: 65 PRs landed, then 50+ minutes passed with no lands.
Cause
The batch stage's message names a request, so a retry could never find the batch a previous attempt had created; it minted a new one and abandoned the old one.
Fixed by #590, which splits dependency analysis into its own stage keyed by batch ID. That fix rests on three constraints: the dependency-analysis topic must partition by queue, the duplicate-batch guard must stay in place, and the index writes must be individually idempotent.
Problem
A batch that failed between becoming dependency-eligible (
created) and being announced to the speculate topic was orphaned increatedforever. Becausecreatedis dependency-eligible, every later batch took the orphan as a dependency. The queue wedged completely, while speculation kept re-planning and rebuilding.Observed on a 100-PR demo run: 65 PRs landed, then 50+ minutes passed with no lands.
Cause
The batch stage's message names a request, so a retry could never find the batch a previous attempt had created; it minted a new one and abandoned the old one.
Fixed by #590, which splits dependency analysis into its own stage keyed by batch ID. That fix rests on three constraints: the dependency-analysis topic must partition by queue, the duplicate-batch guard must stay in place, and the index writes must be individually idempotent.