Skip to content

spec(dag): parallel workspace writers with implementation evidence aggregator #293

Description

@LeXwDeX

Ticket 1/4 of the harness workstream. Primary source: docs/harness-review-2026-08-16.md (appendix A). Blocking edge: blocks the follow-up ticket in the config repo (opencode-dag-config: spec wording, plan disjoint-write-set contract, runtime-compat.json bump). No upstream blockers.

Problem Statement

The orchestration harness promises "trade concurrency for time", but every coding/prototype block is silently chained into one serial lane by the block compiler before the graph reaches the runtime. Saved references advertise "parallel slices" that never actually overlap. The only parallelism that survives compilation is read-only lanes (explore/review), which is where time pressure is lowest. Authors have no way to express genuinely parallel implementation work, and any route that tries to review two independent writers fails compilation entirely ("no canonical serialized implementation writer" — currently dead code, since serialization always forces a total order first).

Secondarily, the verify block contract promises "bind evidence to the supplied implementation fingerprint" but the compiler never supplies any fingerprint to verify nodes — the binding is an empty promise.

Solution

Make writer parallelism real at the single compilation boundary, with zero runtime changes:

  • Writers with no total order compile to truly parallel nodes (the runtime's existing max_concurrency semaphore then schedules them concurrently).
  • A compiled-in implementation aggregator node sits between parallel writers and the mandatory verify gate: it mechanically checks the writers' declared changed-file sets for overlap (failing loudly on intersection), computes the union, and produces the single implementation fingerprint at the convergence point. Review binds to the aggregator, preserving every existing review-lifecycle invariant.
  • Writer chains that already have a total order compile byte-identically to today.

User Stories

  1. As a workflow author, I want two coding blocks with disjoint responsibilities to run concurrently, so that wall-clock time scales with max_concurrency instead of hiding behind a secret serial chain.
  2. As a workflow author, I want my graph's visible dependency edges to be the true execution order, so that what I wrote is what runs.
  3. As a workflow author, I want a reviewable route over several independent writers to compile successfully, so that parallel implementation tracks can still reach a diff review gate.
  4. As a workflow author, I want a chain of writers (each declared to depend on the previous) to compile exactly as it does today, so that my existing routes do not change behaviour.
  5. As a workflow author, I want to keep writing the same block fields (id, kind, depends_on, instruction), so that adopting parallelism requires no new authoring vocabulary.
  6. As a parent orchestrator, I want a route with three independent slices to actually finish faster on a four-slot concurrency budget, so that the "trade space for time" thesis holds in practice.
  7. As a parent orchestrator, I want the compiled graph to stay within the declared max_total_nodes accounting, so that one injected aggregator node does not silently breach ceilings.
  8. As a child worker implementing one slice, I want clear ownership of my declared changed files, so that I can report them honestly and my output can be cross-checked mechanically.
  9. As the aggregator node, I want each writer's declared changed-file list supplied as structured input, so that I can detect write-set overlap without re-deriving it.
  10. As the aggregator node, I want to fail my own node loudly when two writers touched the same file, so that a corrupted merge state never reaches verification or review.
  11. As the aggregator node, I want to compute one fingerprint at the convergence point, so that downstream verification and review bind to a single post-merge state instead of N racing self-reports.
  12. As a verify worker, I want the implementation fingerprint and changed-file union actually injected into my inputs, so that my contract's "bind evidence to the supplied fingerprint" stops being an empty promise.
  13. As a review worker, I want implementation_fingerprint, changed_files, and verification output bound exactly as today, so that the diff-review gate and fingerprint echo settlement work unchanged.
  14. As a reviewer of compiled graphs, I want the review lifecycle, settlement, and recovery behaviour untouched, so that the fingerprint echo chain keeps its existing security properties.
  15. As a maintainer, I want the runtime (scheduling loop, durable lifecycle, spawn, capture) untouched, so that the blast radius of this change is confined to the authoring boundary.
  16. As a maintainer, I want previously-serialized graphs that were chains to remain byte-identical after compilation, so that regression risk is structurally bounded.
  17. As a maintainer, I want the block guide text to describe the real compiler behaviour, so that the model-facing docs never promise serialization again.
  18. As a maintainer, I want an ADR recording the aggregation decision, so that future readers find the rationale next to the existing authoring-authority ADR.
  19. As the config-repo maintainer, I want the runtime merge SHA clearly attributable to this change, so that runtime-compat.json can be bumped in the same change as the affected templates.
  20. As the config-repo maintainer, I want the "parallel slices" wording in the full project-development reference to be truthful after this lands, so that saved routes stop advertising serial behaviour.
  21. As a future author, I want the overlap check to be mechanical rather than prompt-discipline, so that the most common parallel-writer failure mode is caught by the engine, not by hope.

Implementation Decisions

  1. One seam, one change site. All work lands at the Block compilation step inside the single Workflow Authoring Check authority (per ADR-0001). The runtime consumes compiled node configs exactly as before; no new validation path, no runtime mutation.
  2. Two compilation regimes.
    • Total-ordered writers (chain): canonical-writer selection keeps today's behaviour byte-identically.
    • Parallel/partially-ordered writers: the compiler injects one aggregator node per affected implementation-review route; the previously-dead "no canonical writer" error becomes unreachable.
  3. Aggregator node shape. ID follows the existing -- child-node naming convention under the review block (collision with an author-defined ID throws, consistent with the existing duplicate-node error). It depends on every writer in the route, uses the read-only exploration worker (which has shell access but no write permission), is required: true, does not report to parent, and reuses the existing implementation output schema (union changed_files + single fingerprint) — so every downstream review-lifecycle check passes unchanged.
  4. Graph rewiring. The verify block's writer dependencies are re-pointed to the aggregator (non-writer dependencies preserved); the diff review's implementation reference points at the aggregator while its input-mapping keys and verification reference stay identical.
  5. Overlap gate at convergence. The aggregator contract compares writers' declared changed_files inputs; a non-empty intersection fails the node (loud, terminal) instead of submitting. Disjoint sets produce the union plus one fingerprint computed once at the convergence point.
  6. Verify fingerprint binding fixed. The canonical/aggregator implementation output (changed files + fingerprint) is bound into the verify node's inputs, making its existing contract text true.
  7. Serialization removed unconditionally. Writer blocks the author ordered with explicit dependencies stay ordered by those declarations alone; the compiler never injects ordering edges again.
  8. Fingerprint semantics unchanged. Fingerprint remains a worker-reported string verified by echo (implementation report → reviewer echo → settlement equality). The change centralizes where it is produced, not what it is.
  9. Documentation in the same change. The block guide's claim that the compiler serializes unordered writers is replaced by the parallel-plus-aggregator description and the triple-disjoint criterion (source files, generated artifacts, lockfiles must all be disjoint, and no shared build may be triggered) as plan-block contract guidance.
  10. ADR-0002 in the DAG ADR series records the aggregation decision and the preserved invariants, following the existing ADR format.

Testing Decisions

  • A good test here asserts compile-in → compile-out behaviour of the authoring boundary: given a block graph, the compiled node configs have the right edges, schemas, bindings, and review references. Tests must not reach into internal helpers or duplicate compiler logic.
  • The block compiler is a pure function, so all new behaviour is covered by unit tests at the compilation seam — the highest existing seam; no new seam is introduced.
  • The one existing test asserting forced serialization is rewritten to assert true parallelism plus aggregator injection. New cases: chain compiles unchanged; partial order (A→B with C independent) aggregates all three; aggregator ID collision throws; verify rewiring preserves non-writer dependencies; review bindings point at the aggregator; verify receives the fingerprint binding.
  • Prior art: the existing blocks compilation unit tests (compile-level assertions) and the review-lifecycle tests (hand-built node configs). The review-lifecycle tests remain untouched and must stay green — they are the proof that the review surface is invariant.
  • No new runtime tests are required (runtime untouched); the existing DAG lifecycle/review test suites must pass unchanged as the regression guard.
  • Verification command is the package's typecheck plus its test runner from the package directory (never from the repo root), per repository rules.

Out of Scope

  • The REJECT pathability fix (guard relocation + loop no-fail): separate ticket 2/4.
  • Pause-on-REJECT with the orchestrator watchdog (A4): decision deferred until ticket 2 produces operational data.
  • The verdict-disposal classifier table and tier-escalation wording: ticket 4/4, blocked by ticket 2.
  • The config-repo follow-up (truthful "parallel slices" wording, triple-disjoint plan contract, removal of the hand-rolled delivery-contract workaround, runtime-compat.json bump): separate repo, blocked by this ticket's merge SHA.
  • Author-declared write_set block fields; runtime-computed cryptographic fingerprints; per-node worktree isolation.
  • Lockfile / codegen-cache contention detection: mechanically unreachable by the overlap check; owned by plan-contract discipline in the config-repo ticket.

Further Notes

  • Implement this ticket through the project's own DAG routes (project-development) — the workstream is the first real exercise of the supervision loop it repairs.
  • Branch/review flow follows repo rules: feat/… branch, PR to dev (typecheck gate), then dev → main (full unit/E2E gate).
  • Cross-repo invariant per runtime-compat.json policy: this change alters block compilation rules and counts as an incompatible block rule — the config repo must merge its template updates and SHA bump together against this runtime merge SHA.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions