diff --git a/.github/aw/work-queue.md b/.github/aw/work-queue.md index 3995ba5bdf9..b513e011beb 100644 --- a/.github/aw/work-queue.md +++ b/.github/aw/work-queue.md @@ -4,29 +4,25 @@ description: Agent instructions for Git-backed work queue producers, dispatchers # Work Queue -Choose the queue pattern: Git storage for fair scheduling and Claim authority, or -[WorkQueueOps](../../docs/src/content/docs/patterns/workqueue-ops.md) for -issue-backed WorkQueueOps: lists, sub-issues, Discussions or cache backlogs. - -Use `tools.work-queue` for fair scheduling, immutable Work DAGs and Claim-scoped -effects. The causal `work-queue.jsonl` log is authoritative. Weights distribute -Claim opportunities, not CPU or successful completions. Protocol v3 always uses -Git; Issues/PRs are dependencies, not storage. Never set `storage`. WorkQueueOps -is a separate pattern. +Choose Git-backed `tools.work-queue` for fair scheduling, immutable Work DAGs +and Claim-scoped effects; `work-queue.jsonl` is authoritative. Protocol v3 has +no storage selector; Issues/PRs are dependencies, not storage. Weights distribute +Claim opportunities, not CPU or successful completions. For issue-backed WorkQueueOps +alternatives, see [WorkQueueOps](../../docs/src/content/docs/patterns/workqueue-ops.md). ## Select the role -- **Observer:** read/explain only; use ordinary configured report or `noop` - authorization. Do not submit, dispatch or finish Work. -- **Producer/dispatcher:** stage entitled task plans and bounded pool requests. - Queue-control authority does not permit arbitrary resource-writing outputs. -- **Worker:** require a compiler-supplied version-3 `claims` array. Never - construct/override reserved `work_queue_assignment` or caller context. +- **Observer:** read/explain only; never submit, dispatch or finish Work. +- **Producer/dispatcher:** submit entitled plans and request bounded pool work; + queue authority grants no arbitrary resource-writing outputs. +- **Worker:** require compiler-supplied version-3 `claims`; never override + reserved `work_queue_assignment` or caller context. -Trust the compiler/runtime role, never snapshot metadata or assignment alone; -don't downgrade declared workers. Activation authenticates run/workflow/revision, -and reruns cannot inherit attempt 1 authority. Report absent queues as -uninitialized; reject empty, malformed or unsupported logs. +Trust compiler/runtime role, not snapshot metadata; don't downgrade workers. +Activation authenticates run/workflow/revision; reruns cannot inherit attempt 1. +An absent branch is uninitialized/empty. Its first producer submit atomically +creates it with compiled Policy and Work; reads and dispatch alone never do. +Reject malformed or unsupported existing logs. ## Configure deployment @@ -39,17 +35,17 @@ tools: worker: true ``` -Dispatchers need `tools.work-queue: true`, an explicit -`safe-outputs.dispatch-workflow.workflows` allowlist and a bounded `max`. -The allowlist is compiler approval, not authority: installed Policy binds each -profile's exact workflow path, immutable SHA, authenticated principal, trust -domain and effect scope. Queue dispatch ignores moving `target-ref`. +Dispatchers need `tools.work-queue: true`, a worker allowlist and bounded +`dispatch-workflow.max`. Compiler-approved Policy binds each profile's workflow, +immutable SHA, principal, trust domain and effect scope; moving `target-ref` is +ignored. -Require administrator-installed Policy and protected queue-branch writers; -frontmatter provisions neither, and administrator status grants no producer -entitlement. Never give agents or snapshot MCP queue-write credentials. Read the -[deployment guide](../../docs/src/content/docs/guides/deploy-work-queue.md) only -for installation; writer-restriction automation remains deferred. +Configure producer entitlements and branch protections before enabling +producers; rulesets must match before branch creation. First submit installs +compiled Policy; later changes are admin-only. Admin status grants no producer +entitlement. Never expose queue-write credentials to agents/snapshot MCP. See +[deployment](../../docs/src/content/docs/guides/deploy-work-queue.md); +writer-restriction automation remains deferred. ## Mirror admitted Work with Issues @@ -66,16 +62,12 @@ Upgrade all closed-schema readers first. See the ## Plan and dispatch -- Use `work_queue_read` / `work_queue_explain` for the immutable activation - snapshot. Predictions may be stale; sorting cannot override Policy. -- Stage bounded, secret-free task/graph plans with `work_queue_submit`, within - installed producer pools, priorities and accounting keys. -- Request a pool prefix using `work_queue_dispatch_next` with `pool`, - `max_claims` and `max_dispatches`. Never select winning Work or a target, or - use legacy `dispatch_workflow` with `work_queue: {work_id: ...}`. -- Let trusted processing refresh the ledger, enforce Policy/DAG/scope and commit - a compatible fair prefix atomically. A CAS loser discards tentative choices - and charges before recomputing. Ordinary non-queue dispatch is separate. +- Read the immutable snapshot; predictions may be stale and cannot override + Policy. Submit bounded, secret-free plans within producer entitlements. +- Request a pool prefix with `work_queue_dispatch_next` (`pool`, `max_claims`, + `max_dispatches`). Never choose winners/targets or use scalar dispatch. +- Trusted processing refreshes the ledger and commits a compatible fair prefix. + CAS losers discard tentative choices/charges and recompute. ## Execute original Claims @@ -92,11 +84,11 @@ Upgrade all closed-schema readers first. See the - Treat finish as an intent: trusted processing publishes Completion before scoped effects and Result only after independently verified delivery. -Use `` advertised MCP subcommands with one JSON argument (not -structured-tool `command`/`description`). Dispatcher: `uninitialized` plus -null `snapshot_sha` means absent; stop with `noop`, not submit/dispatch. -Existing policyless ledgers are deployment failures; workers require Policy. -`status: "staged"` is no grant or launch. +Use advertised `` subcommands with one JSON argument, not +structured-tool `command`/`description`. `uninitialized` plus null SHA is empty; +first `work_queue_submit` atomically installs compiled Policy and Work. Submit +before dispatch. Existing policyless ledgers are failures; workers require +Policy. `status: "staged"` is no grant or launch. ## Dependencies and recovery diff --git a/.github/workflows/eslint-factory-dispatcher.lock.yml b/.github/workflows/eslint-factory-dispatcher.lock.yml index 9cb488982b9..99ca692a3e8 100644 --- a/.github/workflows/eslint-factory-dispatcher.lock.yml +++ b/.github/workflows/eslint-factory-dispatcher.lock.yml @@ -1,4 +1,4 @@ -# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"54d839d5ff9f5abb7ba653191bedbc6816268048c2e0aa8eb3cfdc86a134f8a2","body_hash":"4bb415c7d7802b8d74ce533efa73793e5d61feb099ab239a152c51f7805fb0c5","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.90"}} +# gh-aw-metadata: {"schema_version":"v4","frontmatter_hash":"54d839d5ff9f5abb7ba653191bedbc6816268048c2e0aa8eb3cfdc86a134f8a2","body_hash":"9f9f6f491d499aeba65b7e28d1a197099bbf12d49647e94881b36e10f937eaed","strict":true,"agent_id":"copilot","engine_versions":{"copilot":"1.0.90"}} # gh-aw-manifest: {"version":1,"secrets":["GH_AW_DEFAULT_OTLP_ENDPOINT","GH_AW_DEFAULT_OTLP_HEADERS","GH_AW_GITHUB_MCP_SERVER_TOKEN","GH_AW_GITHUB_TOKEN","GITHUB_TOKEN"],"actions":[{"repo":"actions/checkout","sha":"3d3c42e5aac5ba805825da76410c181273ba90b1","version":"v7.0.1"},{"repo":"actions/download-artifact","sha":"9000827ccba6bdab643e8b6fd33ac0654aef8333","version":"v8.0.2"},{"repo":"actions/github-script","sha":"3a2844b7e9c422d3c10d287c895573f7108da1b3","version":"v9.0.0"},{"repo":"actions/setup-node","sha":"949feb2413d6458794dcd2491c4babbbce0c15c1","version":"v7.1.0"},{"repo":"actions/upload-artifact","sha":"cf430e030ddbb5b0abf93d22962f4752f3646cd9","version":"v7.0.2"}],"containers":[{"image":"ghcr.io/github/gh-aw-firewall/agent:0.28.50","digest":"sha256:91fe4af8373c723d4d2e0706a0c35d310c2eeadaea5e0738c725b3861042f620","pinned_image":"ghcr.io/github/gh-aw-firewall/agent:0.28.50@sha256:91fe4af8373c723d4d2e0706a0c35d310c2eeadaea5e0738c725b3861042f620"},{"image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.28.50","digest":"sha256:11f12bcd3dde377ba822df7c670d6b937e9e0cebc46218946d4edc972d7fd965","pinned_image":"ghcr.io/github/gh-aw-firewall/api-proxy:0.28.50@sha256:11f12bcd3dde377ba822df7c670d6b937e9e0cebc46218946d4edc972d7fd965"},{"image":"ghcr.io/github/gh-aw-firewall/squid:0.28.50","digest":"sha256:cc9b6b979edf4aafbb3811c2bdc4891039d5346473afa88e304aa692167004f9","pinned_image":"ghcr.io/github/gh-aw-firewall/squid:0.28.50@sha256:cc9b6b979edf4aafbb3811c2bdc4891039d5346473afa88e304aa692167004f9"},{"image":"ghcr.io/github/gh-aw-mcpg:v0.4.30","digest":"sha256:ab5a436a1490438db473e4e3d4c973cb1d75e3cb233fb08b73d31b42d7d18fba","pinned_image":"ghcr.io/github/gh-aw-mcpg:v0.4.30@sha256:ab5a436a1490438db473e4e3d4c973cb1d75e3cb233fb08b73d31b42d7d18fba"},{"image":"ghcr.io/github/gh-aw-node","digest":"sha256:11c2c544876cdbaffefe0f7f1283b74838c209654b4934a78e23ed11f750490f","pinned_image":"ghcr.io/github/gh-aw-node@sha256:11c2c544876cdbaffefe0f7f1283b74838c209654b4934a78e23ed11f750490f"},{"image":"ghcr.io/github/github-mcp-server:v1.12.2","digest":"sha256:508a0857ec762b1ab1cece29193345b501fab1dd9d1228a7b617062954cecac6","pinned_image":"ghcr.io/github/github-mcp-server:v1.12.2@sha256:508a0857ec762b1ab1cece29193345b501fab1dd9d1228a7b617062954cecac6"}],"mcp_servers":[{"name":"github","tools":["get_commit","get_file_contents","get_latest_release","get_me","get_pull_request","get_pull_request_comments","get_pull_request_diff","get_pull_request_files","get_pull_request_review_comments","get_pull_request_reviews","get_pull_request_status","get_release_by_tag","get_tag","issue_read","list_branches","list_commits","list_issue_types","list_issues","list_pull_requests","list_releases","list_starred_repositories","list_tags","pull_request_read","search_code","search_issues","search_pull_requests","search_repositories"]},{"name":"safeoutputs","tools":["dispatch_workflow","eslint_miner","eslint_monster","eslint_refiner","missing_data","missing_tool","noop","report_incomplete"]},{"name":"work-queue","tools":["work_queue_claim_finish","work_queue_dispatch_next","work_queue_explain","work_queue_read","work_queue_submit"]}],"threat_detection":{"mode":"enabled"}} # This file was automatically generated by gh-aw. DO NOT EDIT. To debug this workflow, load the skill at https://github.com/github/gh-aw/blob/main/debug.md # diff --git a/.github/workflows/eslint-factory-dispatcher.md b/.github/workflows/eslint-factory-dispatcher.md index 4616fbd480c..74ab9ecb5f1 100644 --- a/.github/workflows/eslint-factory-dispatcher.md +++ b/.github/workflows/eslint-factory-dispatcher.md @@ -52,9 +52,10 @@ Do not invent additional tasks or broaden resource scopes. Read the activation snapshot first with `work_queue_read`, using `{"pool":"default","limit":32}`. Check `queue_state`, not just `total`: `"uninitialized"` with a null snapshot SHA means the queue branch does not -exist yet. Treat it as an empty backlog and stop with `noop`; there is no -installed Policy to authorize submission or dispatch. Surface read/tool errors -explicitly; never turn an error or an existing policyless ledger into an +exist yet. Treat it as an empty backlog; the first trusted producer submission +will atomically bootstrap the branch with its compiler-approved Policy and Work. +Do not dispatch until that submission has been admitted. Surface read/tool +errors explicitly; never turn an error or an existing policyless ledger into an empty-backlog report. When `` advertises the wrappers, invoke their subcommands with one JSON @@ -66,7 +67,7 @@ work-queue work_queue_read '{"pool":"default","limit":32}' Use the advertised MCP tools directly when wrappers are not available. -For an initialized queue, submit the prepared cohort: +Submit the prepared cohort whether the queue is initialized or absent: Call `work_queue_submit` once with `{"nodes": }` before requesting any grants. Date-keyed graph/node identities make repeated submissions on the same UTC day idempotent, @@ -99,8 +100,7 @@ worker-name allowlist; it does not replace the installed policy's profile, revision, or principal binding. Do not call ordinary `dispatch_workflow` or typed per-worker dispatch tools. -The authenticated operator provisions the queue Policy and grants this -dispatcher's authenticated principal producer entitlement before it can submit -or dispatch. Do not bootstrap Policy. If producer entitlement is missing from -an existing queue, report the admission failure explicitly; do not call `noop` -or ordinary worker dispatch. +Safe-output processing uses only the compiler-approved Policy proposal; never +choose or modify Policy. The proposal must grant this dispatcher's principal +producer entitlement. If bootstrap or admission fails, report the failure; do +not bypass it with `noop` or ordinary worker dispatch. diff --git a/actions/setup/js/eslint_factory_portfolio.test.cjs b/actions/setup/js/eslint_factory_portfolio.test.cjs index ca9b19ff04c..0b83ab8271d 100644 --- a/actions/setup/js/eslint_factory_portfolio.test.cjs +++ b/actions/setup/js/eslint_factory_portfolio.test.cjs @@ -102,7 +102,8 @@ describe("ESLint factory producer and dispatcher", () => { expect(source).toContain("run.created_at"); expect(source).toContain("cat /tmp/gh-aw/agent/eslint-factory-plan.json"); expect(source.indexOf("Call `work_queue_submit` once")).toBeLessThan(source.indexOf("request the trusted scheduler")); - expect(source).toContain("Do not bootstrap Policy"); + expect(source).toContain("will atomically bootstrap the branch with its compiler-approved Policy and Work."); + expect(source).toContain("Safe-output processing uses only the compiler-approved Policy proposal; never"); }); it("executes the actual preparation step with only two metadata reads and no ambient filesystem or credentials", async () => { diff --git a/actions/setup/js/work_queue_control_adapter.cjs b/actions/setup/js/work_queue_control_adapter.cjs index f520ee0c40f..2c22326fd6d 100644 --- a/actions/setup/js/work_queue_control_adapter.cjs +++ b/actions/setup/js/work_queue_control_adapter.cjs @@ -2,7 +2,7 @@ "use strict"; const { SAFE_OUTPUT_E001 } = require("./error_codes.cjs"); const { parseStrictJSON, utf8Compare } = require("./work_queue_codec.cjs"); -const { loadQueue } = require("./work_queue_binding.cjs"); +const { loadQueue, policyProposalFor } = require("./work_queue_binding.cjs"); const { readStagedIntentBatch } = require("./work_queue_intents.cjs"); const { DEFAULT_INTENT_PATH } = require("./work_queue_mcp_server.cjs"); const { API_VERSION, nativeId } = require("./work_queue_native.cjs"); @@ -126,6 +126,11 @@ function createCompilerDependencyResolver(options) { return Object.freeze({ scopes, getClient }); } +function dependencyStateForIntents(latest, intents, options) { + const bootstrapPolicy = !latest.projection.policy && latest.sha === null && intents.some(intent => intent.kind === "submit") ? policyProposalFor(options) : undefined; + return bootstrapPolicy ? { ...latest.projection, policy: bootstrapPolicy } : latest.projection; +} + /** @param {CompilerControlOptions} [options] */ async function main(options = {}) { /** @type {CompilerControlOptions & {githubClient: object, context: {repo: {owner: string, repo: string}}}} */ @@ -147,7 +152,7 @@ async function main(options = {}) { } if (intents.some(intent => ["submit", "dispatch_next"].includes(intent.kind)) && !preview) { const latest = await loadQueue({ ...configured, policyProposal: undefined, initializationContext: undefined }); - configured.dependencyResolver = createCompilerDependencyResolver({ ...configured, state: latest.projection }); + configured.dependencyResolver = createCompilerDependencyResolver({ ...configured, state: dependencyStateForIntents(latest, intents, configured) }); } if (launching) { const token = configured.config?.["github-token"]; @@ -164,4 +169,4 @@ async function main(options = {}) { return require("./work_queue_dispatch.cjs").main(configured); } -module.exports = { credentialBindings, readOnlyRepositoryClient, createCompilerDependencyResolver, main }; +module.exports = { credentialBindings, readOnlyRepositoryClient, createCompilerDependencyResolver, dependencyStateForIntents, main }; diff --git a/actions/setup/js/work_queue_control_adapter_checks.cjs b/actions/setup/js/work_queue_control_adapter_checks.cjs index 9a9db24ecf9..092359d75c6 100644 --- a/actions/setup/js/work_queue_control_adapter_checks.cjs +++ b/actions/setup/js/work_queue_control_adapter_checks.cjs @@ -4,7 +4,7 @@ const assert = require("node:assert/strict"); const fs = require("node:fs"); const path = require("node:path"); const { temporaryDirectory } = require("./work_queue_effect_test_helpers.cjs"); -const { createCompilerDependencyResolver, credentialBindings, main } = require("./work_queue_control_adapter.cjs"); +const { createCompilerDependencyResolver, credentialBindings, dependencyStateForIntents, main } = require("./work_queue_control_adapter.cjs"); const { queueFixture, WORKFLOW, DISPATCHER, REF, REPOSITORY } = require("./work_queue_lifecycle.test_helpers.cjs"); const { main: captureIntentOrigin } = require("./capture_work_queue_intent_origin.cjs"); const { canonical } = require("./work_queue_codec.cjs"); @@ -57,6 +57,16 @@ function registerTests({ describe, it }) { } }); + it("uses the compiled Policy proposal only to resolve dependencies for a first submit on an absent branch", () => { + const fixture = queueFixture({ granted: false }); + const latest = { sha: null, projection: { policy: null, credential_generation: "initial" } }; + const state = dependencyStateForIntents(latest, [{ kind: "submit" }], { policyProposal: fixture.policy }); + assert.equal(canonical(state.policy), canonical(fixture.policy)); + assert.equal(latest.projection.policy, null); + assert.equal(dependencyStateForIntents(latest, [{ kind: "dispatch_next" }], { policyProposal: fixture.policy }), latest.projection); + assert.equal(dependencyStateForIntents({ ...latest, sha: "existing" }, [{ kind: "submit" }], { policyProposal: fixture.policy }), latest.projection); + }); + it("rejects spoofed credential kinds, App metadata, human bot lookups, lossy IDs and unavailable identity APIs without a fallback", async () => { let reads = 0; const client = { diff --git a/actions/setup/js/work_queue_dispatch.cjs b/actions/setup/js/work_queue_dispatch.cjs index 6e21042f7c0..24b50369ef3 100644 --- a/actions/setup/js/work_queue_dispatch.cjs +++ b/actions/setup/js/work_queue_dispatch.cjs @@ -11,7 +11,7 @@ const { assignmentsForRequest, generateRequestOperations, validateWorkerContinua const store = require("./work_queue_store.cjs"); const { normalizeAssignment, normalizeClaimScope, readClaimScopeContext } = require("./work_queue_claim_scope.cjs"); const { readStagedIntentBatch, requestIdForIntent, requestForIntent, normalizeDispatchParameters, normalizeSubmitParameters } = require("./work_queue_intents.cjs"); -const { assertPolicyProposal, loadQueue, publishOperations, validateStoredAssignment, expectedWorkerRun, bindingForRun } = require("./work_queue_binding.cjs"); +const { assertPolicyProposal, loadQueue, policyProposalFor, publishOperations, validateStoredAssignment, expectedWorkerRun, bindingForRun } = require("./work_queue_binding.cjs"); const { authenticatePublisher, authenticateIntentPublisher, immutableRef, fetchNativeRun, postQueueDispatch, validateNativeRun } = require("./work_queue_native.cjs"); const { dependencyKey, allowedScope, resolveDependencies, resolveExternalEdges } = require("./work_queue_dependency_resolver.cjs"); const { lifecycleEvidence, reconcileDispatch, reconcileQueue, releaseAssignment } = require("./work_queue_reconciler.cjs"); @@ -437,16 +437,18 @@ async function processParsedWorkQueueIntents(options, runtime, intents, errors) } const trustedContext = await intentContext(options, intent, { recoverAccepted: true }); const latest = await loadQueue({ ...options, policyProposal: undefined, initializationContext: undefined }); - if (!latest.projection.policy) throw new Error("policy_missing"); + const bootstrapPolicy = !latest.projection.policy && latest.sha === null && ["producer", "dispatcher"].includes(trustedContext.role) ? policyProposalFor(options) : undefined; + const admissionState = bootstrapPolicy ? { ...latest.projection, policy: bootstrapPolicy } : latest.projection; + if (!admissionState.policy) throw new Error("policy_missing"); if (!Number.isSafeInteger(trustedContext.created_at) || trustedContext.created_at < 0) throw new Error("publisher_origin_time_missing"); const prior = latest.projection.requests.get(requestIdForIntent(trustedContext, intent.intent_id)); - if (!prior) assertPolicyProposal(latest.projection, options); + if (!prior && !bootstrapPolicy) assertPolicyProposal(latest.projection, options); const parameters = prior ? acceptedSubmissionParameters(latest.projection, trustedContext, intent.parameters, prior) : await resolveAdmissionResources( options, - latest.projection, - normalizeSubmitParameters(inheritWorkerSubmission(latest.projection, trustedContext, intent.parameters), latest.projection.policy, trustedContext.created_at, latest.projection), + admissionState, + normalizeSubmitParameters(inheritWorkerSubmission(admissionState, trustedContext, intent.parameters), admissionState.policy, trustedContext.created_at, admissionState), trustedContext ); const request = requestForIntent(trustedContext, intent.intent_id, "submit", parameters); @@ -459,6 +461,7 @@ async function processParsedWorkQueueIntents(options, runtime, intents, errors) context: trustedContext, actor: actorFromContext(trustedContext), request, + ...(bootstrapPolicy === undefined ? {} : { policyProposal: bootstrapPolicy }), remediationVerifier: options.remediationVerifier, generateOperations: (state, stable, actor, at, id) => { assertPolicyProposal(state, options); diff --git a/actions/setup/js/work_queue_documentation.test.cjs b/actions/setup/js/work_queue_documentation.test.cjs index 03b162f717f..57d20abba31 100644 --- a/actions/setup/js/work_queue_documentation.test.cjs +++ b/actions/setup/js/work_queue_documentation.test.cjs @@ -58,8 +58,8 @@ describe("work-queue deployment documentation", () => { expect(instructions.trim().split(/\s+/).length).toBeLessThanOrEqual(800); expect(instructions).not.toContain("```json"); expect(instructions).toContain("issue-backed WorkQueueOps"); - expect(instructions).toContain("null `snapshot_sha` means absent; stop with `noop`"); - expect(instructions).toContain("Existing policyless ledgers are deployment failures"); + expect(instructions).toContain("`uninitialized` plus null SHA is empty"); + expect(instructions).toContain("Existing policyless ledgers are failures"); expect(readRepositoryFile(referencePath)).toContain("Work queues can be Git-backed or issue-backed"); for (const file of [deploymentPath, referencePath, specificationPath]) { expect(instructions).toContain(file.replace(/^docs\//, "../../docs/")); @@ -92,8 +92,8 @@ describe("work-queue deployment documentation", () => { const dispatcher = readRepositoryFile(".github/workflows/eslint-factory-dispatcher.md"); expect(dispatcher).toContain('work-queue work_queue_read \'{"pool":"default","limit":32}\''); expect(dispatcher).toContain("Check `queue_state`, not just `total`"); - expect(dispatcher).toContain("Treat it as an empty backlog and stop with `noop`"); - expect(dispatcher).toMatch(/no\s+installed Policy to authorize submission or dispatch/); + expect(dispatcher).toContain("Treat it as an empty backlog; the first trusted producer submission"); + expect(dispatcher).toContain("Do not dispatch until that submission has been admitted"); expect(dispatcher).toContain("existing policyless ledger"); expect(dispatcher).toContain('work-queue work_queue_dispatch_next \'{"pool":"default","max_claims":3,"max_dispatches":3}\''); expect(dispatcher).toContain('`status: "staged"`'); diff --git a/actions/setup/js/work_queue_replay.cjs b/actions/setup/js/work_queue_replay.cjs index 9edc582ef74..ee8cd730f56 100644 --- a/actions/setup/js/work_queue_replay.cjs +++ b/actions/setup/js/work_queue_replay.cjs @@ -74,7 +74,7 @@ const OP_FIELDS = { const REQUEST_KINDS = { policy: ["Policy"], control: ["Control", "WorkPriority"], - submit: ["Work"], + submit: ["Policy", "Work"], dispatch_next: ["Observation", "Claim"], finish: ["Completion", "ClaimCancellation", "WorkCancellation"], observe: ["Observation"], @@ -89,7 +89,7 @@ const REQUEST_KINDS = { }; const OP_ROLES = { - Policy: ["administrator"], + Policy: ["administrator", "producer", "dispatcher"], Control: ["administrator"], Work: ["producer", "dispatcher", "worker", "administrator"], Claim: ["dispatcher", "administrator", "worker"], @@ -262,7 +262,11 @@ function validateCommit(commit) { if (!REQUEST_KINDS[commit.request.kind].includes(operation.kind) || !OP_ROLES[operation.kind].includes(commit.actor.role)) throw queueError("unauthorized_operation", `${commit.actor.role} cannot publish ${operation.kind} through ${commit.request.kind}`); } - if (commit.request.kind === "submit" && canonical(commit.request.parameters.nodes) !== canonical(commit.operations)) throw queueError("request_invalid", "submission differs from validated stable semantics"); + if (commit.request.kind === "submit") { + const ordinary = canonical(commit.request.parameters.nodes) === canonical(commit.operations); + const bootstrap = commit.operations[0]?.kind === "Policy" && commit.operations.slice(1).every(operation => operation.kind === "Work") && canonical(commit.request.parameters.nodes) === canonical(commit.operations.slice(1)); + if (!ordinary && !bootstrap) throw queueError("request_invalid", "submission differs from validated stable semantics"); + } if ( commit.request.kind === "checkpoint" && (commit.operations.length !== 1 || @@ -975,12 +979,29 @@ function replayOrdered(ordered) { if (ordinal === 0 && !offset) state.repository = commit.actor.repository; const priorRequest = state.requests.get(commit.request.id); if (priorRequest) throw queueError("request_reused", priorRequest.request.fingerprint === commit.request.fingerprint ? "committed request cannot appear in another commit" : "request identity has different semantics"); + const bootstrap = + ordinal === 0 && + !offset && + commit.request.kind === "submit" && + ["producer", "dispatcher"].includes(commit.actor.role) && + commit.operations[0]?.kind === "Policy" && + commit.operations.length > 1 && + commit.operations.slice(1).every(operation => operation.kind === "Work") && + canonical(commit.request.parameters.nodes) === canonical(commit.operations.slice(1)); if (ordinal === 0 && !offset && (commit.operations.length !== 1 || commit.operations[0].kind !== "Policy" || commit.request.kind !== "policy" || commit.actor.role !== "administrator")) throw queueError("policy_missing", "genesis must install exactly one mandatory Policy"); if (state.policy && commit.actor.repository !== state.repository) throw queueError("actor_unauthorized", "foreign queue repository origin"); const policies = commit.operations.filter(operation => operation.kind === "Policy"); if (policies.length) { - if (policies.length !== 1 || commit.operations.length !== 1 || commit.policy_epoch !== policies[0].epoch || epochs.has(policies[0].epoch) || (state.policy && !quiescent(state))) + if ( + policies.length !== 1 || + (!bootstrap && commit.operations.length !== 1) || + (bootstrap && (commit.operations[0] !== policies[0] || commit.actor.role === "administrator")) || + (!bootstrap && state.policy && commit.actor.role !== "administrator") || + commit.policy_epoch !== policies[0].epoch || + epochs.has(policies[0].epoch) || + (state.policy && !quiescent(state)) + ) throw queueError("policy_not_quiescent", "Policy epoch changes require a drained queue and new immutable identity"); epochs.add(policies[0].epoch); state.policy = policies[0].policy; diff --git a/actions/setup/js/work_queue_store.cjs b/actions/setup/js/work_queue_store.cjs index 321c493ac87..720cae5390e 100644 --- a/actions/setup/js/work_queue_store.cjs +++ b/actions/setup/js/work_queue_store.cjs @@ -411,9 +411,19 @@ async function publishWorkQueueRequest({ const recovered = stableRequestResult(current, stable, stableActor); if (recovered) return recovered; if (initializeOnly && current.sha) return { ...current, operations: [], assignments: [], publishedNow: false, reused: false, persisted: false, recovered: false, idempotent: false, rejected: [] }; + let bootstrapOperation; if (!current.sha) { if (stableActor.role === "worker") throw queueError("claim_scope_invalid", "a worker cannot have effective Claim authority in an uninitialized queue"); - if (stable.kind !== "policy") { + if (stable.kind === "submit" && ["producer", "dispatcher"].includes(stableActor.role) && initializationContext === undefined) { + if (policyProposal === undefined) throw queueError("policy_missing", "first producer submission requires a compiler-approved Policy proposal"); + const policy = validatePolicy(structuredClone(policyProposal)); + const seed = createHash("sha256").update(stable.id, "utf8").digest("hex"); + bootstrapOperation = { kind: "Policy", epoch: `epoch_${seed}`, policy }; + current = { + ...current, + state: { ...current.state, repository: stableActor.repository, policy, policy_epoch: bootstrapOperation.epoch }, + }; + } else if (stable.kind !== "policy") { log.debug("request.genesis.required"); // Genuine genesis only: separate explicit Policy request, never overlay an // installed epoch with a compiled proposal after a publication conflict. @@ -453,6 +463,7 @@ async function publishWorkQueueRequest({ const generated = await generateOperations(generateOperations === generateRequestOperations ? current.state : structuredClone(current.state), structuredClone(stable), structuredClone(stableActor), at, id, structuredClone(observations)); const decision = Array.isArray(generated) ? { operations: generated } : generated; if (!decision || !Array.isArray(decision.operations)) throw queueError("request_invalid", "candidate generator must return operations"); + if (bootstrapOperation) decision.operations.unshift(bootstrapOperation); log.debug("request.candidate.generated", { operations: decision.operations.length, observations: observations.length }); if (!decision.operations.length) { log.debug("request.publish.no_operations"); diff --git a/actions/setup/js/work_queue_store_checks.cjs b/actions/setup/js/work_queue_store_checks.cjs index 502c12ee294..936db9b116b 100644 --- a/actions/setup/js/work_queue_store_checks.cjs +++ b/actions/setup/js/work_queue_store_checks.cjs @@ -890,6 +890,26 @@ function registerTests({ describe, it }) { } assert.equal(fake.state.updates, 0); }); + it("provisions an absent queue branch from the compiled Policy on the first producer submission", async () => { + const fake = fakeGitHub(); + const policy = defaultPolicy({ repository: "owner/repo", principal: "1001", ref: fake.state.defaultRevision }); + const nodes = [newWork({ task: "first safe-output submission" }, "bootstrap-graph", "root", "default", policy, 100)]; + const request = newRequest("producer-bootstrap", "submit", producer, { nodes }); + const result = await publishWorkQueueRequest(options(fake, request, producer, { policyProposal: policy, maxRetries: 0 })); + assert.equal(result.publishedNow, true); + assert.equal(fake.refs.has("work-queue"), true); + assert.equal(result.transactions.length, 1); + assert.equal(result.commit.actor.role, "producer"); + assert.deepEqual( + result.commit.operations.map(operation => operation.kind), + ["Policy", "Work"] + ); + assert.deepEqual(result.state.policy, policy); + assert.ok(fake.state.calls.includes(`getContent:.github/workflows/worker.lock.yml@${fake.state.defaultRevision}`)); + assert.ok(fake.state.calls.includes("getWorkflow:worker.lock.yml")); + await publishWorkQueueRequest(options(fake, request, producer, { policyProposal: policy, maxRetries: 0 })); + assert.equal(fake.log().length, 1); + }); it("installs mandatory defaults only on genuine genesis and does not overlay authoritative installed Policy", async () => { const fake = fakeGitHub(); const policy = defaultPolicy({ repository: "owner/repo", principal: "1001" }); diff --git a/actions/setup/js/write_work_queue_snapshot.cjs b/actions/setup/js/write_work_queue_snapshot.cjs index a2551a0bc5f..e078526d62d 100644 --- a/actions/setup/js/write_work_queue_snapshot.cjs +++ b/actions/setup/js/write_work_queue_snapshot.cjs @@ -10,9 +10,17 @@ const { readInboundWorkQueueAssignment, resolveWorkQueueRuntime } = require("./a const { authenticatePublisher, validateNativeRun } = require("./work_queue_native.cjs"); const { bindWorkerAssignment, loadQueue, validateStoredAssignment, expectedWorkerRun } = require("./work_queue_binding.cjs"); const { isStagedMode } = require("./safe_output_helpers.cjs"); +const { WORK_QUEUE_BRANCH } = require("./work_queue_store.cjs"); const SNAPSHOT_PATH = "/tmp/gh-aw/work-queue.snapshot.json"; +function activationMessage({ worker, staged, latest, runtime, branch }) { + if (worker) return `${worker.claims.length} immutable Claims authenticated${staged ? " for staged preview (not bound)" : " and bound"}`; + if (latest.sha === null) return `queue uninitialized; the first trusted producer submission can provision the protected queue branch '${branch}' and install its compiled Policy`; + if (runtime.role === "observer") return "read-only observer"; + return "unassigned queue-control context"; +} + function resolveWorkerAssignment(payload, transactions) { const assignment = readInboundWorkQueueAssignment(payload); if (!assignment) return null; @@ -66,9 +74,7 @@ async function main(options = {}) { fs.writeFileSync(outputPath, encoded, { mode: 0o444 }); fs.chmodSync(outputPath, 0o444); configuration.core.setOutput?.("work_queue_origin", canonical(snapshot.origin)); - configuration.core.info( - `Work queue activation: ${worker ? `${worker.claims.length} immutable Claims authenticated${staged ? " for staged preview (not bound)" : " and bound"}` : runtime.role === "observer" ? "read-only observer" : "unassigned queue-control context"}; snapshot captured` - ); + configuration.core.info(`Work queue activation: ${activationMessage({ worker, staged, latest, runtime, branch: options.branch ?? WORK_QUEUE_BRANCH })}; snapshot captured`); return snapshot; } diff --git a/actions/setup/js/write_work_queue_snapshot.test.cjs b/actions/setup/js/write_work_queue_snapshot.test.cjs index bcc30821d8f..f556db8a731 100644 --- a/actions/setup/js/write_work_queue_snapshot.test.cjs +++ b/actions/setup/js/write_work_queue_snapshot.test.cjs @@ -4,7 +4,7 @@ import fs from "fs"; import path from "path"; import os from "node:os"; import { main, resolveWorkerAssignment } from "./write_work_queue_snapshot.cjs"; -import { loadWorkQueueSnapshot } from "./work_queue_mcp_server.cjs"; +import { loadWorkQueueSnapshot, readWorkQueueState } from "./work_queue_mcp_server.cjs"; import { normalizeRuntimeMessage, readClaimScopeContext } from "./work_queue_claim_scope.cjs"; import { queueFixture, REF, REPOSITORY, WORKFLOW, DISPATCHER } from "./work_queue_lifecycle.test_helpers.cjs"; import { newRequest, newState, replayTransactions, serializeTransactionLog } from "./work_queue_replay.cjs"; @@ -177,6 +177,21 @@ describe("authenticated immutable activation snapshots", () => { expect(loadWorkQueueSnapshot(options.snapshotPath).projection.policy_epoch).toBe("e1"); }); + it("reports an absent queue as uninitialized without claiming worker readiness or installing Policy", async () => { + const { options } = setup({ granted: false }); + const initialize = vi.fn(); + const snapshot = await main({ + ...options, + branch: "custom-queue", + readWorkQueueLog: async () => ({ sha: null, transactions: [], state: newState() }), + initializeWorkQueue: initialize, + }); + expect(snapshot).toMatchObject({ sha: null, worker: null, role: "dispatcher" }); + expect(initialize).not.toHaveBeenCalled(); + expect(readWorkQueueState(loadWorkQueueSnapshot(options.snapshotPath)).queue_state).toBe("uninitialized"); + expect(options.core.info).toHaveBeenCalledWith(expect.stringContaining("protected queue branch 'custom-queue'")); + }); + it("rejects actual oversized encoded framing before creating a snapshot or publishing origin output", async () => { const { fixture, options } = setup({ granted: false }); const before = fixture.transactions; diff --git a/docs/src/content/docs/guides/deploy-work-queue.md b/docs/src/content/docs/guides/deploy-work-queue.md index c8280b50775..339fff97849 100644 --- a/docs/src/content/docs/guides/deploy-work-queue.md +++ b/docs/src/content/docs/guides/deploy-work-queue.md @@ -1,6 +1,6 @@ --- title: How to deploy a work queue -description: Configure trusted producers, dispatchers, workers and an administrator-installed Git-backed queue policy. +description: Configure trusted producers, dispatchers, workers and first-submit bootstrap for a Git-backed queue. --- Use this guide to deploy a native Git-backed queue with `tools.work-queue`. @@ -10,11 +10,12 @@ use GitHub read tools and safe outputs, but do not provide native fair scheduling, Claim authority, or verified dependency graphs. Native version-3 `tools.work-queue` always uses Git; no storage selector is available. -Before you begin, prepare a trusted producer identity and an approved worker. -The producer submits tasks; the dispatcher requests assignments; the worker -processes them. You also need a Policy (the queue's scheduling and authorization -rules) and a protected queue branch. Workflow frontmatter does not install -Policy or configure restrictions on who can write to that branch. +Before you begin, prepare a trusted producer identity, an approved worker, and +a compiler-approved Policy proposal (the queue's scheduling and authorization +rules). The first accepted producer submission installs that proposal with its +Work when the branch is absent. Protect the queue branch before enabling +producers; workflow frontmatter alone does not install Policy or configure +restrictions on who can write to that branch. ## Publish the worker and dispatcher @@ -125,12 +126,12 @@ The ledger is the queue's transaction log. Its 64 MiB ordinary budget and to 80 MiB. If you add accounting keys, retain `"": 1` and explicitly grant producers permission to use each additional key. -## Protect the queue branch and install Policy +## Protect the queue branch and bootstrap Policy -Before installing Policy, configure branch protections separately. Allow only -the trusted operator or runtime host to write to the queue branch. Prevent -force updates and branch deletion, and ensure that workflow-agent credentials -cannot bypass these rules. +Before enabling a producer, configure a branch/ruleset pattern for the queue +branch. Allow only trusted publishers to write it, prevent force updates and +deletion, and ensure that workflow-agent credentials cannot bypass these rules. +The ruleset must apply before the branch exists. > [!WARNING] > Automated verification and provisioning of writer restrictions are still @@ -138,9 +139,14 @@ cannot bypass these rules. > You must establish them independently. See the > [implementation coverage table](/gh-aw/specs/work-queue-specification/#91-implementation-coverage-and-remaining-requirements). -Make sure the configured worker route is active. Then run the following command -as an explicitly authenticated administrator with permission to update the -protected queue branch: +Make sure the configured worker routes are active and the trusted producer +workflow embeds its complete compiler-approved Policy proposal. On an absent +branch, the first accepted `work_queue_submit` safe output atomically creates +the branch with one Policy-and-Work genesis commit. No administrator seed is +required. Dispatch-only requests, reads, and snapshots do not provision it. + +For an explicit operator-managed genesis or a later Policy update, an +authenticated administrator may still run: ```bash gh aw work-queue --repo github/gh-aw policy \ @@ -148,13 +154,16 @@ gh aw work-queue --repo github/gh-aw policy \ ``` The default queue branch is `work-queue`. To use another protected branch, put -`--branch QUEUE_BRANCH` before `policy`. +`--branch QUEUE_BRANCH` before `policy`. If Policy is seeded explicitly, the +subsequent producer submission must match that installed Policy proposal. Before changing Policy later, make the queue quiescent: settle all nonterminal (unfinished) Work, outstanding reservations, and unresolved delivery of completed Work. Cancellation alone does not stop a worker or release its reservation; reconcile exact termination or nonlaunch evidence separately. -Frontmatter never installs or updates Policy. +Frontmatter alone does not install or update Policy; the first accepted producer +submission installs its compiled proposal, and later changes remain +administrator-only. ## Enable backing Issue projection diff --git a/docs/src/content/docs/patterns/daily-report-portfolio.md b/docs/src/content/docs/patterns/daily-report-portfolio.md index 0a576753b7c..f49ff08648a 100644 --- a/docs/src/content/docs/patterns/daily-report-portfolio.md +++ b/docs/src/content/docs/patterns/daily-report-portfolio.md @@ -17,9 +17,10 @@ Completion and verified delivery share that transaction log. > [!IMPORTANT] > The sources are an orchestration example, not an already provisioned live -> deployment. An authenticated administrator must install the complete queue -> Policy and verified immutable worker bindings. Compiling a workflow does not -> install policy, protect the queue branch, or launch workers. +> deployment. Configure the complete compiler-approved Policy proposal, +> verified immutable worker bindings, and queue-branch protections before +> enabling a producer. Its first submission atomically bootstraps Policy and +> Work; compiling a workflow alone does not protect the branch or launch workers. ## Ten reports, one dispatcher @@ -114,8 +115,8 @@ dispatcher requires a new calendar-budget design. ## Dispatcher source and consuming prompt The following source excerpt omits engine settings and the preparation step. -The dispatch allowlist approves routes; the installed Policy supplies actual -immutable revisions, launch principals and resource scope. +The dispatch allowlist approves routes; the compiler-approved Policy proposal +supplies actual immutable revisions, launch principals and resource scope. ```aw wrap title=".github/workflows/daily-report-dispatcher.md (excerpt)" --- diff --git a/docs/src/content/docs/patterns/linter-factory.md b/docs/src/content/docs/patterns/linter-factory.md index ca0cf6b382b..2c40507df70 100644 --- a/docs/src/content/docs/patterns/linter-factory.md +++ b/docs/src/content/docs/patterns/linter-factory.md @@ -25,14 +25,17 @@ ledgers. | Workflow | Assigned work | Intended outputs | | --- | --- | --- | -| [Dispatcher](https://github.com/github/gh-aw/blob/main/.github/workflows/eslint-factory-dispatcher.md) | Admit the daily cohort, then request a fair prefix within the installed pool policy | Three independent tasks and a bounded dispatch request; report an absent Policy explicitly | +| [Dispatcher](https://github.com/github/gh-aw/blob/main/.github/workflows/eslint-factory-dispatcher.md) | Admit the daily cohort, then request a fair prefix within the compiled pool policy | Three independent tasks and a bounded dispatch request; treat an absent queue as empty and bootstrap on first submit | | [Miner](https://github.com/github/gh-aw/blob/main/.github/workflows/eslint-miner.md) | Find and implement a useful new ESLint rule | At most one draft rule PR per Claim; cancel write-capable Work when no rule is needed | | [Refiner](https://github.com/github/gh-aw/blob/main/.github/workflows/eslint-refiner.md) | Identify false positives, missing edge cases, or weak diagnostics | Up to three issues, a discussion, and one immutable memory snapshot per Claim | | [Monster](https://github.com/github/gh-aw/blob/main/.github/workflows/eslint-monster.md) | Group actionable diagnostics and arrange remediation | Issue updates, Copilot assignments, and a discussion; cancel write-capable Work after a clean scan | -An authenticated operator must first install Policy, producer entitlements, -worker profiles, immutable workflow revisions, and resource scopes. An -authorized producer explicitly admits Work with its output contract. The +Configure producer entitlements, worker profiles, immutable workflow revisions, +resource scopes, and a matching protected-branch ruleset before enabling the +factory. The compiler embeds the validated Policy proposal. The first trusted +producer submit installs that Policy with its Work and creates the queue branch; +no administrator needs to seed the queue. An authorized producer explicitly +admits Work with its output contract. The supplied workflows do **not** automatically create a miner-to-refiner-to-monster DAG, and creating a refinement issue does not submit another Work. @@ -50,7 +53,15 @@ does not itself admit any Work. Workers still require authenticated assignments; they do not seed tasks. The separate `daily-report-dispatcher` uses the `daily-reports` pool and is not this factory's producer. -### Provision the factory policy +### Bootstrap the factory policy + +Activation reports a missing queue branch as `queue_state: "uninitialized"` +without writing to it. The dispatcher can continue as a producer: its first +`work_queue_submit` safe output atomically creates the branch with the compiled +Policy and admitted Work. A dispatch-only request cannot bootstrap the queue. +Protect the `work-queue` branch name with a ruleset pattern before enabling the +dispatcher; do not substitute an ordinary workflow dispatch for a queue +assignment. The checked-in policy generator provides three immutable worker profiles, singleton assignments, three native slots, and a 30-pending-task limit: @@ -63,13 +74,15 @@ node actions/setup/js/eslint_factory_portfolio.cjs policy \ --file eslint-factory-policy.json --epoch eslint-factory-v1 ``` -Generation does not authenticate principals, install Policy, protect the queue -branch, or launch workers. Use verified positive decimal principal IDs from the -trusted producer and dispatch credential flows, not display names or assumed -`github.actor` values. Follow the [deployment guide](/gh-aw/guides/deploy-work-queue/) -before the administrator-only installation command. If the queue already serves -other pools, retain their configuration when preparing a quiescent Policy update; -do not replace it with this single-pool template. +Generation does not authenticate principals, protect the queue branch, or launch +workers. The generated policy must be represented by the dispatcher's +compiler-validated work-queue configuration for safe-output bootstrap. Use +verified positive decimal principal IDs from trusted producer and dispatch +credential flows, not display names or assumed `github.actor` values. Follow the +[deployment guide](/gh-aw/guides/deploy-work-queue/) to configure branch rules. +An administrator may still install Policy explicitly; later policy changes +require a drained queue. If the queue already serves other pools, retain their +configuration rather than replacing it with this single-pool template. Prepared payloads freeze the repository's verified numeric identity and worker-specific output contracts. The miner permits one draft PR; the refiner @@ -99,21 +112,24 @@ go build -o ./gh-aw ./cmd/gh-aw ./gh-aw work-queue --repo github/gh-aw explain --pool default --json ``` -`queue_missing: queue branch work-queue does not exist` means deployment has not -initialized the queue. It is not an initialized empty backlog. The agent's -`work_queue_read` snapshot reports this distinction as -`queue_state: "uninitialized"`. Report the missing Policy explicitly rather than -recording a successful empty-queue `noop`. Install Policy and producer entitlements -through the [deployment guide](/gh-aw/guides/deploy-work-queue/); do not invent -worker principal IDs or bypass queue-branch writer protections. +`queue_missing: queue branch work-queue does not exist` means the queue has not +yet been provisioned; the producer safe-output path treats it as empty and +creates the branch only when it atomically commits its first Policy and Work. +The agent's `work_queue_read` snapshot reports +`queue_state: "uninitialized"`. Continue with the trusted producer plan and +`work_queue_submit`; do not claim a successful empty-queue `noop`. Missing +compiler Policy, invalid producer entitlements, or a denied Git read still block +publication. Do not invent worker principal IDs or bypass queue-branch writer +protections. Hosted activation can fail at **Snapshot work queue state**, before the agent starts. Diagnose that boundary separately from the agent's queue tools. Installation-token repository metadata can report `permissions.pull: false` despite usable contents access; actual Git reads establish visibility, not collaborator flags. A denied Git read is not an empty queue. Even with readable -refs, dispatcher activation requires installed Policy and otherwise fails with -`work_queue_policy_missing`; prompt instructions cannot bootstrap it. +refs, first-submit bootstrap requires the compiler's validated Policy proposal. +The snapshot alone does not bootstrap the branch; trusted safe-output processing +does so only when it accepts a producer submission. Inspect an existing dispatcher run without launching another: @@ -178,8 +194,9 @@ safe-outputs: --- Do not select Work IDs, workers, revisions, or targets from a queue snapshot. -Read work_queue_read first and check queue_state. Report an uninitialized queue -with missing_data; it is not an empty backlog and must not produce noop. +Read work_queue_read first and check queue_state. An uninitialized queue is +empty; submit the trusted daily plan to provision it rather than reporting +missing_data or producing noop. Read the trusted daily plan and stage its exact nodes with work_queue_submit. Repeated admissions of the same UTC date are idempotent. Request the trusted scheduler's fair prefix with work_queue_dispatch_next: diff --git a/docs/src/content/docs/patterns/workqueue-ops.md b/docs/src/content/docs/patterns/workqueue-ops.md index a3176266af7..d6635ffb6dc 100644 --- a/docs/src/content/docs/patterns/workqueue-ops.md +++ b/docs/src/content/docs/patterns/workqueue-ops.md @@ -24,7 +24,7 @@ flowchart LR ## Native Work Queue -The native queue stores immutable tasks and lifecycle events in `work-queue.jsonl` on a dedicated Git branch. A trusted producer submits tasks, a dispatcher requests assignments, and workers process them. The scheduler selects eligible tasks under an administrator-installed Policy; the agent does not choose which task receives a Claim. +The native queue stores immutable tasks and lifecycle events in `work-queue.jsonl` on a dedicated Git branch. A trusted producer submits tasks, a dispatcher requests assignments, and workers process them. On an absent branch, the first producer submit atomically installs its compiler-approved Policy with Work; later Policy updates remain administrator-only. The scheduler selects eligible tasks under Policy; the agent does not choose which task receives a Claim. A Claim authorizes one attempt at a task. Workers stage safe outputs and a finish intent for each original Claim. Trusted processing records Completion before authorizing those effects and records Result only after independently verified delivery. Dependent tasks wait for their predecessors' Results, not merely for a worker run to finish. @@ -44,7 +44,7 @@ tools: worker: true ``` -These declarations are only part of deployment. Configure approved worker routes, producer permissions, capacity limits, and queue-branch writer restrictions using the [work-queue deployment guide](/gh-aw/guides/deploy-work-queue/). Frontmatter does not install Policy or protect the queue branch, and automated enforcement of writer restrictions remains deferred. +These declarations are only part of deployment. Configure approved worker routes, producer permissions, capacity limits, and queue-branch writer restrictions using the [work-queue deployment guide](/gh-aw/guides/deploy-work-queue/). Frontmatter alone does not install Policy or protect the queue branch; the first trusted producer submit bootstraps Policy, and automated enforcement of writer restrictions remains deferred. ```mermaid flowchart LR diff --git a/docs/src/content/docs/reference/work-queue.md b/docs/src/content/docs/reference/work-queue.md index 38dd55e4686..476fe375ca6 100644 --- a/docs/src/content/docs/reference/work-queue.md +++ b/docs/src/content/docs/reference/work-queue.md @@ -30,7 +30,7 @@ The native protocol distinguishes the following records and lifecycle events: | --- | --- | | Work | An immutable task definition. Tasks can form a directed acyclic graph (DAG), a dependency graph with no cycles. | | Claim | Authorization for one attempt at a Work item. Its original `claim_handle` identifies that attempt. | -| Policy | Administrator-installed rules for scheduling, producer permissions, worker routing, and limits. | +| Policy | Compiler-approved rules for scheduling, producer permissions, worker routing, and limits. The first trusted producer submission installs the proposal with Work when the branch is absent; later changes are administrator-only. | | Pool | A group of tasks with shared worker routes and capacity limits. | | Reservation | Native worker capacity held for an assignment, including while launch or termination is uncertain. | | Native launch | The request to start a GitHub Actions worker run. A committed assignment is not proof of launch. | @@ -306,7 +306,7 @@ initialize Policy over an invalid log. | `trace --request-id ID` or `trace --claim-id ID` | Read causal events within size limits, without exposing payloads or receipt contents. `--offset` and `--limit` control pagination. | | `compact` | Replace the current log prefix with a version-3 checkpoint of deterministic replay state. The checkpoint names the prior Git commit and preserves fairness accounting, ownership, delivery barriers, and request identities. | | `policy --file policy.json --epoch EPOCH` | Install an authorized Policy for future work only when the queue is quiescent. | -| `submit-work --file work.json` | Admit an immutable payload with default priority 3 and shared accounting key `""`, subject to installed producer permissions. | +| `submit-work --file work.json` | Admit an immutable payload with default priority 3 and shared accounting key `""`, subject to installed producer permissions. The CLI requires an existing Policy; a workflow producer's first safe-output submit can bootstrap an absent branch. | | `submit-graph` | Atomically admit a normalized graph within size limits, including issue and pull request dependency nodes. | | `dispatch-next --pool POOL --max-claims N --max-dispatches N` | Commit the next assignments selected by deterministic fair scheduling and their reservations. Does not send a workflow-dispatch POST. | | `control`, `cancel-work` | Apply authorized pause, cutover, or cancellation decisions without refunding accounted service or force-releasing a reservation for a possible native run. | @@ -324,8 +324,9 @@ A Policy update requires a quiescent queue: no nonterminal (unfinished) Work, outstanding reservations, or unresolved delivery barriers for completed Work. Installing Policy does not configure or verify queue-branch writer restrictions. Those restrictions must be established independently; automated enforcement -remains deferred. The agent cannot install Policy or receive queue-write -credentials. +remains deferred. A trusted workflow producer can publish only the exact +compiler-approved proposal with its first Work on an absent branch; the agent +cannot choose or update Policy or receive queue-write credentials. Worker finish is a Claim-scoped MCP intent, not an operator command that can impersonate a worker. Direct `claim --work-id`, legacy scalar assignments diff --git a/docs/src/content/docs/specs/work-queue-specification.md b/docs/src/content/docs/specs/work-queue-specification.md index a006f0988f9..0bc07d7975e 100644 --- a/docs/src/content/docs/specs/work-queue-specification.md +++ b/docs/src/content/docs/specs/work-queue-specification.md @@ -496,7 +496,18 @@ The words **MUST**, **SHOULD**, and **MAY** below describe the proposed feature, Every queue MUST have an installed scheduling policy, and every new Claim MUST be created through the trusted scheduler. Scheduling cannot be disabled through configuration, an omitted setting, an API argument, or an operator command. -When policy configuration is omitted for a new queue, trusted initialization MUST install the default `weighted-priority` policy as a transaction in the same canonical log before accepting Work or Claims. Explicit disable values and unrecognized modes MUST be rejected. An existing ledger without a valid current-protocol policy MUST fail; it MUST NOT be silently initialized over old Work or Claims. +An absent queue ref denotes an empty queue. On the first safe-output `submit` by +an authenticated producer or dispatcher, if the compiler supplies a validated +Policy proposal, publication MUST atomically create the protected queue branch +with one genesis commit containing that Policy followed by the submitted Work. +The proposal MUST authorize the submitting principal and the Work's pool, +priority, and accounting keys; the runtime MUST validate its immutable worker +routes before writing. This replaces administrator-seeded genesis for normal +workflow bootstrap. Reads and dispatch-only requests MUST NOT initialize an +absent queue. Existing nonempty ledgers without a valid Policy, empty or +malformed logs, and unsupported histories MUST fail closed rather than being +overwritten. Later Policy epochs remain administrator-only and require a drained +queue. One scheduling **pool** is one durable decision domain with a fixed worker-capability class, worker-routing policy, capacity model, and authoritative policy epoch. All dispatchers in that domain MUST use the same policy. Arbitrary agent-chosen filters MUST NOT redefine the competition set. @@ -724,6 +735,14 @@ Use **one `QueueCommit` JSON object per line of the existing `work-queue.jsonl`* | `operations` | Nonempty ordered array of typed operations, accepted atomically | | `trace` | Optional validated correlation metadata; never authority or a scheduling input | +For workflow bootstrap, the first producer/dispatcher `submit` may carry a +genesis `operations` array containing exactly one `Policy` operation first, +followed by the submitted Work nodes. Its request parameters still bind only +those Work nodes, and `policy_epoch` binds the Policy operation. Replay permits +this combined form only at ordinal zero; a producer cannot update Policy in an +existing ledger. The ordinary administrator-only Policy request remains valid +for explicit operator installation and later quiescent updates. + The request fingerprint binds the actor and the validated logical intent, not the observed branch SHA or tentative selected Work. Reusing a request ID with different meaning MUST fail. A conflict retry may regenerate the tentative operations and predecessor, but must reuse the same logical request. Once committed, its identity and result are immutable. Keep the logical request's origin fixed across job retries of the same serialized intent. A new publisher attempt belongs in diagnostics/trace metadata, not in a regenerated fingerprint that makes the same request look different. New agent runs/intents receive new handles; recovering an old handle preserves its trusted origin. @@ -912,10 +931,11 @@ Every queue MUST refuse unscheduled new Claims and old writers. There is no lega The deployment MUST restrict queue-branch updates to configured trusted publisher principals, forbid force updates/deletion, and keep publication -credentials out of agent jobs and untrusted workflow code. Use enforceable -repository branch/ruleset controls; if the host cannot restrict the required -writers, reject that deployment rather than claim enforced scheduling. An -authorized operator uses the same checked publisher, not a raw Git bypass. +credentials out of agent jobs and untrusted workflow code. Configure a matching +branch/ruleset pattern before the branch is created. Use enforceable repository +branch/ruleset controls; if the host cannot restrict the required writers, +reject that deployment rather than claim enforced scheduling. An authorized +operator uses the same checked publisher, not a raw Git bypass. `actor` is provenance asserted by that trusted boundary, not authentication. Derive it from authenticated workflow/job/operator context and validate the @@ -923,6 +943,9 @@ operation's role: producers submit allowed Work, dispatchers request fair grants, workers finalize assigned Claims, reconcilers establish trusted lifecycle evidence, and administrators change policy/controls. Commit author strings, trace IDs, and agent-supplied actor fields are not proof of identity. +Only the first compiler-approved producer/dispatcher submit may install Policy +as part of genesis; the request is authenticated, its Work is checked against +that Policy, and every later Policy change requires administrator authority. Workers' trusted bootstrap/finalization steps may hold publisher credentials; the worker agent and its snapshot MCP may not. @@ -2686,7 +2709,7 @@ unexecuted formal, supported-host, performance, or deployment-security gate. | Full JS typecheck/existing dependency-based tests | The earlier `create_project.cjs` SDK layout error is fixed in merge checkpoint `2dcce92c1c`, using validated literals and endpoint-derived request typing without unsafe casts. Genuine TypeScript 7.0.2 typecheck passes; 36 project tests include 12 layout/endpoint/invalid-input regressions. That merge's final component checks passed 2,122 setup-JavaScript tests in 52 files after correcting a stale workflow lock. The later `303b402810` merge ran 2,323 tests in 56 files: 2,322 passed and one real multi-repository fixture exceeded its 10-second deadline. A fixture-local 30-second deadline preserves all assertions and production retry behavior; all 109 repo-memory tests then passed. Impacted Go tests, build, typecheck, standard lint, schema freshness and JavaScript/shell lint passed. After committing the merge, change-scoped custom lint and full 328-workflow drift also passed; the initial custom-lint failures were confined to nine files byte-identical to main that the pre-merge base calculation included. Failed aggregate invocations remain failed records, not retrospectively green; global custom lint is not claimed clean. The approved feed does not supply pinned `@types/node` 26.6.4; existing 26.6.3 remains without changing pins. No dedicated hosted Actions run is claimed | | TypeSpec schema generation | Dependency-free emitter/drift checks and pinned official TypeSpec 1.16.0 compilation/emission pass. The fail-closed supported-subset comparison passes all 45 schemas, including validation constraints and custom identity bounds; it is not general schema or runtime equivalence proof | | Protected launch credentials, immutable effect targets and native delivery verification | Implemented with exact selected-client/profile proofs, immutable Work/profile/ancestor target intersection and private native readback. Both public and compiler control entry points reject missing/invalid protected launch metadata before client construction or queue publication, while previews and submit-only controls remain credential-independent. Local positive and refusal regressions pass; writer deployment automation is separate | -| Workflow administrator bootstrap and live native-dispatch host compatibility | Automatic workflow bootstrap is unsupported: initialize Policy through an explicitly authenticated operator/trusted host. Hosted immutable-SHA dispatch and the pinned run-details response remain unverified live compatibility gates; injected SDK checks are not hosted evidence | +| First-submit Policy bootstrap and live native-dispatch host compatibility | Safe-output first submit can atomically create the absent queue branch with its compiler-approved Policy and Work; local authenticated tests cover genesis replay and ref creation. Hosted branch/ruleset behavior, immutable-SHA dispatch, and the pinned run-details response remain unverified live compatibility gates | | Automated verification/provisioning of queue-branch writer restrictions | **Deferred by user direction; not implemented** | The writer-restriction deferral does not relax the normative trust requirement diff --git a/pkg/workflow/agy_engine_test.go b/pkg/workflow/agy_engine_test.go index 3d673846151..71ce07c5199 100644 --- a/pkg/workflow/agy_engine_test.go +++ b/pkg/workflow/agy_engine_test.go @@ -322,6 +322,10 @@ func TestAgyProductionConformancePermissionsAreBounded(t *testing.T) { case "safe_outputs": allowedPermissions = map[string]string{"actions": "write", "contents": "write"} } + if name == "activation" || name == "safe_outputs" { + assert.Equal(t, allowedPermissions, config.Permissions, "%s must have only its required queue permissions", name) + continue + } } if entry.id == "smoke-agy" && (name == "pre_activation" || name == "activation") { allowedPermissions = map[string]string{ @@ -351,6 +355,11 @@ func TestAgyConformanceEntryPointsShareConfiguration(t *testing.T) { readConformanceFrontmatter(t, "../../.github/workflows/engine-conformance-agy.md", &canonical) assert.Equal(t, map[string]any{"workflow_dispatch": nil}, canonical["on"]) assert.Equal(t, []any{"shared/agy-conformance.md", "shared/engine-conformance-worker.md"}, canonical["imports"]) + assert.Equal(t, map[string]any{ + "work-queue": map[string]any{"worker": true, "require-assignment": true}, + }, canonical["tools"]) + canonical["imports"] = []any{"shared/agy-conformance.md"} + delete(canonical, "tools") assert.EqualValues(t, 5, canonical["max-ai-credits"]) delete(canonical, "name") delete(canonical, "description") diff --git a/pkg/workflow/maintenance_workflow_jobs_parser_test.go b/pkg/workflow/maintenance_workflow_jobs_test_helpers_test.go similarity index 87% rename from pkg/workflow/maintenance_workflow_jobs_parser_test.go rename to pkg/workflow/maintenance_workflow_jobs_test_helpers_test.go index 78aa116974f..89c9fa0d59d 100644 --- a/pkg/workflow/maintenance_workflow_jobs_parser_test.go +++ b/pkg/workflow/maintenance_workflow_jobs_test_helpers_test.go @@ -7,6 +7,7 @@ import ( "github.com/stretchr/testify/require" ) +// parseMaintenanceJobs splits the generated maintenance workflow into top-level job bodies. func parseMaintenanceJobs(t *testing.T, content string) map[string]string { t.Helper() _, jobsSection, found := strings.Cut(content, "\njobs:\n") diff --git a/pkg/workqueue/policy.go b/pkg/workqueue/policy.go index 033b9572db7..b6ef44af519 100644 --- a/pkg/workqueue/policy.go +++ b/pkg/workqueue/policy.go @@ -223,7 +223,9 @@ func (state Projection) quiescent() bool { func permitted(role, kind string) bool { switch kind { - case "Policy", "Control", "Checkpoint": + case "Policy": + return role == "administrator" || role == "producer" || role == "dispatcher" + case "Control", "Checkpoint": return role == "administrator" case "Work": return role == "producer" || role == "dispatcher" || role == "worker" || role == "administrator" @@ -258,7 +260,7 @@ func validateRequest(commit QueueCommit) error { return err } kinds := map[string][]string{ - "policy": {"Policy"}, "control": {"Control", "WorkPriority"}, "submit": {"Work"}, + "policy": {"Policy"}, "control": {"Control", "WorkPriority"}, "submit": {"Policy", "Work"}, "dispatch_next": {"Observation", "Claim"}, "finish": {"Completion", "ClaimCancellation", "WorkCancellation"}, "observe": {"Observation"}, "dispatch": {"Dispatch"}, "release": {"ClaimCancellation", "WorkCancellation", "Release"}, "result": {"Result"}, "delivery_failure": {"DeliveryFailure"}, @@ -298,27 +300,17 @@ func validateRequestIntent(commit QueueCommit, parameterCount int) error { if err := json.Unmarshal(commit.Request.Parameters, ¶ms); err != nil { return queueError("checkpoint_invalid", "invalid checkpoint parameters") } - if err := json.Unmarshal(commit.Operations[0], &op); err != nil || - op.PriorGitSHA != params.PriorGitSHA || op.PriorTip != params.PriorTip || + for _, operation := range commit.Operations { + if err := json.Unmarshal(operation, &op); err != nil { + return queueError("checkpoint_invalid", "checkpoint history differs from stable request") + } + } + if op.PriorGitSHA != params.PriorGitSHA || op.PriorTip != params.PriorTip || op.HistorySHA256 != params.HistorySHA256 || op.StateSHA256 != params.StateSHA256 { return queueError("checkpoint_invalid", "checkpoint history differs from stable request") } case "submit": - var params SubmitParameters - if err := json.Unmarshal(commit.Request.Parameters, ¶ms); err != nil || params.Nodes == nil { - return queueError("request_invalid", "submit requires nodes") - } - if len(params.Nodes) != len(commit.Operations) { - return queueError("request_invalid", "submission differs from stable intent") - } - for i, node := range params.Nodes { - if i < 0 || i >= len(commit.Operations) { - return queueError("request_invalid", "submission differs from stable intent") - } - if !sameJSON(node, commit.Operations[i]) { - return queueError("request_invalid", "submission changes immutable node") - } - } + return validateSubmitIntent(commit) case "dispatch_next": var params DispatchParameters if err := json.Unmarshal(commit.Request.Parameters, ¶ms); err != nil { @@ -359,6 +351,38 @@ func validateRequestIntent(commit QueueCommit, parameterCount int) error { return nil } +func validateSubmitIntent(commit QueueCommit) error { + var params SubmitParameters + if err := json.Unmarshal(commit.Request.Parameters, ¶ms); err != nil || params.Nodes == nil { + return queueError("request_invalid", "submit requires nodes") + } + offset := 0 + if len(commit.Operations) == len(params.Nodes)+1 { + if len(commit.Operations) > 0 { + kind, err := operationKind(commit.Operations[0]) + if err != nil || kind != "Policy" || commit.Actor.Role != "producer" && commit.Actor.Role != "dispatcher" { + return queueError("request_invalid", "first submission Policy bootstrap requires a trusted producer") + } + } else { + return queueError("request_invalid", "first submission Policy bootstrap requires a trusted producer") + } + offset = 1 + } + if len(params.Nodes) != len(commit.Operations)-offset { + return queueError("request_invalid", "submission differs from stable intent") + } + for i, node := range params.Nodes { + operationIndex := i + offset + if operationIndex < 0 || operationIndex >= len(commit.Operations) { + return queueError("request_invalid", "submission differs from stable intent") + } + if !sameJSON(node, commit.Operations[operationIndex]) { + return queueError("request_invalid", "submission changes immutable node") + } + } + return nil +} + func (state Projection) allowedProducer(actor Actor, node WorkDefinition) bool { rule, ok := state.Policy.Producers[actor.Principal] return ok && slices.Contains(rule.Pools, node.Pool) && diff --git a/pkg/workqueue/replay.go b/pkg/workqueue/replay.go index 29d7047dd52..09a32ece583 100644 --- a/pkg/workqueue/replay.go +++ b/pkg/workqueue/replay.go @@ -112,9 +112,17 @@ func (state *Projection) validateReplayCommit(commit QueueCommit, ordinal int) ( } isPolicy = kind == "Policy" } + bootstrap := isBootstrapSubmit(commit, ordinal) + if bootstrap { + isPolicy = true + } if ordinal == 0 && !isPolicy { return false, queueError("policy_missing", "genesis must install exactly one policy") } + if isPolicy && !bootstrap && + (len(commit.Operations) != 1 || commit.Request.Kind != "policy" || commit.Actor.Role != "administrator") { + return false, queueError("policy_not_quiescent", "only an administrator may install Policy outside first-submit bootstrap") + } if !isPolicy && commit.PolicyEpoch != state.PolicyEpoch { return false, queueError("policy_epoch_invalid", "commit names a non-current epoch") } @@ -124,6 +132,52 @@ func (state *Projection) validateReplayCommit(commit QueueCommit, ordinal int) ( return isPolicy, nil } +func isBootstrapSubmit(commit QueueCommit, ordinal int) bool { + if ordinal != 0 || commit.Request.Kind != "submit" || + (commit.Actor.Role != "producer" && commit.Actor.Role != "dispatcher") || + len(commit.Operations) < 2 { + return false + } + if !isPolicyOperation(commit.Operations) { + return false + } + return operationsAreWork(commit.Operations[1:]) && submissionMatchesOperations(commit.Request, commit.Operations[1:]) +} + +func isPolicyOperation(operations []Operation) bool { + if len(operations) > 0 { + kind, err := operationKind(operations[0]) + return err == nil && kind == "Policy" + } + return false +} + +func operationsAreWork(operations []Operation) bool { + for _, operation := range operations { + kind, err := operationKind(operation) + if err != nil || kind != "Work" { + return false + } + } + return len(operations) > 0 +} + +func submissionMatchesOperations(request Request, operations []Operation) bool { + var parameters SubmitParameters + if request.Kind != "submit" || json.Unmarshal(request.Parameters, ¶meters) != nil || len(parameters.Nodes) != len(operations) { + return false + } + for index, node := range parameters.Nodes { + if index < 0 || index >= len(operations) { + return false + } + if !sameJSON(node, operations[index]) { + return false + } + } + return true +} + func (state *Projection) replayCommit(commit QueueCommit, ordinal int, epochs, generations identitySet) error { isPolicy, err := state.validateReplayCommit(commit, ordinal) if err != nil { diff --git a/pkg/workqueue/replay_test.go b/pkg/workqueue/replay_test.go index 13c8dc4f7f5..36302cf7a7a 100644 --- a/pkg/workqueue/replay_test.go +++ b/pkg/workqueue/replay_test.go @@ -83,6 +83,54 @@ func testOperations(t *testing.T, commits []QueueCommit, actor Actor, id, kind s return next } +func TestEmptyLedgerReplayRequiresPolicyGenesis(t *testing.T) { + if _, err := Replay(nil); err == nil || !strings.HasPrefix(err.Error(), "policy_missing:") { + t.Fatalf("empty ledger must report missing Policy genesis, got %v", err) + } +} + +func TestProducerSubmitBootstrapsAbsentQueuePolicy(t *testing.T) { + policy := DefaultPolicy(testPrincipal, testRepository) + node, err := NewWork([]byte(`{"task":"first producer submission"}`), "bootstrap", "root", "default", policy, 1000) + if err != nil { + t.Fatal(err) + } + policyOperation, err := Op(map[string]any{"kind": "Policy", "epoch": "bootstrap-epoch", "policy": policy}) + if err != nil { + t.Fatal(err) + } + workOperation, err := Op(node) + if err != nil { + t.Fatal(err) + } + actor := testActor("producer") + request, err := NewRequest("bootstrap-submit", "submit", actor, SubmitParameters{Nodes: []WorkDefinition{node}}) + if err != nil { + t.Fatal(err) + } + commit := QueueCommit{ + Version: Version, ID: "bootstrap-commit", Request: request, Actor: actor, + PolicyEpoch: "bootstrap-epoch", At: 1000, + Operations: []Operation{policyOperation, workOperation}, + } + state, err := Replay([]QueueCommit{commit}) + if err != nil { + t.Fatal(err) + } + if state.Policy == nil || state.PolicyEpoch != "bootstrap-epoch" || state.Works[node.WorkID] == nil { + t.Fatalf("first submit did not install Policy and Work: %#v", state) + } + + commit.Actor = testActor("administrator") + commit.Request, err = NewRequest("administrator-bootstrap-submit", "submit", commit.Actor, SubmitParameters{Nodes: []WorkDefinition{node}}) + if err != nil { + t.Fatal(err) + } + if _, err := Replay([]QueueCommit{commit}); err == nil { + t.Fatal("administrator submit must not bypass the standalone Policy genesis contract") + } +} + func TestCausalReplayAndCanonicalDedup(t *testing.T) { commits := testGenesis(t, nil) a, b := testNode(t, commits, "a"), testNode(t, commits, "b") diff --git a/specs/eslint-factory/Factory.tla b/specs/eslint-factory/Factory.tla index 103a9e0f6c3..c71f1ba40ea 100644 --- a/specs/eslint-factory/Factory.tla +++ b/specs/eslint-factory/Factory.tla @@ -11,25 +11,26 @@ Profile(k) == IF Worker = "mixed" THEN IF k = 1 THEN "miner" ELSE "monster" ELSE Worker -VARIABLES phase, admitted, original, birth, requested, proposed, scoped, applied, - completedAtWrite, verified, scan, lintClean, quality -vars == <> +VARIABLES phase, admitted, queueBranch, original, birth, requested, proposed, + scoped, applied, completedAtWrite, verified, scan, lintClean, quality +vars == <> Init == /\ phase = <<"absent", "absent">> - /\ admitted = {} + /\ admitted = {} /\ queueBranch = FALSE /\ original = {} /\ birth = {} /\ requested = FALSE /\ proposed = <<-1, -1>> /\ scoped = <> /\ applied = <<-1, -1>> /\ completedAtWrite = <> /\ verified = <> /\ scan = "unrun" /\ lintClean = FALSE /\ quality = "unchecked" -\* Admission is an external authorized producer action, not issue creation. +\* A missing ref is an empty queue. The producer's first checked safe-output +\* submission atomically installs the compiled Policy and Work while creating it. Admit(k) == /\ (Installed \/ Fault = "policy") /\ phase[k] = "absent" /\ phase' = [phase EXCEPT ![k] = "queued"] - /\ admitted' = admitted \cup {k} + /\ admitted' = admitted \cup {k} /\ queueBranch' = TRUE /\ UNCHANGED <> @@ -44,10 +45,10 @@ AssignmentProjection == IF Eligible = {} THEN {} \* One protected prefix request; project only its first compatible assignment. \* Other profile groups/native dispatches are outside this worker activation. Dispatch == - /\ ~requested /\ (Installed \/ Fault = "policy") + /\ ~requested /\ queueBranch /\ (Installed \/ Fault = "policy") /\ original' = AssignmentProjection /\ birth' = AssignmentProjection /\ requested' = TRUE /\ phase' = [k \in Works |-> IF k \in AssignmentProjection THEN "claimed" ELSE phase[k]] - /\ UNCHANGED <> Precheck(s) == @@ -55,14 +56,14 @@ Precheck(s) == /\ s \in {"clean", "warnings", "errors", "installFailed", "buildFailed", "toolFailed"} /\ scan' = s /\ lintClean' = (s = "clean" \/ (s = "warnings" /\ Fault # "warning")) - /\ UNCHANGED <> \* Build/test/low-false-positive bars are agent obligations, not authority. Quality(q) == /\ original # {} /\ quality = "unchecked" /\ q \in {"passed", "failed"} /\ quality' = q - /\ UNCHANGED <> \* An explicit original handle is modeled as TRUE. An implicit selector is @@ -75,19 +76,19 @@ Stage(k, n, explicit) == /\ explicit \/ Cardinality(original) = 1 \/ Fault = "scope" /\ proposed' = [proposed EXCEPT ![k] = n] /\ scoped' = [scoped EXCEPT ![k] = explicit \/ Cardinality(birth) = 1] - /\ UNCHANGED <> Complete(k) == /\ k \in original /\ phase[k] = "claimed" /\ phase' = [phase EXCEPT ![k] = "completed"] - /\ UNCHANGED <> Cancel(k) == /\ k \in original /\ phase[k] \in {"claimed", "completed"} /\ phase' = [phase EXCEPT ![k] = "cancelled"] - /\ UNCHANGED <> \* Resource/principal/run binding is abstracted as trusted successful authority. @@ -99,7 +100,7 @@ Effect(k) == /\ completedAtWrite' = [completedAtWrite EXCEPT ![k] = phase[k] = "completed"] /\ phase' = [j \in Works |-> IF Fault = "dag" /\ phase[j] = "absent" THEN "queued" ELSE phase[j]] - /\ UNCHANGED <> \* "verified" stands for whole-contract independent readback, including memory @@ -109,21 +110,21 @@ Readback(k) == /\ applied[k] = proposed[k] /\ proposed[k] >= 0 /\ proposed[k] >= OutputMin \/ NoWrites \/ Fault = "cardinality" /\ verified' = [verified EXCEPT ![k] = TRUE] - /\ UNCHANGED <> Result(k) == /\ k \in original /\ phase[k] = "completed" /\ verified[k] \/ Fault = "result" /\ phase' = [phase EXCEPT ![k] = "result"] - /\ UNCHANGED <> Shrink == /\ Fault = "membership" /\ Cardinality(original) = 2 /\ \E k \in original : phase[k] = "cancelled" /\ original' = {k \in original : phase[k] # "cancelled"} - /\ UNCHANGED <> Next == @@ -136,7 +137,7 @@ Spec == Init /\ [][Next]_vars TypeOK == /\ phase \in [Works -> {"absent", "queued", "claimed", "completed", "cancelled", "result"}] - /\ admitted \subseteq Works + /\ admitted \subseteq Works /\ queueBranch \in BOOLEAN /\ original \subseteq Works /\ birth \subseteq Works /\ requested \in BOOLEAN /\ proposed \in [Works -> (-1..OutputMax)] /\ applied \in [Works -> (-1..OutputMax)] /\ scoped \in [Works -> BOOLEAN] /\ completedAtWrite \in [Works -> BOOLEAN] @@ -146,6 +147,7 @@ TypeOK == ImmutableMembership == original = birth HomogeneousBatch == \A a, b \in birth : Profile(a) = Profile(b) PolicyRequired == original # {} => Installed +BranchProvisioning == (admitted # {}) <=> queueBranch ScopedIntents == \A k \in Works : proposed[k] >= 0 => scoped[k] EffectRequiresCompletion == \A k \in Works : applied[k] > 0 => completedAtWrite[k] ResultRequiresReadback == \A k \in Works : phase[k] = "result" => verified[k] diff --git a/specs/eslint-factory/README.md b/specs/eslint-factory/README.md index 7adabe1aa7c..42cd52fd59a 100644 --- a/specs/eslint-factory/README.md +++ b/specs/eslint-factory/README.md @@ -13,7 +13,7 @@ This directory models the factory's queue-worker lifecycle and compares its assu | `trace.mjs` | Strict finite-value trace parser, independent abstraction replay and readable trace rendering | | `compare.test.mjs` | Comparison sensitivity, mutated executable fixture and generated trace validation tests | -The universe contains two externally admitted root Works. Each has a provisioned worker profile; equal priority, dependency readiness, capacity, run binding and resource authorization are simplified assumptions. `Dispatch` projects the first compatible worker assignment from one protected prefix request, not the complete native dispatch set. Homogeneous batching is permitted only when `Batch` is true; otherwise the assignment is a singleton. Different profiles cannot share an assignment. Actual mixed-profile probes produce two separate dispatches; leaving the other modeled Work queued is projection, not a claim about scheduler selection. This two-Work projection does not exercise all three dispatch slots, profile trust-domain/share-key restrictions, assignment byte bounds or queue fairness/recovery algorithms. +The universe contains two externally admitted root Works. `queueBranch` starts absent; the first authorized producer submit atomically creates it with the compiler-approved Policy and Work. `Installed` models availability of that compiled Policy proposal, not an administrator-seeded ledger. Dispatch requires the created branch and Policy. Each Work has a provisioned worker profile; equal priority, dependency readiness, capacity, run binding and resource authorization are simplified assumptions. `Dispatch` projects the first compatible worker assignment from one protected prefix request, not the complete native dispatch set. Homogeneous batching is permitted only when `Batch` is true; otherwise the assignment is a singleton. Different profiles cannot share an assignment. Actual mixed-profile probes produce two separate dispatches; leaving the other modeled Work queued is projection, not a claim about scheduler selection. This two-Work projection does not exercise all three dispatch slots, profile trust-domain/share-key restrictions, assignment byte bounds or queue fairness/recovery algorithms. `birth` records original membership and `original` must remain equal to it. Implicit Claim selection is allowed only for an original singleton, regardless of how many members later finish or cancel. `Stage` projects one output family per Claim: miner PR, refiner issue or monster remediation assignment. Its count is **not** the aggregate of every output type. Modeled `OutputMin`/`OutputMax` are scenario assumptions, not the complete worker-specific contracts prepared by `eslint_factory_portfolio.cjs`. A separate no-write scenario models explicit no-write Work. @@ -31,7 +31,7 @@ daily producer's cancellation or live reservation release. ## Checked properties -The safety configurations check types, immutable original membership, homogeneous batches, installed policy, unambiguous original Claim selection, Completion before writes, independent readback before Result, per-Claim result cardinality, warning-only exit-zero behavior, explicit admission and assignment size. With only two Works, `BoundedAssignment <= 3` is a finite-universe bound, not validation of the full three-slot dispatcher budget. `Spec` permits stuttering; deadlock checking is disabled for terminal or stalled workers. No fairness assumption or temporal progress property is checked. +The safety configurations check types, immutable original membership, homogeneous batches, compiled Policy availability, branch creation with first admission, unambiguous original Claim selection, Completion before writes, independent readback before Result, per-Claim result cardinality, warning-only exit-zero behavior, explicit admission and assignment size. With only two Works, `BoundedAssignment <= 3` is a finite-universe bound, not validation of the full three-slot dispatcher budget. `Spec` permits stuttering; deadlock checking is disabled for terminal or stalled workers. No fairness assumption or temporal progress property is checked. Each negative configuration changes exactly one abstraction rule and requires the **named** invariant violation, not merely a nonzero TLC exit: @@ -40,7 +40,7 @@ Each negative configuration changes exactly one abstraction rule and requires th | Permit an implicit selector for an original multi-Claim assignment | `ScopedIntents` | | Shrink original membership after cancellation | `ImmutableMembership` | | Mix worker profiles in a batch | `HomogeneousBatch` | -| Admit/dispatch without installed policy | `PolicyRequired` | +| Admit/dispatch without compiled Policy proposal | `PolicyRequired` | | Write before Completion | `EffectRequiresCompletion` | | Settle without independent readback | `ResultRequiresReadback` | | Verify an under-minimum output count | `ResultCardinality` | @@ -59,7 +59,7 @@ Build and scan fixtures reuse existing source/dependencies via symlinks as build | --- | --- | | Assignment protocol is version 3, original payload is deeply frozen | Native `normalizeAssignment` probes; this normalization alone is not authentication | | Singleton selection differs from original multi-Claim selection | Native scope, foreign-selector, authority-override and nested AsyncLocalStorage escape probes | -| Policy/profile installation and producer entitlement are required | Native replay/policy rejection probes; default profile has `max_claims: 1`, permitted batch has two members, mixed profiles produce separate dispatches | +| First-submit Policy bootstrap, profile installation and producer entitlement are required | Native replay/store tests; default profile has `max_claims: 1`, permitted batch has two members, mixed profiles produce separate dispatches | | Winning Claim/native attempt/principal gate authority | `claimAuthority` probes reject unfinished, cancelled, wrong-principal and later-attempt Claims; positive completed state is a detached fixture mutation, not fabricated durable evidence | | Count contracts are per Claim | Actual collector entrypoint accepts three issues for each of two Claims, rejects one Claim's overflow without charging its sibling, and checks a missing sibling's minimum independently. Native diagnostic delivery checks also reject invalid counts; six accepted intents do not establish six native writes | | Completion/success cannot replace readback | Missing exact receipts/verifier remain unknown; the protected two-argument delivery facade rejects untrusted nominal success | diff --git a/specs/eslint-factory/cases.mjs b/specs/eslint-factory/cases.mjs index 41238a4ed4e..27b59f1af10 100644 --- a/specs/eslint-factory/cases.mjs +++ b/specs/eslint-factory/cases.mjs @@ -3,6 +3,7 @@ export const invariants = [ "ImmutableMembership", "HomogeneousBatch", "PolicyRequired", + "BranchProvisioning", "ScopedIntents", "EffectRequiresCompletion", "ResultRequiresReadback", diff --git a/specs/work-queue/README.md b/specs/work-queue/README.md index 770e1b04860..c53478ba207 100644 --- a/specs/work-queue/README.md +++ b/specs/work-queue/README.md @@ -24,7 +24,7 @@ The design rationale and trade-offs are recorded in [ADR-64955](../../docs/adr/6 The current operator commands and keyboard browser are documented in the [queue reference](../../docs/src/content/docs/reference/work-queue.md#operator-commands). -Policy installation is documented in the +First-submit Policy bootstrap and explicit operator Policy installation are documented in the [deployment guide](../../docs/src/content/docs/guides/deploy-work-queue.md). This directory retains executable specifications and formal evidence, not a second current user guide. The historical material below is model provenance,