[eipasteur]

In short

Opt-in path that serves the built-in aidlc MCP through AgentCore Gateway instead of inside the agent's microVM (enable_mcp_gateway, default false, requires network_mode = "VPC").

• A stdio proxy in the runtime registers the same tool catalog and forwards each call, SigV4-signed, to an AWS_IAM Gateway. The CLIs keep the same aidlc server and tool names.
• The Gateway's Lambda target (lambda/mcp-tools) runs the existing graph and process handlers.
• v2-orchestrator and discussions sign an ES256 grant with an asymmetric KMS key. The key policy allows kms:Sign only to those roles; verifiers hold only kms:GetPublicKey.
• The grant names the space, intent, execution, stage lane and leg, mode and release-policy digest. It travels in a propagated header, never in tool arguments, and is not exposed to command handlers or the agent CLI.
• Authorization runs in a Gateway interceptor (lambda/mcp-interceptor, REQUEST and RESPONSE, pass_request_headers), in front of the target. On every request it lets through (initialize, ping, tools/list, tools/call, notifications) it verifies the grant, binds the session to it (policy digest and stage leg included), rejects duplicate or multi-valued grant/session headers, checks the target and the role's tool allowlist, then re-reads the execution, stage leg and intent ownership as active. A finished, cancelled or superseded leg is denied; a grant is refused once its run is retired on the execution, even while a stage row still names that run. Unknown methods, targets and tools are denied. A denial is answered by the Gateway with a JSON-RPC -32001 error carrying the reason code; the target is never invoked. Any failure in the interceptor denies.
• An allowed request is forwarded with its original body and only the context the interceptor derived: x-aidlc-context (versioned, exact fields, with granted, live-verified and runtime-asserted sections) and x-aidlc-context-policy (the resolved release policy, bound to the signed digest). Each header is capped at 4 KB. The interceptor never copies a caller header, Authorization and AWS tokens included; the grant and session headers stop at the interceptor.
• The RESPONSE interceptor filters tools/list with the same predicate tools/call enforces, re-derived from the request headers. Other responses pass through unchanged.
• mcp-tools no longer verifies the grant, the session or the live scope. It requires the two context headers from bedrockAgentCorePropagatedHeaders, decodes them strictly (version, fields, types, size, digest, expiry) and denies on anything else. It still parses arguments with each tool's Zod shape, pins a lane session to its unit, re-checks the role's tool catalog from the context, and leaves every process-table write behind the store's transactional ownership checks. The KMS dependency goes away from the tools Lambda.
• Invocation isolation. The Gateway role is the only principal with lambda:InvokeFunction on mcp-tools and the interceptor; the runtime role keeps only the broker and source-control functions. terraform/tests/mcp-gateway.tftest.hcl asserts the interceptor wiring, the target's header allowlist, the invoke grants, that the runtime role cannot invoke either function under any qualifier, and that nothing exists with the Gateway off. The interceptor runs in the AgentCore subnets with an egress-only security group and a read-only role (kms:GetPublicKey on the grant key, Neptune read, dynamodb:GetItem on the process table).
• Every allow, deny and failure is logged with its scope by the interceptor, and every tool outcome by mcp-tools. The runtime proxy turns a Gateway -32001 denial back into the local server's denial text, so the agent still reads "Stop and end your turn" for an inactive stage.
• With the Gateway enabled and no grant present, the runtime refuses to start the MCP instead of falling back to the local server.

MCP core modules move to lambda/shared/mcp/; the authorizer (gateway-authorization.js) and the context codec (gateway-context.js) live there as pure orchestration over injected clients. Re-export shims in lambda/agentcore/mcp/ keep existing imports working.

Why inbound stays AWS_IAM

The native CUSTOM_JWT authorizer was evaluated and not adopted. It would replace only the Gateway's inbound check (signature, issuer, audience, expiry); the interceptor and its live checks would stay, because the authorizer validates a token, not the execution's state. Grant retirement, the direct calls the runtime still makes to engine-data, the credential broker and the source-control alias, and the write-time ownership guards are all ours either way. A Gateway has one authorizer type, so switching would also remove SigV4 for the runtime. And it would need an OIDC discovery document and JWKS for a KMS-held key, plus ES256 acceptance by the inbound authorizer, neither of which is documented or verified.

Trust boundary and limits

The Gateway accepts calls signed with the runtime role, and the interceptor admits only those that also carry a valid grant. These are two checks, not two independent barriers: code in the microVM can read both. What the grant adds is that the runtime cannot widen its own scope: it cannot mint a grant for another space, execution or stage, and a superseded leg or run is refused on the next call. mcp-tools trusts the injected context because of the two properties Terraform enforces and tests: only the Gateway may invoke it, and the target forwards no request header except the two the interceptor replaces on every allowed request.

Explicit limits:

• All agents of a stage share one grant. The role inside the stage (author, reviewer, reader), the persona, attempt metadata and the checkpoint and question flags are asserted by the runtime in the unsigned session header. They are marked as asserted in the context and never reach beyond the granted stage, lane and leg; a forged session can raise the sub-role within its own stage, never reach another stage, execution or space.
• Nothing binds a grant to the caller that presents it.
• The live checks are preflight. A leg superseded after a request is admitted is refused by the process table's ownership checks on its writes; Neptune graph writes made by an already-admitted call have no such fence (the same exposure as the previous authorize-then-handler shape).
• Discussion grants rely on a 300 s TTL. Durable callbacks stay on the runtime role.
• Older managed-environment images ignore the new env vars and keep the local MCP path.

Deliberately not included: an agent or persona identity in the grant. The persona is known only to the runtime, so an unsigned value would be forgeable by a compromised agent. Per-persona grants make sense together with CLI sandboxing and are a follow-up.

Scope

First of four PRs: this one, then engine-data (#614), then the credential broker, source-control alias and customer MCP targets (#615), then the removal of the runtime role's direct data permissions (#616). This PR changes where the built-in tools run and how each call is authorized. It does not reduce the runtime role's permissions on its own.

Validation

• Rebased on main. On the stack tip: npm test 7349 passed, 2 todo; test:release 172 passed; terraform test 49/49 (mcp-gateway.tftest.hcl included); lint, format:check, dep:check, secretlint and sdk:check pass. The interceptor and authorizer have their own suites (grant, session, context negatives, duplicate headers, tools/list filtering, deny shapes); mcp-tools keeps the handler, bridge and audit regressions on the context path and runs a question write against DynamoDB Local to show a rewind landing after authorization is still rejected by the store.
• Not yet verified on a deployed Gateway: that the grant and session headers reach the interceptor and that the injected context headers reach mcp-tools through bedrockAgentCorePropagatedHeaders; that the MCP SDK client surfaces the -32001 denial body; that the interceptor is invoked for initialize and tools/list; that Mcp-Session-Id survives the RESPONSE passthrough; that the Gateway is updated in place rather than replaced; and the runtime role's inability to invoke either Lambda.

Upgrade notes

With the flag off, every new resource is count-gated and the grant key env var is not set. Enabling it creates the Gateway, interceptor, target, tools Lambda, KMS key, security groups and schema object, and switches the runtime to the proxy. The new roles follow iam_role_path and iam_permissions_boundary_arn. The interceptor is configured whenever the Gateway exists; it is not a separate option.

Executions started before the flag have no grant issuer and would fail the next time they invoke the runtime. deploy-terraform.sh therefore refuses a plan that switches the runtime to the Gateway while any execution is started, running or parked, and lists them. Stop starting intents until the deployment has succeeded, let running executions park or finish, cancel the parked and queued ones, apply, then rewind each to its current stage (its first stage if it had not reached one): the rewind relaunches it with grants. AIDLC_ALLOW_ACTIVE_EXECUTIONS=1 applies anyway. A direct terraform apply skips the check.

[JWThewes]

[P1] Preserve existing executions before enabling the mandatory-grant runtime path

@eipasteur An execution started with Gateway disabled can park at a human gate, lose its warm runtime session, and then resume after enable_mcp_gateway is enabled. Its published durable orchestrator still has the old configuration and no grant issuer, while the standard runtime can restart on the new configuration: standard revisions have runtimeEndpoint: null, and resolveRuntimeTarget() does not pin the invocation using runtimeVersion. The resumed request therefore reaches this check without a grant and fails with GATEWAY_GRANT_MISSING.

This also affects executions started after deploying the new code with flags off. In #614's engine-data mode, the request fails even earlier: resolveInvocationAgentAuth() reads META through the remote store before calling the broker. My local reproduction using the real dispatcher/auth resolver/store returned HTTP 500 with No engine data grant for this invocation; broker and handler calls were both zero. Leaving enforce_runtime_grants=false does not protect that path.

Please either keep existing executions routed to a compatible pinned runtime revision/configuration, or require and enforce draining them before the first remote-path flag is enabled. The rollout guide in #616 currently drains them only after enabling Gateway and engine-data. Keep the missing-grant rejection itself; an unauthenticated fallback would weaken the boundary.

Regression/acceptance checks:

• Start an execution with flags off, park it, stop its runtime session, enable Gateway, and resume it.
• Repeat for engine-data transition mode with strict enforcement off.
• Verify compatible resumption, or prevent activation while such executions remain; include parked executions in the drain condition.
• Keep missing/forged grants rejected on the remote path.

The failure was reproduced locally at stack head 22ec812c; the live AWS migration scenario was not exercised.

[eipasteur]

Agreed, thanks for the repro. I kept the missing-grant rejection and went with draining before activation, enforced rather than documented:

• scripts/check-active-executions.mjs runs in deploy-terraform.sh on every saved plan (and again at apply). It detects the runtime gaining the Gateway (this PR) or engine-data (feat(engine-data): route the runtime's engine data through grant-verified engine-data (opt-in) #614) configuration, so a retry after an apply that failed before the runtime was updated is still caught, and refuses the plan while any execution is CREATED, RUNNING or WAITING, listing them. Fresh installs and plans that switch nothing skip the table read.
• Operator path: stop starting intents until the deployment has succeeded, let running executions park or finish (/cancel does not accept RUNNING), cancel the parked and queued ones, apply, then rewind each to its current stage (its first stage if it had not reached one). The rewind starts a new orchestrator run on the new configuration, so it relaunches with grants; artifacts and workspace are kept.
• AIDLC_ALLOW_ACTIVE_EXECUTIONS=1 overrides, and still prints the executions that will fail.
• The rollout guide in feat(terraform): remove the runtime role's direct data access under enforced runtime grants #616 now drains before each of the two flags, not after.

Tests cover activation per flag, retry after a partial apply, queued/running/parked executions vs drafts, fresh install, read failures and the override; the script was also run against a real process table. Not exercised: the live migration scenario on a deployed stack. A direct terraform apply skips the check (documented), and new starts during the apply are prevented by the procedure, not enforced. Pinning the runtime revision per execution would allow migrating without the drain, but it is a much larger change; happy to take it as a follow-up if we need hot migration.

[jeromevdl]

P2 — Reject grants after the execution’s run ID changes.

startup-context.js:149–151 checks the execution’s status, but compares orchestratorRunId only against the stage row. Recompose retires the execution’s run ID before resetting stage rows. During that interval, the execution can remain WAITING and the stage still carries the old run/callback IDs. The Gateway therefore accepts the retired grant. I reproduced this through the real grant signer, verifier, and Gateway handler: a signed old-run grant reached a stubbed createArtifact writer and was audited as allow/ok. DynamoDB write fencing does not protect that graph mutation, so stale work can modify artifacts after recompose has archived them.

Fix: compare the signed run ID with execution META during authorization, alongside the existing stage check. Add a regression test where META is retired but the stage still names the old run. This belongs in #613’s existing authorization logic; it does not require importing #615’s retirement machinery.

[eipasteur]

Fixed in 3463512: authorization now also compares the signed run with META.orchestratorRunId. The regression test has META retired while the stage row still names the old run; it fails without the fix. The same check is in engine-data's stage grants (#614, 84e5efe), since graph writes have no store-side fence there either.

[jeromevdl]

Apart from the one P2, the stack boundaries are good with #614, #615 and #616., architecture is reasonable (SOLID/KISS).
Also need a rebase.

[eipasteur]

Rebased on main, including #603: the new roles (mcp-tools, mcp-gateway, engine-data) follow iam_role_path and the permissions boundary, and the signer role ARNs in the grant key policy include the path.

[github-actions]

Complexity and test coverage

Automated quality report using code complexity and test coverage to highlight potential hot spots in this PR.
Review 🟣 and 🔴 items first: consider simplifying the code, improving test coverage, or both. This report is advisory.

	File / function	Complexity	Δ	Branch coverage	Δ	Uncovered branches	Depending files
🟠	lambda/shared/gateway-grant.js:202 — verifyGatewayGrant	— → 17	—	— → 81.1%	—	— → 10	7
🟠	lambda/v2-orchestrator/index.js:2712 — runStage / <anonymous>	— → 18	—	— → 86.7%	—	— → 6	0
🟠	lambda/mcp-interceptor/interceptor.js:125 — createGatewayInterceptor / <anonymous>	— → 17	—	— → 86.7%	—	— → 4	1
🟠	lambda/shared/mcp/startup-context.js:133 — validateStartupContext	— → 28	—	— → 97.6%	—	— → 1	3

🟢 Improving · 🟠 Needs attention · 🔴 High risk · 🟣 Highest risk

— = unavailable · n/a = not applicable · * = incomplete tests

CI run and full report artifacts

[jeromevdl]

@eipasteur Previous findings are solved, thank you.

Two focused questions on the Gateway integration after our meeting this morning:

• JWT validation: Is there a reason we use AWS_IAM inbound plus a custom JWT verified in the target Lambda, rather than Gateway’s native CUSTOM_JWT authorizer? I understand execution/stage checks would remain application-owned, and direct Lambda calls still need verification. Would native validation simplify the MCP path, or introduce more infrastructure than it removes?
• Tool permissions: Did you consider AgentCore Policy for the tool allowlist? Would it replace meaningful application code, or mostly duplicate the existing shared rules? Since the runtime currently asserts the role, would trusted role claims be a prerequisite?

Please also rebase the stack.

[eipasteur]

Tool permissions. Not in this PR. With AWS_IAM inbound, Policy sees the runtime role's ARN, which is the same for every agent, and the role inside a stage (author, reviewer, reader) is asserted by the runtime in the session header. A Policy allowlist would duplicate the shared rules without replacing any code, so yes, trusted role claims are the prerequisite. It becomes worthwhile once the role is a signed claim and the Gateway fronts more than the built-in target, which is the custom MCP follow-up from Wednesday. I'll answer the JWT question separately.

Rebase. The stack is rebased on main, including #534 and #623. Two tests needed adapting to the new contracts; the production code did not change:

• the fix(derive): re-derive any artifact the finishing stage changed #623 derive test now uses this PR's head-lookup fake;
• the durable grant-retirement replay test now goes through the completion-read fixture added by Prevent incomplete intents from reaching success and publication #534.

I also documented the Gateway trust boundary in architecture.md, including what the grant does and does not protect against.

[eipasteur]

Thanks. Both are now addressed in the PR.

JWT validation. Inbound stays AWS_IAM, and the verification moved out of the target Lambda into a Gateway REQUEST interceptor that runs in front of every target. It checks the grant, the session, retirement and the live execution and stage, then passes a verified context to the target. CUSTOM_JWT would only replace the signature and expiry checks: retirement and the live checks would still need the interceptor, the direct engine-data, broker and source-control calls keep their own verification, and it would add an OIDC discovery document and a JWKS. It becomes useful with per-user OAuth, which is out of scope here.

Tool permissions. No Policy for now. With AWS_IAM, Policy only sees the runtime role, and the role inside a stage is asserted by the runtime, so trusted role claims are the prerequisite. Tool authorization stays in the interceptor, including for customer MCP targets in #615.