Skip to content

feat: add optional run-wide cap on enclave tool calls (--max-num-tool-calls) - #9271

Merged
lpcox merged 3 commits into
mainfrom
copilot/add-configurable-tool-call-cap
Sep 30, 2026
Merged

lpcox merged 3 commits into
mainfrom
copilot/add-configurable-tool-call-cap

Conversation

Copilot AI commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Tool-call volume drives agentic workflow cost. This adds an opt-in cap on how many enclave tool calls an agent can make in one AWF run. It is enforced in enclave-mcp-server, which already sees every enclave tool request. When the flag is unset, behavior is unchanged.

Configuration

  • CLI --max-num-tool-calls <n> ↔ config rateLimiting.maxNumToolCalls. The key lives under rateLimiting because enclaves is an array of entries with no run-wide slot.
  • The value must be a positive integer. Setting it with no enclave configured fails at startup, so a cap is never silently ignored.
  • The host passes the value to the broker as AWF_ENCLAVE_MAX_TOOL_CALLS, only when set.
rateLimiting:
  maxNumToolCalls: 20
enclaves:
  - agent: { model: gpt-5 }
    repos: [{ repo: octo/private, sensitivity: internal }]

Enforcement (containers/enclave/mcp-server/tool-call-budget.js)

  • Scope: one budget per run, shared by enclave_run_script and enclave_run_agent. It is independent of per-entry maxInvocations and of the disclosure ledger.
  • Counting: every well-formed tools/call to a published tool uses one unit. The check runs before the payload-size check, the busy/single-call check, and the handler, so failed and rejected attempts also count. Requests rejected as -32602 (no published tool named) are not counted.
  • Denial: after exhaustion, calls never reach an executor. They get an isError result with the text "Max tool call count reached, no more tool calls are allowed. Make a decision based on what you already have in context."
    • This is the only enclave error that differs from the canonical {"status":"error"}.
    • The difference is safe because the decision depends only on the caller's own call count and trusted config, never on repository content.
  • Advisory: the limit is appended to each published tool description and returned as initialize.instructions. AWF can't edit the primary agent's system prompt, so this is the closest equivalent.
  • Observability: the first denial logs one warning, Max tool call count reached. {toolName, agentName: <executor kind>, sessionID: <runId>, maxToolCalls}. Later denials are not logged, so a retry loop can't flood the logs.
  • Persistence: {version, runId, used} is written to the broker's private control dir with mode 0600 and an atomic rename, and the temp file is removed if the write fails. A restarted broker resumes the count for the same run id; any other run id starts at zero. A corrupt state file is treated as zero. If a write fails, the in-memory count keeps enforcing the cap.

Decisions on the issue's open questions

  • Global vs. per-session: one global budget per awf run.
  • Include enclave_run_script? Yes, script calls count.
  • Dynamic delegation admission: it happens inside an enclave_run_agent call and is not counted separately.
  • Tool calls inside an enclave agent: not counted; they stay bounded by maxModelRequests and maxModelTokens.
  • Exemptions: none needed. The broker publishes no final-answer tool, and the agent's result/safe-output tools are served by other MCP servers.

Docs

  • docs/awf-config-spec.md: CLI↔config mapping line and a new §14.2a.
  • docs/enclaves-architecture.md: summary paragraph.
  • Schema: docs/awf-config.schema.json, with src/awf-config-schema.json regenerated from it.

Copilot AI changed the title [WIP] Add configurable tool-call cap for cost control feat: add optional run-wide cap on enclave tool calls (--max-num-tool-calls) Sep 30, 2026
Copilot AI requested a review from lpcox September 30, 2026 22:21
@lpcox
lpcox marked this pull request as ready for review September 30, 2026 22:28
Copilot AI balanced review requested due to automatic review settings September 30, 2026 22:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The cap is consistently validated, propagated, enforced, persisted, documented, and covered by focused tests.

Review effort: Balanced
Findings: None

What changed in this PR

Adds an optional run-wide enclave tool-call cap for cost control, enforced and persisted by the enclave MCP broker.

Changes:

  • Adds CLI/config/schema support and validation.
  • Enforces, persists, advertises, and logs the shared call budget.
  • Adds comprehensive tests and documentation.
File Description
src/​types/​enclave-options.ts Defines the cap option.
src/​services/​enclave-mcp-service.ts Passes the cap to the broker.
src/​services/​enclave-dynamic-service.test.ts Tests environment propagation.
src/​enclave/​tool-call-budget.test.ts Tests enforcement and persistence.
src/​config-mapper.ts Maps file configuration.
src/​config-file.ts Types the configuration key.
src/​config-file-validation.test.ts Tests config validation.
src/​config-file-mapping.test.ts Tests config mapping.
src/​commands/​validators/​infrastructure-validator.ts Validates the cap and enclave requirement.
src/​commands/​validators/​infrastructure-validator.test.ts Tests startup validation.
src/​commands/​build-config.ts Builds the runtime option.
src/​cli-options.ts Adds the CLI flag.
src/​awf-config-schema.json Updates the runtime schema.
docs/​enclaves-architecture.md Documents architectural behavior.
docs/​awf-config.schema.json Updates the canonical schema.
docs/​awf-config-spec.md Specifies cap semantics.
containers/​enclave/​mcp-server/​tool-call-budget.js Implements the persistent budget.
containers/​enclave/​mcp-server/​server.js Initializes the budget.
containers/​enclave/​mcp-server/​mcp-protocol.js Enforces and advertises the cap.
containers/​enclave/​mcp-server/​config.js Loads broker configuration.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

  1. Download the docs-preview-pr-9271 artifact from the workflow run
  2. Unzip and open index.html in your browser

Built from commit 0358379

@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 92.71% 92.74% 📈 +0.03%
Statements 91.22% 91.24% 📈 +0.02%
Functions 89.30% 89.30% ➡️ +0.00%
Branches 84.29% 84.32% 📈 +0.03%
📁 Per-file Coverage Changes (3 files)
File Lines (Before → After) Statements (Before → After)
src/nvx/one-shot-adapter.ts 83.5% → 82.9% (-0.60%) 80.3% → 79.8% (-0.56%)
src/log-directory-setup.ts 96.2% → 100.0% (+3.78%) 96.3% → 100.0% (+3.71%)
src/commands/validators/infrastructure-validator.ts 90.3% → 97.3% (+7.06%) 90.4% → 97.4% (+6.95%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • clients2.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "clients2.google.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

✅ Security Guard completed successfully!

PR #9271 security review complete: no vulnerabilities found. The optional run-wide tool-call cap implementation (tool-call-budget.js and mcp-protocol.js) maintains proper isolation, validates inputs, uses atomic file operations with secure permissions (0o600), and enforces the cap before any handler execution. No security-weakening changes to firewall rules, capabilities, or egress controls detected. Feature is optional and backward compatible.

Generated by Security Guard for #9271

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

✅ Build Test Suite completed successfully!

Generated by Build Test Suite for #9271

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Gemini reports failed. Facets need polishing...

💎 Faceted by Smoke Gemini

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Smoke Cloud Hypervisor completed. Cloud Hypervisor + Copilot passed.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

❌ Smoke Claude failed

Generated by Smoke Claude for #9271

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Sep 30, 2026
@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

  • ✅ Allowed domain: api.github.com returned HTTP 403 (curl exit 0)
  • ✅ Blocked domain: example.com blocked (TLS cert error)

Overall: PASS — @lpcox

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Copilot: PASS ✅

  • ✅ GitHub MCP (last merged: "Update vulnerable development dependencies")
  • ✅ github.com connectivity (HTTP 200)
  • ✅ File write/read

PR: feat: add optional run-wide cap on enclave tool calls (--max-num-tool-calls)
Author @Copilot · Assignees @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Cloud Hypervisor + Copilot

  1. list_pull_requests (github/gh-aw-firewall): PASS — tool call succeeded, returned PR data.
  2. curl https://github.com (expect 200/301): FAIL — sandbox denied curl execution (permission request could not be granted, non-interactive).
  3. Write/read temp file in /tmp/gh-aw/agent/: PASS — wrote and read back unique line successfully.
  4. curl (example.com/redacted) (expect blocked 000/403): FAIL — sandbox denied curl execution, so firewall block could not be verified.

Overall: 2/4 PASS. Network checks (2 & 4) were blocked by the sandbox's non-interactive permission layer before reaching the firewall/network, so egress filtering itself could not be exercised.

Cloud Hypervisor + Copilot smoke test by Smoke Cloud Hypervisor
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Services smoke test:

  • Redis PING: ✅ PONG
  • pg_isready: ✅ accepting connections
  • PostgreSQL SELECT 1: ✅ 1

Overall: PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

OTEL smoke test

  • ✅ S1 Module loading: otel.js exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled
  • ✅ S2 Tests: otel.test.js 39/39 passed
  • ✅ S3 Env forwarding: trace ID/parent span ID in env-passthrough.ts; OTEL vars in api-proxy-env-config.ts
  • ✅ S4 Token tracker: onUsage callback present in token-tracker-http.js
  • ✅ S5 Diagnostics: /tmp/gh-aw/otel.jsonl has 1 span record

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct) Mode ✅

Test Results:

  • ✅ GitHub MCP connectivity (PRs fetched)
  • ✅ github.com connectivity (HTTP 200)
  • ✅ File write/read (smoke-test-copilot-byok.txt present)
  • ✅ BYOK inference (direct mode active)

Mode: Direct BYOK (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com

Status: PASS

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia ❌ not run ❌ FAIL
Bun hono ❌ not run ❌ FAIL
C++ fmt ✅ N/A ✅ PASS
C++ json ✅ N/A ✅ PASS
Deno oak N/A not run ❌ FAIL
Deno std N/A not run ❌ FAIL
.NET hello-world ✅ N/A ✅ PASS
.NET json-parse ✅ N/A ✅ PASS
Go color ✅ pass ✅ PASS
Go env ✅ pass ✅ PASS
Go uuid ✅ pass ✅ PASS
Java gson ✅ 1/1 passed ✅ PASS
Java caffeine ✅ 1/1 passed ✅ PASS
Node.js clsx ✅ pass ✅ PASS
Node.js execa ✅ pass ✅ PASS
Node.js p-limit ✅ pass ✅ PASS
Rust fd ✅ 1/1 passed ✅ PASS
Rust zoxide ✅ 1/1 passed ✅ PASS

Overall: 6/8 ecosystems passed — FAIL

Failures:

  • Bun and Deno: the install scripts (curl ... | bash and curl ... | sh) were blocked with "Permission denied and could not request permission from user". Neither bun nor deno is on PATH. I did not try to get around the block, so no tests ran for either ecosystem.
  • Java: ~/.m2/repository could not be created. I ran Maven with -Dmaven.repo.local=/tmp/gh-aw/agent/m2 instead, and both projects then passed.

The build-test label was not added because of the failures.

Generated by Build Test Suite for #9271 · copilot · auto · 27.7 AIC · ⊞ 11.8K · ◷
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test

  • [Test Coverage] src/docker-manager.ts
  • docs: use description and classification for routing
  • GitHub reads ✅
  • Playwright title ✅
  • File write ✅
  • AWF build ✅
  • Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • clients2.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "clients2.google.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python Python 3.12.14 Python 3.12.14 ✅ YES
Node.js v24.21.0 v22.23.2 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Result: Not all tests passed (Node.js version mismatch), so the smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@lpcox
lpcox enabled auto-merge (squash) September 30, 2026 23:06
@lpcox
lpcox merged commit 4efc3ef into main Sep 30, 2026
139 of 144 checks passed
@lpcox
lpcox deleted the copilot/add-configurable-tool-call-cap branch September 30, 2026 23:10

This branch was previously deployed

1 inactive deployment
aoai-model — bda23204 Deployed Sep 30, 2026 by lpcox via conclusion #1834
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add configurable tool-call cap for cost control (enclave-mediated tools)

3 participants