Enforce AWF domain policy on Claude hosted web search and fetch #8928

Summary

Add Claude-specific hosted-web policy enforcement to the trusted API-proxy sidecar. When an Anthropic Messages request contains a versioned web_search_* or web_fetch_* server tool, AWF must be able to inject and enforce an AWF-configured domain policy before forwarding the request upstream.

This closes a semantic egress gap: Claude can ask Anthropic infrastructure to search or fetch a domain that the agent container itself cannot reach through Squid, and the result returns inside the already-allowed Anthropic API response. Squid sees only the Anthropic endpoint, not the searched or fetched destination.

This issue is intentionally Claude-only. Do not add Codex/OpenAI enforcement in this change. We should create a separate Codex issue after the Claude contract and implementation are proven.

Related compiler work: github/gh-aw#62957.

Problem and threat model

The API-proxy sidecar is trusted and has unrestricted HTTP/HTTPS egress through Squid so that it can reach configured LLM provider endpoints. The agent does not receive the real Anthropic credential, but it can submit Anthropic Messages request bodies through the sidecar.

Anthropic's hosted web tools execute outside the AWF network boundary:

1. The agent submits a permitted request to the Anthropic API through API-proxy.
2. The request declares a hosted web_search_* or web_fetch_* tool.
3. Anthropic searches or fetches an external domain from provider infrastructure.
4. External content returns in the Anthropic API response.

The external destination is therefore invisible to Squid ACL enforcement and Squid access logs. This can bypass the intended domain policy, enable indirect prompt injection, and create a blind exfiltration channel through model-generated queries or URLs.

Anthropic supports provider-side domain filters on hosted web tool definitions:

• allowed_domains
• blocked_domains
• max_uses

Anthropic does not permit allowed_domains and blocked_domains on the same tool definition. AWF therefore needs a canonical, unambiguous policy representation and fail-closed conflict handling.

Both search and fetch must be covered. Restricting search alone is insufficient because a request can directly fetch a URL, or search can discover a URL that fetch retrieves in full.

Proposed AWF configuration contract

Add a config-only provider policy under apiProxy.hostedWeb.claude:

apiProxy:
  hostedWeb:
    claude:
      enabled: true
      allowedDomains:
        - docs.github.com
        - nodejs.org
      maxUses: 5

Blocklist mode is also supported:

apiProxy:
  hostedWeb:
    claude:
      enabled: true
      blockedDomains:
        - untrusted.example
        - ads.example

To prohibit Claude hosted web tools entirely:

apiProxy:
  hostedWeb:
    claude:
      enabled: false

Schema requirements

apiProxy.hostedWeb and apiProxy.hostedWeb.claude must be closed objects with additionalProperties: false.

claude properties:

The first implementation should not include an implicit unrestricted enabled: true state. If enabled is true, exactly one of allowedDomains or blockedDomains must be present and non-empty. This avoids a success-shaped configuration that appears protected but forwards unrestricted hosted web access.

For backward compatibility, absence of apiProxy.hostedWeb.claude preserves current pass-through behavior. The schema and specification must explicitly warn that omission does not constrain Claude-hosted egress. The gh-aw compiler can opt into enforcement by always emitting the object when workflow frontmatter declares a network policy.

Domain values must use one canonical syntax accepted by Anthropic and normalized by AWF. At minimum:

• lowercase DNS hostnames only;
• no URL scheme, path, query, fragment, credentials, or port;
• no raw IP addresses, CIDRs, wildcards, localhost names, Docker service aliases, or empty strings;
• duplicates removed after normalization;
• document whether a parent domain includes subdomains, matching Anthropic's actual semantics.

Do not expose network.sensitiveAllowDomains to Anthropic. Sensitive domains exist for direct, credential-bearing network access and must never become provider-hosted search/fetch destinations.

Compiler-facing design requirement

This contract must be easy for gh-aw to generate from workflow frontmatter. The compiler should only need to produce a stable AWF config document and pipe it to:

awf --config - -- <command>

The compiler must not need to understand Anthropic API request JSON, tool-version names, or API-proxy environment variables. AWF owns provider-specific translation and enforcement.

For a workflow allowlist, a compiler-generated stdin document should be as direct as:

{
  "network": {
    "allowDomains": ["api.github.com", "docs.github.com"]
  },
  "apiProxy": {
    "hostedWeb": {
      "claude": {
        "enabled": true,
        "allowedDomains": ["docs.github.com"]
      }
    }
  }
}

The hosted-web list is intentionally explicit rather than automatically copying all network.allowDomains. The network list can contain provider API endpoints, internal services, ecosystem expansions, or other entries that are inappropriate to disclose to or authorize for provider-hosted web retrieval. gh-aw may derive the hosted-web list from frontmatter, but it must emit the resulting list explicitly in AWF config.

Normative precedence rules

Document these rules in both docs/awf-config-spec.md and the relevant JSON Schema description fields. The schema descriptions must not merely point to prose for the security-critical behavior.

Configuration-source precedence

The effective configured Claude hosted-web policy is selected before request processing:

1. Explicit CLI option, if a CLI equivalent is added later.
2. AWF configuration document, including JSON or YAML supplied via --config -.
3. AWF internal default/current compatibility behavior.

No CLI or environment alias is required for the initial implementation. Treat apiProxy.hostedWeb.claude as config-only. If an environment compatibility variable is introduced internally to transport policy to the sidecar, it is an implementation detail and must not independently override the validated AWF config.

Request-level precedence

The AWF-configured policy is an immutable security boundary. Request-provided tool settings may narrow it but must never broaden, remove, replace, or disable it.

1. enabled: false always wins. Any matching hosted tool causes the request to be rejected before upstream dispatch.
2. In configured allowlist mode:
   • omitted request allowed_domains means AWF injects the configured allowlist;
   • request allowed_domains is reduced to the intersection with the configured allowlist;
   • an empty intersection rejects the request;
   • request blocked_domains conflicts with the configured provider representation and rejects the request rather than dropping either restriction.
3. In configured blocklist mode:
   • omitted request blocked_domains means AWF injects the configured blocklist;
   • request blocked_domains is unioned with the configured blocklist;
   • request allowed_domains conflicts with the configured provider representation and rejects the request rather than dropping either restriction.
4. If a request illegally contains both allowed_domains and blocked_domains, reject it explicitly.
5. Configured maxUses is an upper bound:
   • if the request omits max_uses, inject the configured value;
   • otherwise set the effective value to min(configured, requested);
   • malformed, zero, negative, or non-integer values are rejected.
6. AWF-generated values replace the relevant request fields only after the effective policy has been calculated. Never silently strip a hosted tool or silently forward an unprotected tool.

Cross-mode request restrictions are deliberately rejected because Anthropic cannot represent allow and block lists together. Rejecting is safer and more diagnosable than pretending both policies were enforced.

API-proxy behavior

Apply the policy only to Anthropic Messages API request bodies that contain Anthropic-hosted web tools.

Tool matching

Recognize supported versioned tool types, including current forms such as:

• web_search_20250305
• web_search_20260209
• web_search_20260318
• web_fetch_20250910
• later documented web_fetch_* versions

Do not maintain a permanently incomplete exact-name list. Implement a validated matcher for web_search_YYYYMMDD and web_fetch_YYYYMMDD, while rejecting malformed lookalikes. Add focused tests for known versions and unknown future date versions.

If Anthropic introduces a materially different tool shape under one of these prefixes, malformed/unrecognized fields must fail closed rather than pass through without policy.

Transformation point

• Parse and validate the JSON body before upstream dispatch.
• Apply Claude hosted-web enforcement after generic body parsing and before the request is sent to Anthropic.
• Compose it with existing Anthropic model rewriting, prompt-cache, and tool transforms without allowing another transform to re-expand or remove the enforced policy.
• Parse the serialized policy once at sidecar startup and fail startup on invalid internal policy rather than discovering it on the first request.
• Return structured API-proxy errors with stable error types for disabled tools, policy conflicts, empty intersections, malformed tool definitions, and invalid domain values.
• Do not log full prompts, URLs, queries, or request bodies as part of policy errors.

Custom Anthropic-compatible endpoints may not implement hosted server tools. Policy enforcement should activate only when a matching hosted tool is present. A configured policy must not mutate ordinary Messages requests that contain no matching tool.

Required implementation surfaces

Update every relevant config, runtime, documentation, and test surface; do not implement this only in the proxy transform.

Configuration and types

• docs/awf-config.schema.json — canonical public schema.
• src/awf-config-schema.json — runtime/bundled schema copy.
• scripts/generate-schema.mjs or its canonical schema-generation source so generated schema copies remain identical.
• src/config-file.ts / AwfFileConfig types — typed config shape and validation.
• src/config-mapper.ts — map the config-only policy into runtime options.
• CLI/runtime option types used by WrapperConfig and API-proxy service configuration.
• src/services/api-proxy-env-config.ts — serialize one normalized policy object into a dedicated sidecar environment variable.

The sidecar environment variable must be generated from already validated config and must not accept an unvalidated, agent-controlled override.

API-proxy

• containers/api-proxy/server.js — load and validate normalized policy at startup and compose the transform.
• containers/api-proxy/providers/anthropic.js — install the Claude hosted-web transform in a deterministic order.
• containers/api-proxy/anthropic-transforms.js or a focused adjacent module — implement policy calculation and request mutation.
• containers/api-proxy/body-handler.js — preserve explicit fail-closed transform errors and appropriate HTTP status/error bodies.

Documentation

• docs/awf-config-spec.md — add a normative Claude hosted-web section, complete config/request precedence tables, stdin examples, errors, and compatibility behavior.
• docs/api-proxy-sidecar.md — explain why Squid cannot observe provider-hosted destinations, the sidecar trust boundary, and how enforcement works.
• Any README/config examples that enumerate apiProxy policy fields.
• Release/schema documentation if the new schema property affects published artifacts.

Stdin and schema parity requirements

--config - is a first-class compiler interface and must receive explicit coverage. The feature must behave identically for:

• JSON file;
• YAML file;
• JSON stdin;
• YAML stdin.

Validation errors from stdin must identify stdin as the source and reject before containers start.

Keep all schema artifacts synchronized. Existing checks that compare docs/awf-config.schema.json, src/awf-config-schema.json, and generated output must continue to pass. Add schema tests for valid allowlist, valid blocklist, disabled mode, mutual exclusion, required non-empty lists, domain syntax, maxUses, unknown properties, and nested closed-object behavior.

Test matrix

Add unit/integration coverage for at least:

• config mapping from JSON/YAML files and JSON/YAML stdin;
• schema copies and generated schema staying byte-for-byte/in-memory equivalent;
• no matching web tool: request unchanged;
• enabled: false: search rejected;
• enabled: false: fetch rejected;
• allowlist injection when request omits filters;
• allowlist intersection when request narrows policy;
• empty allowlist intersection rejected;
• client allowlist broadening cannot expand the configured list;
• blocklist injection when request omits filters;
• blocklist union when request adds restrictions;
• client cannot remove a configured blocked domain;
• allow-mode request containing blocked_domains rejected;
• block-mode request containing allowed_domains rejected;
• request containing both fields rejected;
• configured and requested max_uses use the lower value;
• malformed domain/filter/tool definitions rejected before upstream dispatch;
• supported current search and fetch versions;
• syntactically valid future date-version tool names;
• malformed prefix lookalikes rejected or left untouched according to an explicitly tested matcher contract;
• multiple matching tools in one request are all enforced;
• existing Anthropic prompt-cache/model transforms cannot undo policy;
• network.sensitiveAllowDomains never appears in provider request bodies or sidecar policy;
• structured errors do not include sensitive request content;
• API-proxy startup fails on an invalid serialized internal policy.

Where practical, assert the exact upstream JSON body observed by a mock Anthropic endpoint, not only helper return values.

Acceptance criteria

• A conforming AWF config can allowlist, blocklist, or disable Claude hosted web search and fetch.
• The same config works identically from a file and --config -, in JSON and YAML.
• AWF policy is an immutable upper bound; request fields can narrow only in the same policy mode.
• Conflicting or unrepresentable policies fail closed with stable, explicit errors.
• Every matching web_search_* and web_fetch_* tool is enforced before upstream dispatch.
• Ordinary Anthropic Messages requests are unchanged.
• Sensitive/internal network entries are not implicitly disclosed or authorized for hosted retrieval.
• Public and runtime schema copies, generated schema output, TypeScript types, config mapping, specs, and docs are synchronized.
• Schema property descriptions include defaults, mutual exclusions, stdin behavior, and precedence rules needed by compiler authors.
• gh-aw can compile frontmatter into a stable AWF config object without knowing Anthropic request-body details.
• Claude-focused tests cover the complete precedence and failure matrix.
• No Codex/OpenAI hosted-search support is included in this issue.