[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-09-10

[github-actions[bot]]

Executive Summary

A Claude Code user (no GitHub Copilot) can complete Quick Start using ANTHROPIC_API_KEY, but the docs still steer new users toward Copilot as the assumed default, and the CLAUDE_CODE_OAUTH_TOKEN pitfall — flagged for 19 consecutive daily runs since 2026-08-22 — remains buried in CLI prose instead of surfaced where a Claude Code user actually authenticates. Key finding: the bigger accessibility gap this run isn't auth clarity, it's example parity — Copilot ships ~2x as many workflow examples and 9 dedicated smoke tests for advanced config (Azure auth, ARM runners, model tiers, sub-agents) versus 3 for Claude with no equivalent coverage.

Severity Findings

Critical / Major / Minor findings (click to expand)

Critical Blockers: none this run (unchanged from the last 2 runs — the OAuth-token issue remains downgraded to Major since it's now documented in prose).

Major Obstacles

1. CLAUDE_CODE_OAUTH_TOKEN (e.g. from claude login) is silently ignored, causing an opaque downstream Claude CLI auth failure that never names the token — documented only as plain prose at docs/src/content/docs/setup/cli.md:240, absent from the Claude tab a new user actually follows at docs/src/content/docs/setup/quick-start.mdx:134-138. Unresolved across 19 consecutive daily runs since 2026-08-22.
2. No worked Anthropic Workload Identity Federation (WIF) example anywhere — docs/src/content/docs/introduction/how-they-work.mdx:33 gives only a bare link, unlike Gemini's explicit inline recommendation ("To avoid a long-lived key, configure Google Workload Identity Federation") at docs/src/content/docs/setup/quick-start.mdx:151.
3. Custom agent/MCP scaffolding (.github/agents/agentic-workflows.md, .github/mcp.json, copilot-setup-steps.yml) is generated automatically only on the Copilot gh aw init path (docs/src/content/docs/setup/cli.md:115,139); Claude users are told to hand-author an agent file in "your own agent's format" with no worked example.
4. Example/smoke-test imbalance: Copilot has 9 dedicated smoke tests exercising advanced config (Azure OpenAI auth, ARM runners, SDK mode, model tiers, MAI reasoning, sub-agents) vs. only 3 for Claude, none demonstrating equivalent knobs (e.g., no Bedrock/Vertex or sub-agents smoke test).

Minor Confusion

1. Engine list is inconsistent: docs/src/content/docs/introduction/architecture.mdx:17 lists 4 engines (omits Pi) while docs/src/content/docs/introduction/how-they-work.mdx:26 lists 5.
2. The MCP sandboxing diagram at architecture.mdx:304 labels the engine box "(Copilot, Claude, Codex)" only, leaving Gemini/Pi coverage undocumented there.
3. docs/src/content/docs/reference/tools.md:144 gives Codex a dedicated tool-restriction callout (web-search disabled by default) but no equivalent Claude-specific tool-availability note beyond a one-line timeout default (tools.md:255).
4. Pi's per-model-prefix auth resolution (Copilot default, anthropic/ prefix, openai/codex/ prefix) is described only at a high level at how-they-work.mdx:36, with no worked example — relevant to Claude users routing through Pi.

Engine & Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	Default, no extra account (quick-start.mdx:73)	~122 of 283 tagged files (~43%, incl. engine-omitted defaults per reference/engines.md:22)	copilot-requests write or COPILOT_GITHUB_TOKEN (how-they-work.mdx:32)	5/5
Claude	ANTHROPIC_API_KEY (quick-start.mdx:135) or WIF (link only)	56 files, only 3 smoke tests	ANTHROPIC_API_KEY; OAuth-token silently ignored (cli.md:240)	3/5
Codex	OPENAI_API_KEY/CODEX_API_KEY (quick-start.mdx:140-144), precedence documented clearly	74 files	Clear precedence rules	4/5
Custom	Hand-authored agent format; no scaffolding outside Copilot init (cli.md:139)	46 files (mostly smoke-*)	Engine-dependent	2/5

Tool classification (from reference/tools.md): 11 tools are universal/engine-agnostic (edit, github, bash, web-fetch, playwright, cache-memory, etc.); 1 Codex-specific restriction (web-search off by default, tools.md:144); 1 Claude-specific note (60s vs Codex 120s timeout default, tools.md:255); 1 Pi-specific requirement (tools.cli-proxy: true + gh-proxy mode, quick-start.mdx:156).

Parity note from the example scan: for a Claude-only user, roughly 43% of tagged example workflows aren't directly usable as Claude references, and the smoke-test suite demonstrates Copilot's advanced auth/runner/model-tier surface far more thoroughly than any Claude equivalent.

Auth Gaps

• Required secrets: Copilot → copilot-requests/COPILOT_GITHUB_TOKEN; Claude → ANTHROPIC_API_KEY or unlinked WIF; Codex → CODEX_API_KEY (precedence) or OPENAI_API_KEY; Gemini → GEMINI_API_KEY + linked WIF; Pi → inherits from model-prefix engine.
• Setup steps: Claude's is the shortest (create key in Anthropic Console → gh secret set ANTHROPIC_API_KEY, quick-start.mdx:135-137) but has no WIF walkthrough, unlike Gemini's.
• Explicit warning gap: cli.md already uses [!NOTE]-style callouts for lower-severity issues (e.g. line 416 Codespaces permission note, line 468 HTTP 404 log warning) but the OAuth-token trap at line 240 — arguably more disruptive since it produces a misleading error — is still plain prose, not a callout.
• Missing instruction: quick-start.mdx's Claude tab (134-138) never mentions the OAuth-token pitfall or a WIF alternative at all; a user following only Quick Start (the documented entry point) gets no warning before hitting the opaque Claude CLI failure.

Recommended Actions

Priority 1

• Move the CLAUDE_CODE_OAUTH_TOKEN warning from cli.md:240 prose into a [!WARNING] callout, and duplicate/link it directly in quick-start.mdx's Claude tab (near line 134-138) where users actually authenticate.
• Add a concrete Anthropic WIF setup example (or at least a one-line recommendation mirroring quick-start.mdx:151) instead of a bare link.

Priority 2

• Add one Claude-engine smoke test demonstrating an advanced config knob (e.g., Bedrock/Vertex backend or sub-agents) to close the parity gap the example scan identified.
• Replace the generic "author an agent file in your own format" pointer (cli.md:139) with a worked Claude Code subagent example.

Priority 3

• Reconcile the engine list across architecture.mdx:17 and how-they-work.mdx:26 (Pi omission) and update the MCP sandboxing diagram (architecture.mdx:304) to note Gemini/Pi coverage explicitly.

Trend: This is the 19th consecutive daily run flagging the OAuth-token documentation gap (first seen 2026-08-22, downgraded from Critical to Major on 2026-09-08 once prose coverage landed). It remains unresolved at the quick-start entry point.

References:

• §34477855975 (this run)
• §34352185840 (2026-09-09)
• §34227198249 (2026-09-08)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 367.2 AIC · ⌖ 7.63 AIC · ⊞ 6.3K · ◷

• expires on Sep 11, 2026, 4:53 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-09-11T12:53:24.620Z.

Closed by Workflow