Skip to content

API proxy routes Copilot model requests to GHES API instead of Copilot API on Enterprise Server #1300

Closed

Description

@lpcox

Environment

  • gh aw v0.58.1
  • Agentic Workflow Firewall v0.24.1
  • Repository hosted on GitHub Enterprise Server (GHES)
  • Engine: GitHub Copilot CLI

Problem

When running agentic workflows on a GHES-hosted repository, the API proxy routes Copilot API requests (model listing, chat completions) to the GHES API endpoint (api.<ghes-host>) instead of the Copilot API (api.githubcopilot.com or api.enterprise.githubcopilot.com).

The Copilot CLI inside the agent container fails immediately with:

Error loading models: Error: Failed to list models: 400 Bad Request

0 turns, 0 token usage — the agent never starts a conversation.

Root Cause

The lock file passes GITHUB_API_URL: ${{ github.api_url }} to the agent job. On GHES, this resolves to https://api.<ghes-host>. The API proxy appears to use this as the upstream for all API requests, including Copilot model/chat requests.

Squid proxy access logs confirm the API proxy sent requests to the GHES API:

172.30.0.30:47750 api.<ghes-host>:443 CONNECT 200 TCP_TUNNEL:HIER_DIRECT
172.30.0.30:47762 api.<ghes-host>:443 CONNECT 200 TCP_TUNNEL:HIER_DIRECT

The Copilot models endpoint does not exist on the GHES API, so it returns 400.

Copilot CLI agent log

[DEBUG] Using COPILOT_API_URL from environment: http://172.30.0.30:10002
[ERROR] Request to GitHub API at http://172.30.0.30:10002/agents/swe/custom-agents/<org>/<repo>?... failed with status 404
[ERROR] Error loading models: Error: Failed to list models: 400 Bad Request
[ERROR] MCP transport for github closed
[DEBUG] runPromptMode: executePromptDirectly returned succeeded=false
[DEBUG] runPromptMode: exiting with code 1

Expected Behavior

The API proxy should route Copilot API requests (token exchange, model listing, chat completions) to the Copilot API service (api.githubcopilot.com / api.enterprise.githubcopilot.com), while using the GHES API URL (api.<ghes-host>) only for GitHub REST API calls.

Workaround

None known — this appears to require changes to the API proxy to distinguish between GitHub REST API and Copilot API routing on GHES.

Related

Activity

  1. lpcox commented on Mar 14, 2026

    @lpcox
    CollaboratorAuthor

    Additional Context: Why this only affects GHES

    On github.com, the Copilot CLI performs a token exchange against api.github.com using the COPILOT_GITHUB_TOKEN. The token exchange response includes the actual Copilot API URL (e.g., api.githubcopilot.com). The CLI then directs all model listing and chat completion requests to that discovered URL. The API proxy handles this correctly because both api.github.com (for token exchange) and api.githubcopilot.com (for model/chat) are well-known endpoints.

    On GHES, the same token exchange call is sent to api.<ghes-host>, which either:

    1. Does not support the Copilot token exchange endpoint at all, or
    2. Returns a response that does not include the correct Copilot API URL

    As a result, the CLI never discovers the real Copilot API endpoint and falls back to routing all requests (including model listing) through the API proxy to api.<ghes-host>, which does not serve the Copilot models endpoint — resulting in 400 Bad Request.

    Suggested Fix

    The API proxy may need to handle GHES environments by:

    • Routing the initial token exchange to api.github.com (or the appropriate Copilot token endpoint) rather than the GHES API
    • Or allowing an explicit COPILOT_API_URL override separate from GITHUB_API_URL so the Copilot API discovery/routing can be configured independently of the GitHub REST API endpoint
  2. lpcox commented on Mar 14, 2026

    @lpcox
    CollaboratorAuthor

    Update: Diagnostic Results & Root Cause Analysis

    After adding diagnostic steps to the workflow, we have a much clearer picture of the failure chain on GHES.

    Diagnostic Approach

    We added a step before the Copilot CLI execution that tests the token exchange endpoint (/copilot_internal/v2/token) with both available tokens, probes Copilot-related GHES endpoints, and checks org-level Copilot configuration.

    Results

    Test Result Interpretation
    COPILOT_GITHUB_TOKEN → GHES /copilot_internal/v2/token 403 "Resource not accessible by personal access token" Endpoint exists but rejects PATs
    github.token (Actions token) → GHES /copilot_internal/v2/token 403 "Resource not accessible by integration" Endpoint exists but rejects Actions tokens too
    COPILOT_GITHUB_TOKEN → api.github.com/copilot_internal/v2/token 401 "Bad credentials" Expected — GHES-scoped token is not valid on github.com
    GHES /copilot_internal/v2/models 404 Models endpoint does not exist on GHES API
    GHES /copilot, /models 404 No Copilot model endpoints on GHES
    GHES /meta → copilot field ✅ Contains IP ranges GHES knows about Copilot infrastructure
    /orgs/<org>/copilot 404 Copilot not configured at org level
    /orgs/<org>/copilot/billing 403 Billing endpoint exists but requires higher permissions
    /orgs/<org>/copilot/billing/seats 403 Same — exists but forbidden

    Root Cause

    Navigating to the enterprise Copilot settings page (/enterprises/<enterprise>/ai-controls/copilot) showed:

    "There are no policies to manage at the moment. Once you enable access to Copilot you'll have the ability to manage policies and feature access for GitHub Copilot inside your enterprise account."

    Copilot has not been licensed/enabled at the enterprise level. This means:

    • The /copilot_internal/v2/token endpoint exists on GHES (it returns 403, not 404) but will reject all tokens until Copilot is provisioned
    • No seats can be assigned at the org level because there is no enterprise Copilot subscription
    • The Copilot CLI fails at model listing because the token exchange never succeeds

    Two Separate Issues

    This investigation surfaced two distinct problems for GHES agentic workflows:

    1. This issue (API proxy routing): Even once Copilot is licensed, the API proxy needs to route Copilot requests to the correct endpoint (likely api.githubcopilot.com or a GHES-specific Copilot API) rather than api.<ghes-host>. The GHES API does not serve /copilot_internal/v2/models (returns 404), confirming the proxy must route to a different host for Copilot model/chat requests.

    2. Copilot licensing prerequisite: The gh aw setup wizard does not check whether Copilot is actually enabled on the target GHES instance. It would be helpful if the wizard or the workflow itself validated Copilot availability early and surfaced a clear error message like "Copilot is not enabled for this enterprise/organization" instead of the opaque Error loading models: 400 Bad Request.

    Suggested Improvements

    • Better error messaging: When the token exchange returns 403, surface a user-friendly message explaining that Copilot may not be enabled for the enterprise/org
    • Pre-flight check: Add a validation step during gh aw add or at workflow startup that verifies Copilot API access before attempting to run the agent
    • GHES documentation: Document the Copilot licensing prerequisite for GHES agentic workflows (requires Copilot Business/Enterprise subscription + GitHub Connect)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions