Skip to content

Codex OpenAI proxy fails because OPENAI_API_KEY is excluded from AWF sandbox #32446

Closed

Description

@chrizbo

Summary

After trying the v0.74.2 prerelease, the original Codex model-selection issue appears fixed: generated lock files now invoke Codex with --model "$GH_AW_MODEL_AGENT_CODEX" instead of -c model=....

However, workflows using the OpenAI Codex engine now fail because the sandboxed Codex process reports:

Missing environment variable: `OPENAI_API_KEY`.

Reproduction Context

Repo/PR:

Workflow tested:

  • .github/workflows/daily-standup-prep.lock.yml
  • Triggered manually against branch openai-aw-updates
  • Repo secret OPENAI_API_KEY is configured
  • CODEX_API_KEY is not configured

What Worked

The prerelease fixes the model flag generation. In the failed run, the logs show the requested model reaching Codex:

codex --model openai/gpt-5-mini exec ...
model: "openai/gpt-5-mini", model_provider_id: "openai-proxy"
Model: openai/gpt-5-mini

What Failed

The agent fails before useful work with:

Missing environment variable: `OPENAI_API_KEY`.

The generated lock validates OPENAI_API_KEY successfully during activation and later sets both env vars for the Codex step:

CODEX_API_KEY: ${{ secrets.CODEX_API_KEY || secrets.OPENAI_API_KEY }}
OPENAI_API_KEY: ${{ secrets.CODEX_API_KEY || secrets.OPENAI_API_KEY }}

But the generated awf command excludes both variables from the sandboxed execution:

--exclude-env CODEX_API_KEY ... --exclude-env OPENAI_API_KEY

The generated Codex config also points Codex at the OpenAI proxy while still declaring an env var requirement:

model_provider = "openai-proxy"

[model_providers.openai-proxy]
name = "OpenAI AWF proxy"
base_url = "http://172.30.0.30:10000"
env_key = "OPENAI_API_KEY"
supports_websockets = false

Expected Behavior

When gh-aw runs Codex through the AWF OpenAI proxy, the sandboxed Codex process should not fail because the raw provider secret is intentionally excluded from the agent container.

Question

Should gh-aw:

  • inject a non-secret placeholder OPENAI_API_KEY for the proxy provider,
  • configure Codex to use a different auth path when openai-proxy is active, or
  • omit/avoid env_key = "OPENAI_API_KEY" in the generated Codex config for proxied execution?

Related

This was discovered while testing the prerelease suggested in #32413 (comment). I also left an initial comment there before realizing this should probably be tracked as a separate issue.

Activity

  1. locked and limited conversation to collaborators on May 15, 2026
  2. unlocked this conversation on May 15, 2026
  3. pelikhan commented on May 15, 2026

    @pelikhan
    Collaborator

    /scout investigate the run and figure out if we are configuring codex properly as with something like ollama

  4. lpcox commented on May 15, 2026

    @lpcox
    Collaborator
  5. chrizbo commented on May 20, 2026

    @chrizbo
    ContributorAuthor

    Follow-up after testing the newer AW prerelease:

    • gh-aw v0.74.6 is now available and compiling my repo with it does pick up gh-aw-firewall v0.25.49.
    • The generated lock files still include env_key = "OPENAI_API_KEY" in the Codex openai-proxy provider block.
    • The generated awf command still excludes both CODEX_API_KEY and OPENAI_API_KEY from the sandboxed execution.

    Example from .github/workflows/daily-standup-prep.lock.yml after compiling with v0.74.6:

    model_provider = "openai-proxy"
    
    [model_providers.openai-proxy]
    name = "OpenAI AWF proxy"
    base_url = "http://172.30.0.30:10000"
    env_key = "OPENAI_API_KEY"
    supports_websockets = false

    And the same generated step still has:

    --exclude-env CODEX_API_KEY ... --exclude-env OPENAI_API_KEY
    

    So the firewall version bump appears to have landed in AW, but the compiler/generated Codex config still seems to preserve the problematic env_key requirement. I restored the compile probe locally afterward, so this is just a generated-output check rather than a committed repo change.

  6. locked and limited conversation to collaborators on May 20, 2026
  7. unlocked this conversation on May 20, 2026
  8. pelikhan commented on May 21, 2026

    @pelikhan
    Collaborator

    /scout investigate failure

  9. github-actions commented on May 21, 2026

    @github-actions
    Contributor

    🔭 Recon complete! Scout has charted the territory. Map ready! 🗺️

  10. github-actions commented on May 21, 2026

    @github-actions
    Contributor

    🔍 Scout Research Report

    Triggered by @pelikhan via /scout investigate failure (original prompt: "investigate the run and figure out if we are configuring codex properly as with something like ollama")

    Executive Summary

    The Codex AWF proxy config is not modeled like an Ollama provider, and that's the root cause. In Codex's own config schema, local/proxy providers (Ollama, custom relays) omit env_key entirely — env_key is only meant for providers that need the agent process to read an API key from its environment. The AWF openai-proxy provider does its own upstream auth, but pkg/workflow/codex_mcp.go:168 hardcodes env_key = "OPENAI_API_KEY", which then collides directly with the sandbox's --exclude-env OPENAI_API_KEY flag emitted by pkg/workflow/codex_engine.go:293. The config demands the var; the sandbox strips it; Codex aborts.

    Critical Findings

    • The two conflicting lines:
      • pkg/workflow/codex_mcp.go:168 — yaml.WriteString(indent + "env_key = \"OPENAI_API_KEY\"\n") in renderOpenAIProxyProviderToml
      • pkg/workflow/codex_engine.go:293 — ExcludeEnvVarNames: ComputeAWFExcludeEnvVarNames(workflowData, []string{"CODEX_API_KEY", "OPENAI_API_KEY"})
      • Emission of --exclude-env flags: pkg/workflow/awf_helpers.go:392-403
    • Test pins the broken behavior: pkg/workflow/codex_engine_test.go:347 asserts env_key = "OPENAI_API_KEY" is present in the rendered TOML, so this is intentionally generated, not accidental.
    • Compiled golden output confirms the collision: pkg/workflow/testdata/TestWasmGolden_AllEngines/codex.golden:494 shows --env-all --exclude-env CODEX_API_KEY ... --exclude-env OPENAI_API_KEY alongside the same config block.
    How Ollama / Pi avoids this in gh-aw and in Codex CLI

    In Codex CLI itself, the Ollama provider block has no env_key. From the [Codex provider configuration guide]((www.morphllm.com/redacted)

    [model_providers.ollama]
    name = "Ollama"
    base_url = "(localhost/redacted)
    wire_api = "chat"
    # No env_key — Codex doesn't try to read an API key from the environment

    Providers that do need client-side auth (OpenRouter, Morph, etc.) declare env_key; local/proxy providers that handle their own auth simply omit it.

    In gh-aw, the Pi engine (which talks to Ollama via the AWF gateway) doesn't use env vars for the gateway secret at all. It writes a models.json file mounted into the sandbox at pkg/workflow/pi_engine.go:124-142:

    func buildPiModelsJSON(gatewayPort int, secretEnvVarName, modelID string) string {
        payload := map[string]any{
            "providers": map[string]any{
                "aw-gateway": map[string]any{
                    "baseUrl": fmt.Sprintf("(apiproxy/redacted) gatewayPort),
                    "api":     "openai-completions",
                    "apiKey":  secretEnvVarName, // name only; value comes from mounted file
                    "models":  []map[string]any{{"id": modelID}},
                },
            },
        }
        ...
    }

    So the answer to the original question "are we configuring codex properly as with something like ollama?" is no — Ollama-style providers in Codex omit env_key, while gh-aw's Codex path hardcodes it.

    Why the fall-through reasoning matters

    The AWF proxy at (172.30.0.30/redacted) is the thing that holds the actual upstream OpenAI/Codex credentials. The agent container shouldn't need them at all — that's the whole point of --exclude-env` for sandboxed engines. Right now gh-aw is doing both: stripping the secret from the sandbox and telling Codex to look for it there. One of those two has to give.

    Codex will accept a provider definition without env_key (verified by the Codex model_providers.ollama pattern). It uses the provider's base_url for the actual request and only consults env_key when authenticating. With the proxy doing auth, env_key is dead weight that causes a startup-time check to fail.

    Recommendations

    1. Preferred fix — omit env_key from the openai-proxy block. Match the Ollama pattern in Codex CLI: a proxy provider that handles auth upstream should not declare env_key. Change pkg/workflow/codex_mcp.go:161-170 to drop line 168, and update pkg/workflow/codex_engine_test.go:343-347 accordingly. This requires no change to --exclude-env and keeps the sandbox secrets-free.
    2. Consider also setting wire_api. Codex's Ollama-style providers usually set wire_api = "chat" (Chat Completions) vs. the built-in openai provider's Responses API. Verify whether the AWF OpenAI gateway speaks Responses or Chat Completions and add wire_api if needed — otherwise model behavior may differ from native OpenAI calls. Reference: [Codex provider configuration]((www.morphllm.com/redacted)
    3. If a fallback env_key is desired for non-proxied mode, make the line conditional on whether the firewall/proxy is active (the same condition that already gates renderOpenAIProxyProviderToml). When the firewall is off, the lock file uses the built-in openai provider via the normal env var path and the issue doesn't arise.
    4. Avoid the placeholder workaround. Injecting a non-secret OPENAI_API_KEY=dummy into the sandbox would work, but it re-introduces an env-var contract that future refactors can break, and it muddles the security model (the sandbox is supposed to never see provider credentials, real or fake).

    Key Sources

    Suggested Next Steps

    1. Patch renderOpenAIProxyProviderToml to drop the env_key line (and update the matching test assertion).
    2. Re-run the failing workflow in chrizbo/agentics-beyond-code PR [copilot] Restrict frontmatter schema with additionalProperties: false, align with GitHub Actions schema, and add tests #131 against a freshly compiled lock to confirm Codex starts without the env-var error.
    3. Decide whether to add wire_api = "chat" to the proxy block — this depends on the gateway's API surface and is worth a quick check before shipping.
    4. Add a golden test that asserts the proxy block has no env_key and that the matching --exclude-env OPENAI_API_KEY stays in the awf args, so the regression can't reappear.

    🔭 Intelligence gathered by Scout · ● 3.7M · ◷

  11. lpcox commented on May 21, 2026

    @lpcox
    Collaborator

    This is already handled on the AWF side when --enable-api-proxy is active.

    AWF fix: src/services/api-proxy-service.ts lines 250-261

    When the api-proxy sidecar is enabled, AWF:

    1. Excludes the real OPENAI_API_KEY / CODEX_API_KEY from the agent container (credential isolation)
    2. Injects placeholder values (sk-placeholder-for-api-proxy) so Codex's startup auth check passes
    3. Sets OPENAI_BASE_URL to the sidecar (http://172.30.0.30:10000)
    4. The sidecar injects the real key before forwarding upstream

    The issue reported here is on the gh-aw compilation side: the generated lock file uses --exclude-env OPENAI_API_KEY without also enabling --enable-api-proxy (which would trigger the placeholder injection). The fix should ensure that when gh-aw generates an AWF command for Codex with a proxied openai-proxy provider, it passes --enable-api-proxy and --openai-api-key so AWF handles the credential isolation + placeholder injection automatically.

  12. lpcox commented on May 21, 2026

    @lpcox
    Collaborator

    How to fix via stdin config

    gh-aw's lock compiler should generate an AWF config with apiProxy.enabled: true instead of (or in addition to) --exclude-env OPENAI_API_KEY. Example:

    {
      "apiProxy": {
        "enabled": true
      },
      "network": {
        "allowDomains": ["api.openai.com"]
      }
    }

    Piped to AWF:

    echo '{"apiProxy":{"enabled":true},"network":{"allowDomains":["api.openai.com"]}}' | \
      awf --config - -- codex exec "..."

    When apiProxy.enabled is true and OPENAI_API_KEY is in the host environment, AWF will:

    1. Strip the real key from the agent container
    2. Inject OPENAI_API_KEY=sk-placeholder-for-api-proxy so Codex's auth check passes
    3. Set OPENAI_BASE_URL=http://172.30.0.30:10000 (the sidecar)
    4. The sidecar injects the real key before forwarding upstream

    This eliminates the need for --exclude-env OPENAI_API_KEY — credential isolation is automatic when the api-proxy is enabled.

    See: docs/awf-config-spec.md §5 CLI Mapping and src/services/api-proxy-service.ts lines 250-261

  13. added a commit that references this issue on May 21, 2026
    7e98dbb
  14. locked and limited conversation to collaborators on May 21, 2026
  15. unlocked this conversation on May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions