Skip to content

bug(testing): opencode-terminal live suite times out waiting for the permission condition on every version #1009

Open

Description

@Juliusolsson05

Expected

packages/opencode-terminal-headless/src/OpencodeTerminalHeadless.live.test.ts (opt-in, OPENCODE_TERMINAL_HEADLESS_LIVE=1) completes a real turn: prompt → bash tool call → permission condition detected and answered through the package. This suite is the per-version compatibility pass the opencode acceptance discipline in support/upstream-versions.json relies on.

Actual

The test times out at the permission wait (src/testing/replay.ts waitUntil, thrown from OpencodeTerminalHeadless.live.test.ts:126) — identical failure against both:

  • the bundled 1.18.31 binary (third_party/opencode/cache/darwin-arm64/opencode, archive+binary sha256 verified per the feat(setup): bundle the OpenCode CLI as a managed runtime binary #994 manifest), and
  • a freshly downloaded, hash-matching 1.18.30 release binary — the same version the Stage 0 live recordings (testing/fixtures/live, 2026-09-11) were captured against and passed with.

Evidence (2026-09-19)

  • OPENCODE_TERMINAL_HEADLESS_LIVE=1 OPENCODE_BINARY=<1.18.31 cache binary> … vitest run --config vitest.live.config.ts → timed out waiting for permission condition after ~139 s.
  • Same invocation against a 1.18.30 darwin-arm64 release download → identical timeout, ~139 s.
  • Control: the free model path itself works — opencode run -m opencode/big-pickle "Reply with exactly: pong" in the same throwaway-HOME isolation returns pong (exit 0) on 1.18.30. So the failure is not auth, network, or the free model being gone outright.
  • The production app is driving 1.18.31 TUI panes successfully (conditions included) in daily use, so the package's runtime path works; it is the recorded scenario's tool-call → permission surface that no longer reproduces.

Impact

  • The compatibility pass cannot currently discriminate any two opencode versions, so acceptance of version bumps (e.g. the feat(setup): bundle the OpenCode CLI as a managed runtime binary #994 bundled pin at 1.18.31) cannot ride the live suite; acceptance had to fall back to deterministic suites + production evidence.
  • Any future opencode release that changes the permission/question surface would go undetected by this suite — it is red for everything, so red no longer means anything.

Likely suspects (unverified)

  • The free opencode/big-pickle backing model no longer issues the bash tool call for the scenario prompt (plain replies work; tool-calling may have changed).
  • The TUI permission surface or its server-bus vocabulary changed in a release deployed to the free-model path in a version-nonspecific way (both tested versions were built after the recordings).

Environment

macOS darwin-arm64, Node 24, package at current main, binaries: official upstream release assets v1.18.30 and v1.18.31.

Activity

  1. Juliusolsson05 commented on Sep 27, 2026

    @Juliusolsson05
    OwnerAuthor

    Triage (W1, 2026-09-26), opencode 1.18.32:

    The free model still calls the tool. In a throwaway HOME, opencode run --format json -m opencode/big-pickle with the live scenario's exact prompt emitted tool_use bash {"command": "ls -1"} and then answered "2 entries." (exit 0, 28 s). So suspect 1 (big-pickle no longer issues the bash call) is ruled out. The failure is on the TUI / permission-condition side (suspect 2): the permission surface, or how the package detects it.

    Today's live-suite run could not reach that step. At load average ~45 (about 20 agents running), prepareOpencodeTerminalLaunch returned dbPath: null. opencode db path overran its documented 20 s budget, since the ~143 MB Bun binary starts slowly under pressure, and the second test timed out waiting for the TUI. That is the known transient this package already documents, not the issue's failure.

    Next step: one live-suite run on a quiet machine (OPENCODE_TERMINAL_HEADLESS_LIVE=1, 1.18.32, with the PTY stream recorded) to capture what the TUI paints when the bash permission is requested, then compare it with the package's opencode.permission detector.

  2. Juliusolsson05 commented on Sep 27, 2026

    @Juliusolsson05
    OwnerAuthor

    Second live attempt (warm binary): the TUI server never reported its URL within 120 s at load ~45, the same starvation behind this machine's failed opencode reviewer spawns. The permission screen can only be captured on a quiet machine. Labelled needs-evidence; released by W1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    class:C9-testsTest hygieneneeds-evidenceNeeds a recording/real use to settleprovider:opencodeAffects OpenCode integrationsev:P2Real bug with a workaroundtype:bugSomething works wrong

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions