Skip to content

fix(sandbox): wait out OpenShell 0.1.2's first settings poll before network use - #7922

Merged
ggallen merged 3 commits into
mainfrom
fullsend-3809-settle
Oct 1, 2026
Merged

ggallen merged 3 commits into
mainfrom
fullsend-3809-settle

Conversation

@waynesun09

Copy link
Copy Markdown
Member

Summary

On OpenShell 0.1.2 the sandbox supervisor polls its settings every 10 s (OPENSHELL_POLICY_POLL_INTERVAL_SECS, default 10). Its first poll treats the unchanged provider environment as changed, reloads the policy, and closes any connection in flight. This is NVIDIA/OpenShell#3809, fixed upstream by #3819, which is not in any stable release yet (0.1.3 is pending).

A run's first network request usually lands right at that poll, which caused:

This PR waits, before the pre-flight check, until 12 s have passed since the sandbox became Ready: the 10 s interval plus a margin for supervisor startup. Bootstrap already uses most of that time, so a run typically waits only 3–5 s.

Temporary: remove openShellFirstPollSettle and its call when fullsend pins an OpenShell release that includes #3819.

Changes

  • internal/cli/openshell_settle.go: waitForOpenShellFirstPoll sleeps only for what remains of 12 s since Ready, logs the wait, and returns immediately if the time has passed. Test seams for the clock and sleep.
  • internal/cli/run.go: records when the sandbox becomes Ready after create and waits before the pre-flight check (step 9b-1). Nothing before that step uses the network: bootstrap and copies go over the OpenShell exec channel.

Testing

  • Unit tests: waits the remainder, no wait once the time has passed, works with a nil printer. go test ./internal/cli/ passes, and package time is unchanged.
  • Local runs on OpenShell 0.1.2 (macOS arm64), pi triage with an OpenAI model:
    • With the wait (2 runs): waited 3.5 s and 4.2 s. The first connection opened at +12.4 s / +12.9 s after supervisor start. No retries, and no stale-policy denial.
    • Without it (main, 4 runs): 1 run had its first OpenAI request (opened at +9.6 s) closed at +10.1 s with L7 tunnel closed before inspection because policy changed: policy generation is stale, and pi retried. The other 3 happened to miss the poll.

…etwork use

OpenShell 0.1.2's supervisor polls its settings every 10 s, and its
first poll needlessly reloads the policy and closes connections in
flight (NVIDIA/OpenShell#3809, fixed upstream by #3819 after 0.1.2).
The sandbox log shows it as "L7 tunnel closed before inspection because
policy changed: policy generation is stale". A run's first network
request usually lands around then: the GitHub pre-flight check saw EOF,
and pi's first model request saw "terminated" and had to retry.

Before the pre-flight check, wait until 12 s have passed since the
sandbox became Ready. Bootstrap already uses most of that, so a run
typically waits 3-5 s. Remove this once fullsend pins an OpenShell
release with #3819.

Verified on OpenShell 0.1.2 (macOS arm64): with the wait, two pi triage
runs opened their first connection at +12.4 s / +12.9 s and neither
retried. Without it, one of four control runs had the first OpenAI
request closed at +10.1 s by the stale-policy denial and retried.

Assisted-by: Claude
Signed-off-by: Wayne Sun <gsun@redhat.com>
@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

Wait out OpenShell’s first policy poll before sandbox network use

🐞 Bug fix 🧪 Tests 🕐 10-20 Minutes

Grey Divider

AI Description

• Prevent OpenShell 0.1.2’s first policy poll from interrupting initial sandbox network requests.
• Wait only until 12 seconds after sandbox readiness, allowing bootstrap time to count.
• Test the remaining wait, immediate return, and operation without a printer.
Diagram

graph TD
  Ready["Sandbox Ready"] --> Bootstrap["Sandbox bootstrap"] --> Elapsed{"12s elapsed?"}
  Elapsed -->|No| Wait["Wait remainder"] --> Preflight["GitHub pre-flight"] --> Agent["Agent requests"]
  Elapsed -->|Yes| Preflight
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Retry interrupted requests
  • ➕ Avoids delaying runs whose requests miss the poll.
  • ➖ Requires handling failures in both pre-flight and agent requests.
  • ➖ Still exposes users to a failed first request.
2. Pin an upstream-fixed OpenShell release
  • ➕ Removes the need for a timing workaround.
  • ➖ The upstream fix is not yet available in a stable release.

Recommendation: Keep the readiness-anchored wait as a temporary safeguard: it covers both affected request paths while charging only for time bootstrap has not already consumed. Remove it when the OpenShell pin includes the upstream fix.

Files changed (3) +111 / -0

Bug fix (2) +53 / -0
openshell_settle.goAdd a readiness-anchored first-poll wait +45/-0

Add a readiness-anchored first-poll wait

• Defines a 12-second settle period for OpenShell 0.1.2 and sleeps only for the time remaining since sandbox readiness. Reports the wait when a printer is available and returns immediately when the period has passed.

internal/cli/openshell_settle.go

run.goSettle the sandbox before GitHub pre-flight +8/-0

Settle the sandbox before GitHub pre-flight

• Records the time after sandbox creation succeeds and invokes the settle wait immediately before the GitHub connectivity check. This places the initial check and subsequent agent requests after the first policy poll.

internal/cli/run.go

Tests (1) +58 / -0
openshell_settle_test.goTest settle timing and printer behavior +58/-0

Test settle timing and printer behavior

• Uses clock and sleep test seams to verify the remaining wait, no wait after the deadline, and waiting without a printer.

internal/cli/openshell_settle_test.go

@fullsend-ai-review

fullsend-ai-review Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 3:31 PM UTC · Completed 3:43 PM UTC

Commit: ae19168 · View workflow run →

Runtime: pi · Model: openai/gpt-6.1-sol → gpt-6.1-sol · Effort: high · Cost: $2.25

@codecov

codecov Bot commented Oct 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@qodo-code-review

qodo-code-review Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Remediation recommended

1. Settle wait can expire before a slower or reconfigured poll ✗ Dismissed
Description
waitForOpenShellFirstPoll sleeps a fixed openShellFirstPollSettle of 12 seconds measured from
sandboxReadyAt, which is hard-coded as 10s poll interval + 2s margin even though the comment
itself notes the interval is controlled by the configurable OPENSHELL_POLICY_POLL_INTERVAL_SECS
env var. If a sandbox is started with that variable set above roughly 10 seconds, or the
supervisor's startup/poll scheduling takes longer than the 2 s margin assumes, the first policy poll
can still land after the wait completes and close the GitHub pre-flight or first model connection in
flight, exactly the failure this PR is meant to eliminate.
Code

internal/cli/openshell_settle.go[R19-23]

+// "terminated". Waiting past that first poll keeps both clear of it; the 2 s
+// over the poll interval covers the supervisor's startup before the loop.
+//
+// Remove this once fullsend pins an OpenShell release that includes #3819.
+const openShellFirstPollSettle = 12 * time.Second
Relevance

●●● Strong

Recent CLI precedents accept fixes for timeout-related reliability risks, especially when behavior
depends on slower external systems.

PR-#2015
PR-#7234

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The comment documents that the poll interval is configurable (OPENSHELL_POLICY_POLL_INTERVAL_SECS,
default 10), yet the constant openShellFirstPollSettle is a fixed 12s with no mechanism to read or
honor an overridden interval, so the fix's correctness depends entirely on the default never being
changed.

internal/cli/openshell_settle.go[10-23]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The settle duration used to wait out OpenShell's first policy poll is hard-coded to 12 seconds, based on an assumed default `OPENSHELL_POLICY_POLL_INTERVAL_SECS=10`. If a sandbox is configured with a larger poll interval, the fixed wait is insufficient and the first network request can still be terminated by the stale-policy reload.

## Fix Focus Areas
- internal/cli/openshell_settle.go[10-23]

## Recommended Fix
Read the effective `OPENSHELL_POLICY_POLL_INTERVAL_SECS` value (from the sandbox policy/environment configuration passed into the create call, or from an explicit override) when computing `openShellFirstPollSettle`, adding the same startup margin, so the wait scales with the actual configured interval instead of assuming the default.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. End-to-end checks lack a passing result ✗ Dismissed
Description
runAgent now waits before sandbox network use, but the PR reports package tests and manual runs
without a passing make e2e-test result. The end-to-end workflow is configured to run for Go
changes, but no status for this commit is available to confirm that the checks ran and passed.
Code

internal/cli/run.go[2176]

+	waitForOpenShellFirstPoll(sandboxReadyAt, printer)
Relevance

●●● Strong

The repository accepts additional validation for changed CLI execution paths, and the explicit e2e
rule supports requiring a passing result.

PR-#5192
PR-#7234

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The changed line is under internal/cli/, so rule 1062051 applies. The repository documents `make
e2e-test` for changes in that directory and configures an end-to-end workflow, but the supplied PR
description reports only unit tests and manual runs; no passing end-to-end result was provided.

Rule 1062051: Run end-to-end tests for critical internal modules before merge
internal/cli/run.go[2173-2176]
docs/contributing/go-code.md[62-67]
.github/workflows/e2e.yml[168-171]


Grey Divider

Context sources
✅ Compliance rules (platform): 72 rules
✅ Cross-repo context — repo relationships
  Explored: repo: fullsend-ai/agents (sha: c0939c0a) — View relationship
Review mode: ⚖️ Balanced: This behavioral timing change affects sandbox startup and first network requests, so it carries meaningful runtime risk despite being localized and does not warrant extended redundancy.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Findings visible per group, which tucks the rest behind a View link

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread internal/cli/run.go Resolved
Comment thread internal/cli/openshell_settle.go Resolved
@fullsend-ai-review fullsend-ai-review Bot added the risk/moderate PR risk: moderate label Oct 1, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Risk Assessment: moderate (2/5)

Details

Small scoped bug fix with test coverage and no protected path changes, but touches the high-churn CLI runner without a toggle mechanism.

Previous run

Risk Assessment: moderate (2/5)

Details

Small, well-tested bug fix addressing OpenShell connection terminations, with risk moderated to level 2 due to touching high-churn run.go and modifying startup timing without a feature flag toggle.

Previous run (2)

Risk Assessment: moderate (2/5)

Details

Small, well-tested bug fix addressing OpenShell connection terminations, with risk moderated to level 2 due to touching high-churn run.go and modifying startup timing without a feature flag toggle.

@fullsend-ai-review

fullsend-ai-review Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Looks good to me

Previous run

Looks good to me

Previous run (2)

Review

Findings

Medium

  • [cancellation-handling] internal/cli/openshell_settle.go:43 — The new delay calls time.Sleep without accepting runAgent's context. Cancellation during the wait cannot interrupt the remaining delay (up to 12 seconds), postponing return and sandbox cleanup. The caller then proceeds directly into preflight without checking cancellation. The existing preflight's lack of caller-context support is pre-existing; the additional uninterruptible wait is introduced here.
    Remediation: Pass context.Context into the helper, select between a timer and ctx.Done(), and propagate cancellation before entering preflight. Add tests for an already-cancelled context and cancellation during the wait.

Low

  • [conventions] internal/cli/openshell_settle.go:27 — settleNow and settleSleep do not follow the explicit injectable-function convention in docs/contributing/go-code.md: an xxxFn suffix and individual comments beginning with the variable name and containing an “Override in tests to...” sentence.
    Remediation: Rename the seams to settleNowFn and settleSleepFn, update their references, and add the required individual comments.

Labels: This fixes an OpenShell sandbox startup defect, matching the repository's type/bug and component/sandbox labels.


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added type/bug Confirmed defect in existing behavior component/sandbox OpenShell sandbox environment labels Oct 1, 2026
Review follow-ups:

- waitForOpenShellFirstPoll takes the run's context and waits on a timer
  or ctx.Done(), so a cancelled run no longer sits out up to 12 s before
  cleanup. runAgent returns the cancellation instead of continuing into
  the pre-flight check. Tests cover an already-cancelled context and
  cancellation during the wait.
- Rename the test seams to settleNowFn and settleAfterFn with their own
  doc comments, per the repository's injectable-function convention.

Assisted-by: Claude
Signed-off-by: Wayne Sun <gsun@redhat.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 3:49 PM UTC · Completed 3:58 PM UTC

Commit: 0180c6a · View workflow run →

Runtime: pi · Model: openai/gpt-6.1-sol → gpt-6.1-sol · Effort: high · Cost: $1.52

@fullsend-ai-review
fullsend-ai-review Bot dismissed their stale review October 1, 2026 15:58

Superseded by updated review

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Oct 1, 2026
Review follow-up: waitForOpenShellFirstPoll returned success as soon as
the 12 s window had passed, before looking at the context, so a run
cancelled during a slow bootstrap carried on into the pre-flight check.
Check the context first, and say in the doc comment that the 12 s
assumes the default poll interval, which fullsend never changes.

Assisted-by: Claude (fix), Codex (review)
Signed-off-by: Wayne Sun <gsun@redhat.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 4:03 PM UTC · Completed 4:14 PM UTC

Commit: d7bd3ef · View workflow run →

Runtime: pi · Model: openai/gpt-6.1-sol → gpt-6.1-sol · Effort: high · Cost: $2.29

@fullsend-ai-review
fullsend-ai-review Bot dismissed their stale review October 1, 2026 16:14

Superseded by updated review

@ggallen
ggallen added this pull request to the merge queue Oct 1, 2026
Merged via the queue into main with commit 6ff5063 Oct 1, 2026
41 checks passed
@ggallen
ggallen deleted the fullsend-3809-settle branch October 1, 2026 21:12

This branch was previously deployed

1 inactive deployment
dev — d7bd3eff Deployed Oct 1, 2026 by waynesun09 via behaviour #14833
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/sandbox OpenShell sandbox environment ready-for-merge All reviewers approved — ready to merge risk/moderate PR risk: moderate type/bug Confirmed defect in existing behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants