Skip to content

fix(e2e): extend Docker Hub login retries - #8766

Merged
jyaunches merged 3 commits into
mainfrom
codex/fix-e2e-docker-auth-timeout
Aug 10, 2026
Merged

jyaunches merged 3 commits into
mainfrom
codex/fix-e2e-docker-auth-timeout

Conversation

@jyaunches

@jyaunches jyaunches commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The linked Slack E2E never reached the test because one runner exhausted the shared three-attempt Docker Hub authentication budget during a transient registry connectivity stall. This change gives the shared login path five bounded attempts while preserving credential isolation, immutable provenance, and fail-closed behavior.

Changes

  • Extend Docker Hub authentication from three to five bounded attempts in the shared E2E helper and the equivalent sandbox-image workflow path.
  • Pin the E2E workflow to the signed helper commit and update the enforced action, helper, and workflow provenance hashes.
  • Add regression coverage for recovery on attempts four and five, five-attempt exhaustion, exact sleep counts, and opaque-input changed-test routing.

Failure evidence: E2E run 31428738562, OpenClaw Slack pairing job 93595566444.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This changes an internal CI authentication retry budget; it does not change a public API, CLI, configuration, onboarding flow, UI, or supported runtime behavior, and existing docs do not define the retry count.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent Codex security review passed all nine required categories with no findings; trusted-run gating, stdin-only token transfer, isolated Docker config, immutable pinning, bounded timeouts, cleanup, and fail-closed behavior remain intact.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: Internal CI Docker Hub authentication retry and regression-contract updates only; no public or supported user-facing behavior changed, and existing documentation does not define the retry budget.
  • Agent: Codex Desktop

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 94/94 focused E2E-support boundary tests, 5/5 watch-trigger integration tests, 500/500 affected tests, and the final 12/12 Docker-auth rerun passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Julie Yaunches jyaunches@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Improved Docker Hub authentication reliability by retrying failed logins up to five times with five-second delays.
    • Enhanced warning and error messages to clearly report retry settings.
  • Tests

    • Expanded end-to-end coverage for authentication retries and workflow validation.
    • Added monitoring so relevant workflow and authentication changes trigger the appropriate tests.

@jyaunches jyaunches added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure labels Aug 10, 2026
@jyaunches jyaunches self-assigned this Aug 10, 2026
@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d53ee2fc-e128-442b-906b-fb0b60c0dc22

📥 Commits

Reviewing files that changed from the base of the PR and between 3362694 and 9499c8c.

📒 Files selected for processing (1)
  • test/e2e/support/dockerhub-auth-workflow-boundary.test.ts

📝 Walkthrough

Walkthrough

Docker Hub authentication now retries five times with five-second delays. Workflows, provenance checks, boundary tests, and Vitest watch triggers use the updated contract. The CLI artifact workflow boundary hash is also updated.

Changes

Docker authentication retry

Layer / File(s) Summary
Retry behavior and workflow wiring
.github/scripts/docker-auth-setup.sh, .github/workflows/sandbox-images-and-e2e.yaml, .github/workflows/e2e.yaml
Authentication uses five attempts, five-second delays, and dynamic retry messages. The E2E workflow uses the updated action revision.
Boundary validation and provenance
test/e2e/support/dockerhub-auth-workflow-boundary.test.ts, tools/e2e/sandbox-images-workflow-boundary.mts, tools/e2e/workflow-boundary-policy.mts
Tests validate retry counts, delays, tokens, and failure messages. Boundary requirements and action provenance use the updated contract.
Watch-trigger coverage
test/helpers/vitest-watch-triggers.ts, test/vitest-watch-triggers.test.ts
Watch mappings and tests cover Docker authentication and sandbox-images workflow changes.

CLI workflow contract refresh

Layer / File(s) Summary
CLI artifact contract hash
tools/e2e/cli-artifact-workflow-boundary.mts
The expected CLI artifact workflow contract SHA-256 digest is updated.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant E2EWorkflow
  participant DockerAuthSetup
  participant DockerHub
  E2EWorkflow->>DockerAuthSetup: invoke authentication action
  DockerAuthSetup->>DockerHub: submit login token
  DockerHub-->>DockerAuthSetup: return authentication result
  DockerAuthSetup->>DockerAuthSetup: wait five seconds before retry
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: extending Docker Hub login retries in E2E workflows.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-e2e-docker-auth-timeout

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 9499c8c in the codex/fix-e2e-docker... branch remains at 96%, unchanged from commit 567a033 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 9499c8c in the codex/fix-e2e-docker... branch remains at 82%, unchanged from commit 567a033 in the main branch.

Show a code coverage summary of the most impacted files.
File main 567a033 codex/fix-e2e-docker... 9499c8c +/-
src/lib/messagi...annel-config.ts 99% 95% -4%
src/lib/sandbox...rce-identity.ts 86% 86% 0%
src/lib/tunnel/services.ts 80% 80% 0%
src/lib/adapter...shell/client.ts 88% 90% +2%
src/lib/core/bo...e-transcript.ts 93% 97% +4%

Updated August 10, 2026 22:41 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/support/dockerhub-auth-workflow-boundary.test.ts`:
- Around line 514-546: Add a separate runAuth recovery case with successAttempt:
5 in the test covering Docker Hub authentication retries. Assert status 0, five
Docker calls, five token writes, and four sleep entries, while preserving the
existing attempt-four recovery and exhaustion assertions.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b9ee2bd5-60f4-414f-9358-ce7932deb874

📥 Commits

Reviewing files that changed from the base of the PR and between 567a033 and 3362694.

📒 Files selected for processing (9)
  • .github/scripts/docker-auth-setup.sh
  • .github/workflows/e2e.yaml
  • .github/workflows/sandbox-images-and-e2e.yaml
  • test/e2e/support/dockerhub-auth-workflow-boundary.test.ts
  • test/helpers/vitest-watch-triggers.ts
  • test/vitest-watch-triggers.test.ts
  • tools/e2e/cli-artifact-workflow-boundary.mts
  • tools/e2e/sandbox-images-workflow-boundary.mts
  • tools/e2e/workflow-boundary-policy.mts

@github-actions

github-actions Bot commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Failed after a partial review · low confidence · 0 blockers · 0 warnings · 0 suggestions

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@jyaunches

Copy link
Copy Markdown
Contributor Author

Live E2E evidence: trusted main push run 31437795324 at 567a033 completed the OpenClaw Slack pairing job successfully: https://github.com/NVIDIA/NemoClaw/actions/runs/31437795324/job/93623045829. Docker Hub authentication, the live pairing test, artifact upload, and Docker-auth cleanup all passed. This trusted-main run still uses the pre-PR three-attempt helper, so it confirms the original auth failure was transient and that the previously skipped Slack test is green. The PR retry change itself is covered by the five-attempt boundary tests and can exercise authenticated live behavior only after merge, because manual PR E2E intentionally disables Docker Hub credentials.

@jyaunches
jyaunches merged commit b8232eb into main Aug 10, 2026
85 of 89 checks passed
@jyaunches
jyaunches deleted the codex/fix-e2e-docker-auth-timeout branch August 10, 2026 23:20
jyaunches added a commit that referenced this pull request Aug 11, 2026
<!-- markdownlint-disable MD041 -->
## Summary

This PR updates the August 10, 2026 v0.0.106 release entry with gateway
readiness fixes that merged after PR #8756.
PRs #8765, #8767, and #8768 remain outside this entry because they are
open and do not carry the `v0.0.106` release label.

## Changes

- Document acceptance of OpenShell v0.0.101 `Server:` endpoint output
and target-bound process tags when trusted listener evidence matches the
configured gateway.
- Document preservation of selected-gateway stale state so onboarding
can reconcile a registered gateway when a gateway-scoped OpenShell
status check cannot connect.
- Record evidence-backed exclusions for internal image, startup,
qualification, proxy-environment, CI, and test-harness changes in PRs
#8754, #8609, #8762, #8432, #8766, and #8581.
- Exclude PRs #8765, #8767, and #8768 because their changes are absent
from `main` and the PRs do not carry the `v0.0.106` release label.
The release entry must be updated after any of those PRs merges for
v0.0.106.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [x] Existing tests cover changed behavior — justification:
`test/changelog-docs.test.ts` validates dated changelog SPDX placement,
version headings, forbidden terms, and link form.
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `docs-updated`
- Evidence: `docs/changelog/2026-08-10.mdx`; an independent Codex
Desktop subagent reviewed the writing rules and documentation style,
terminology, structure, voice, code-sample presentation, links, source
and test accuracy, release meaning, product scope, and evidence-backed
exclusions at commit `190bf882c`.
- Agent: Codex Desktop
<!-- docs-review-head-sha: 190bf88 -->
<!-- docs-review-agents-blob-sha: c4923a3 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable; `scripts/prepare-dgx-station-host.sh`
is unchanged.
- Station profile/scenario: Not applicable.
- Result: Not applicable.
- Supporting evidence: Not applicable.

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: `npx
vitest run test/changelog-docs.test.ts` passed 6 tests.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: Not applicable to a
documentation-only release-entry update.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) —
result: passed with 0 errors and 2 existing warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
— no page was added.

---
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved gateway readiness detection for OpenShell v0.0.101 endpoint
output.
* Process tags are now accepted only when they match trusted listener
information for the configured gateway.
* Preserved stale gateway status during connection failures to support
accurate onboarding reconciliation.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Carlos Villela <cvillela@nvidia.com>
@wscurran wscurran added area: security Security controls, permissions, secrets, or hardening chore Build, CI, dependency, or tooling maintenance labels Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure area: security Security controls, permissions, secrets, or hardening chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants