Skip to content

ci(e2e): permit model-drift regression budget for PR 12769 - #12776

Merged
prekshivyas merged 1 commit into
mainfrom
codex/12667-e2e-budget-policy
Oct 7, 2026
Merged

prekshivyas merged 1 commit into
mainfrom
codex/12667-e2e-budget-policy

Conversation

@rsliter

@rsliter rsliter commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Outcome

Permit PR #12769 to use one exact E2E assertion-budget transition after this policy reaches main. All other PR numbers and budget contents remain subject to the existing growth gate.

Reason

The model-drift regression needs a live same-name onboarding scenario that verifies replacement identity, OpenClaw's selected model, and the recorded selection. Seven new checks replace three redundant checks, adding four assertion points. The independent gate reads policy from the trusted base, so this entry must land before the test update.

Related issues

Refs #12667. Prerequisite for #12769.

Changes

Verification

  • Existing growth-guardrail-parsers.test.ts: 58 tests passed.
  • Actual policy and proposed budget: exact transition accepted; another PR, changed base bytes, changed candidate bytes, and candidate-only authority rejected.
  • Inspected the single-file diff. No secrets, API keys, credentials, or enforcement-code changes.

Review notes

This is a proposed trusted policy entry requiring normal maintainer review and branch protections. No CI waiver is requested. The dependent E2E change remains uncommitted and has not run live. Self-review verified the exception's PR and digest restrictions; independent review is pending.


Signed-off-by: Rebecca Sliter 571084+rsliter@users.noreply.github.com

Summary by CodeRabbit

  • Chores
    • Updated internal release validation tracking for a planned change. No user-facing features or behavior were changed.

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 7, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA/NemoClaw/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: b31ccd3e-4b0b-49d8-9d57-029d689bbdfc
📥 Commits

Reviewing files that changed from the base of the PR and between b430d4d and ff792d0.

📒 Files selected for processing (1)
  • ci/e2e-assertion-growth-exceptions.json

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.


📝 Walkthrough

Walkthrough

The assertion-growth policy adds a proposed budget exception for PR #12769. The comment records the transition conditions and removal timing, and the exception record contains the PR number and budget hashes.

Changes

Assertion Budget Exception

Layer / File(s) Summary
Document and record the budget exception
ci/e2e-assertion-growth-exceptions.json
The policy comment describes the proposed budget transition, prerequisite, invalidation condition, and removal timing. A new record associates PR #12769 with its base and head budget hashes.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~5 minutes

Change: Other

Suggested reviewers: ericksoa

Merge Risk: ⚪ Minimal · up to ff792

No concrete defect in this policy change is established. The exception’s intended use still depends on PR #12769’s candidate budget matching its recorded digest.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the CI E2E policy exception and the specific budget change for PR 12769. It accurately summarizes the main change.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@rsliter
rsliter marked this pull request as ready for review October 7, 2026 20:44
@github-code-quality

github-code-quality Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall line coverage in commit ff792d0 in the codex/12667-e2e-budg... branch is 97%. The line coverage in commit 63002cd in the main branch is 96%.

Show a line coverage summary of the most impacted files.
File main 63002cd codex/12667-e2e-budg... ff792d0 +/-
nemoclaw/src/onboard/config.ts 98% 96% -2%
nemoclaw/src/index.ts 94% 93% -1%
nemoclaw/src/bl...t-management.ts 100% 100% 0%
nemoclaw/src/co.../config-show.ts 100% 100% 0%
nemoclaw/src/commands/slash.ts 100% 100% 0%
nemoclaw/src/on...native-route.ts 0% 100% +100%

TypeScript / code-coverage/cli

The overall line coverage in commit ff792d0 in the codex/12667-e2e-budg... branch is 85%. The line coverage in commit 63002cd in the main branch is 84%.

Show a line coverage summary of the most impacted files.
File main 63002cd codex/12667-e2e-budg... ff792d0 +/-
src/lib/actions.../status-text.ts 84% 46% -38%
src/lib/state/sandbox.ts 92% 83% -9%
src/lib/onboard...al-inference.ts 84% 90% +6%
src/lib/policy/index.ts 71% 80% +9%
src/lib/state/p...l-retirement.ts 79% 92% +13%
src/lib/onboard.../application.ts 55% 72% +17%
src/lib/inferen...nvidia/index.ts 0% 85% +85%
src/lib/adapter...gnostics-cli.ts 0% 87% +87%
src/lib/onboard...ternal-image.ts 0% 91% +91%
src/lib/securit...ig-structure.ts 0% 98% +98%

Updated October 07, 2026 21:01 UTC

@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor finished for commit ff792d0. Include the Advisor findings in the complete PR feedback collection. Verify and group valid findings before repair.

Request review only when Require no Advisor blockers is green.

All previous runs

@prekshivyas
prekshivyas merged commit 5241468 into main Oct 7, 2026
75 checks passed
@prekshivyas
prekshivyas deleted the codex/12667-e2e-budget-policy branch October 7, 2026 21:58
prekshivyas pushed a commit that referenced this pull request Oct 8, 2026
## Outcome

Renew the exact assertion-budget exception for PR #12769 after main
reduced the concurrent-gateway assertion inventory. The exception
continues to permit only the approved four-point increase for same-name
OpenClaw model-change recreation.

## Reason

The original prerequisite #12776 matched the previous whole-file budget
hashes. Main's seven-assertion reduction in #12608 changed those hashes,
so the exact-match guard correctly rejected the updated candidate. The
changed-model regression contract has not expanded.

### Related issues

Refs #12667. Prerequisite for #12769.

## Changes

Update only PR #12769's base and candidate budget hashes and its
explanation. Preserve all other exceptions, all assertion counts, and
the exact PR identity and digest checks. Remove this exception after
#12769 merges.

## Verification

- Existing growth-guardrail parser tests: 58 passed, including
exact-transition acceptance and rejection cases.
- Independently computed hashes match canonical main
`888e95d0ad01db264575181a620923e568c343c7` (budget unchanged from
`c518e121d23c616da1b083c44c9eb504fb419ebd`) and PR candidate
`a377d75342b1db1c6cbe230300373c14822ecd0a`; all sibling exception
entries are unchanged.
- `git diff --check` passed. The diff contains no secrets, API keys, or
credentials.
- Publication validation and CLI, plugin, and JavaScript configuration
TypeScript hooks passed during the guarded normal push.

---
Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Updated automated validation records to reflect a revised approved
limit while preserving previously accepted adjustments. The records also
clarify the conditions associated with the approval. These changes
affect validation tracking only; they do not change product
functionality or user-facing behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
Co-authored-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@wscurran wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance labels Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure chore Build, CI, dependency, or tooling maintenance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants