Skip to content

Classify the Remaining Subprocess Failures by Status Rather Than Prose - #3455

Merged
ptr727 merged 5 commits into
developfrom
feature/auto-3378
Oct 25, 2026
Merged

ptr727 merged 5 commits into
developfrom
feature/auto-3378

Conversation

@ptr727

@ptr727 ptr727 commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

Summary

Moves the remaining subprocess failures that were classified by message prose onto the CODESTYLE.md "Operations and Failure Messages" status-code rule.

  • spec/audit.py gh(): with ok404, a failure now reads as absence only when the stderr carries the (HTTP 404) notation gh api prints. Bare "Not Found" or "HTTP 404" prose, another printed status, or no status at all, such as a network error, raises as an unexpected failure.

  • scripts/skills_install.py: the expected "already there" case has its own operation now, the listing, instead of a match on "already" in a failed command's output. Where the marketplace listing already names this checkout, the installer skips claude plugin marketplace add. Where claude plugin list --json shows the plugin installed at user scope, it skips claude plugin install. Each command that does run is judged by its exit status alone. Where the plugin listing cannot be read, the install runs and its exit status decides.

  • host-setup/linux/setup-github.sh gh_account() and host-setup/windows/setup-github.ps1 Get-GhAccount: the check for gh's --active flag (added in gh 2.57) is now the exit status of gh auth status --active --help rather than a match on unknown flag: --active. This is the same capability probe pr_review.py uses for --allow-escape-sequences. The PowerShell twin had the same prose match, so it moves too.

  • host-setup/windows/setup-github.ps1 known-host checks: a local review pass found the same class at both ssh-keygen -F github.com sites, which matched found in the output. They now read the exit status, as the Linux twin already does. scripts/README.md "skills_install.py" names the add and install skips.

The scripts/pr_review.py unknown flag retry is #3365's and is not touched here.

Findings from the local pass, fixed on the pull request

  • tests/test_setup_github.py (introduced, design): the new known-host harness repeated the existing Windows harness's parse-and-extract preamble. Fixed in a10982d, where one windows_harness builder writes it for both.

Dropped findings

  • scripts/README.md:391 (style, pre-existing): four sentences this change did not write exceed the advisory sentence-length check.
  • tests/test_install_tools.py and tests/test_bootstrap.py (consolidation, pre-existing): each carries its own copy of the same parse-and-dot-source pwsh preamble.

Closes on promotion: #3378

Verification

  • New tests run on constructed output: GhAbsenceCase in tests/test_audit_helpers.py, the add, install and listing cases in tests/test_skills_install.py RegisterCase, the help-probe calls in both the Linux and the Windows (pwsh) classes of tests/test_setup_github.py, and TestWindowsKnownHost, which fails against the old -match 'found' read.
  • A pwsh probe confirmed ssh-keygen -F exits 0 for a known host, 1 for an empty file, and 255 for a missing one.
  • The capability probe was checked against the installed gh 2.103.2: gh auth status --active --help exits 0, and the same command with an unknown flag exits 1.
  • Full suite under python -m unittest discover -s tests: 2909 tests pass, 7 skipped.
  • ruff format --check, ruff check, mypy, the prose gate against origin/develop, and shellcheck, shfmt and PSScriptAnalyzer via scripts/docker_lint.py all pass.

🤖 Generated with Claude Code

ptr727 and others added 3 commits October 24, 2026 20:09
The audit's gh() reads a 404 only from the "(HTTP 404)" notation gh api
prints, so a message lacking it is an unexpected failure. The skills
installer skips the marketplace add where the listing already names this
checkout and the plugin install where the listing shows it installed,
then judges each command by its exit status alone. Both setup-github
scripts probe for gh's --active flag by the exit status of a help
request rather than by the wording of its refusal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The two ssh-keygen -F checks in setup-github.ps1 now read its exit
status, as the Linux script already does, rather than matching "found"
in its output. The skills_install.py README entry names the add and
install skips the listing now decides.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A stub ssh-keygen answering exit 0 reads as known, and one exiting
non-zero with "found" in its output reads as not known, which the
former match on that word got wrong.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings October 25, 2026 03:18
@ptr727

ptr727 commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head 2ba49a740cd684cc64ebbbae7fe2570c4bb67f28, the content this pull request carries at that commit against develop, and it recorded 1 finding.

@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Both pwsh harnesses parsed setup-github.ps1 and dot-sourced a list of
its functions with the same lines. One builder now writes that
preamble, and each harness supplies only its parameters, its function
list, and its body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ptr727

ptr727 commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head a10982dc06d51a23b6f79cb75a76dccf6bd7c1ca, the content this pull request carries at that commit against develop, and it recorded 0 findings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An unresolved moderate test-coverage finding remains, along with two maintainability findings.

2 open issues
What changed in this PR

This pull request replaces subprocess failure handling based on message prose with exit-status checks, structured HTTP status parsing, and explicit listing operations across the audit, installer, and GitHub host setup tools.

Changes:

  • Classifies gh api absence only from the (HTTP 404) status notation.
  • Uses marketplace and plugin listings to determine whether Claude registration and installation are needed.
  • Replaces prose-based capability and known-host checks with exit-status checks on Linux and Windows, with updated tests and documentation.
File Description Review comments
tests/​test_skills_install.py Tests listing-based skips and status-based installation failures. —
tests/​test_setup_github.py Tests Linux and Windows capability probes and Windows known-host behavior. Nit (P2, 4 votes): Share the duplicated PowerShell harness preamble.
tests/​test_audit_helpers.py Tests structured 404 handling and unexpected failure cases. —
spec/​audit.py Reads marked HTTP status codes when classifying gh api failures. Nit (P2, 4 votes): Reuse a shared HTTP-status parser with the existing implementation.
scripts/​skills_install.py Uses marketplace and plugin listings before running Claude commands. —
scripts/​README.md Documents listing-based skips and status-based failure handling. —
host-setup/​windows/​setup-github.ps1 Uses exit statuses for gh --active and known-host checks. Moderate (2 votes): Add coverage for the Show-Status path with a nonzero ssh-keygen status and found output.
host-setup/​linux/​setup-github.sh Probes gh --active support using exit status. —

Review effort: Lite

Comment thread host-setup/windows/setup-github.ps1 Resolved
Comment thread spec/audit.py Resolved
@codecov

codecov Bot commented Oct 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (develop@636ddff). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #3455   +/-   ##
==========================================
  Coverage           ?   80.89%           
==========================================
  Files              ?       23           
  Lines              ?     8845           
  Branches           ?        0           
==========================================
  Hits               ?     7155           
  Misses             ?     1690           
  Partials           ?        0           
Flag Coverage Δ
python-3.13 80.89% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add-KnownHost and Show-Status each ran ssh-keygen -F and read its exit
status on their own, so only the first was under test. Both now call
Test-GitHubKnownHost, which the known-host tests exercise directly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ptr727

ptr727 commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head 85f0ce608c5aad2552c848a3dfcdd8ed7b27cad6, the content this pull request carries at that commit against develop, and it recorded 0 findings.

@ptr727

ptr727 commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

Answering the Copilot round's threadless nit on tests/test_setup_github.py, 'Share the duplicated PowerShell harness preamble': fixed in a10982d. One windows_harness(params, functions, body) builder now writes the parse-and-dot-source preamble for both the GitHub CLI harness and the known-host harness. Copies of the same pattern in tests/test_install_tools.py and tests/test_bootstrap.py are in files this change does not touch, so they appear under Dropped findings in the body.

@ptr727
ptr727 merged commit c68367a into develop Oct 25, 2026
13 checks passed
@ptr727
ptr727 deleted the feature/auto-3378 branch October 25, 2026 04:06
ptr727 added a commit that referenced this pull request Oct 25, 2026
…nd Nine Fixes to Main (#3472)

## Summary

Promotes the pull requests merged into develop since release 2.0.1166.

- [#3444](#3444): keeps
untracked files out of the pre-commit snippet's prose gate.
- [#3446](#3446): exempts
the BuildKit `check=` parser directive in the prose gate.
- [#3448](#3448): compares
intent staleness against the hub's own copy and names the moved
canonical.
- [#3450](#3450): says
only the prose gate is diff-scoped in the commit hook.
- [#3453](#3453):
reclassifies HolidayLights as an operational config repository in the
registry.
- [#3455](#3455):
classifies the remaining subprocess failures by status rather than by
prose.
- [#3464](#3464): splits
the semicolon series in WORKFLOW.md's filesystem output-seam bullet.
- [#3466](#3466): audits a
caller's `workflow-model` input against the registry.
- [#3469](#3469): emits
one snapshot shape from the skills report's cannot-read paths.
- [#3462](#3462): splits
`pr_review.py`'s pure check and round readers into the leaf modules
`pr_review_checks.py` and `pr_review_rounds.py`.
- [#3470](#3470): holds a
covered held `wait` while any workflow run on the head has not
completed.
- [#3473](#3473): answers
this promotion's review. The `wait` help and `scripts/README.md` now say
what a held head's exit `0` does not prove: a `CHECKS UNREADABLE`
payload, the conditions the unregistered-required-check hold reads, and
the reach of the completed-run read. The exit code itself stays for
#3471.

Closes #3370
Closes #3372
Closes #3371
Closes #3369
Closes #3378
Closes #3454
Closes #3332
Closes #1759
Closes #3451

## Constituent coverage

Each change this promotion carries had coverage on its own pull request
into develop, at its merged head:

| Pull request | Merged head | Coverage |
| --- | --- | --- |
| [#3444](#3444) |
`a61d1e2a` | attested local pass at the merged head, base `develop` |
| [#3446](#3446) |
`0a0efea1` | attested local pass at the merged head, base `develop` |
| [#3448](#3448) |
`c2c71ea7` | attested local pass at the merged head, base `develop` |
| [#3450](#3450) |
`4ec37c34` | attested local pass at the merged head, base `develop` |
| [#3453](#3453) |
`f20a543c` | a Copilot round on the merged head, its one changed file,
no open findings |
| [#3455](#3455) |
`85f0ce60` | attested local pass at the merged head, base `develop` |
| [#3464](#3464) |
`fcd48e2d` | attested local pass at the merged head, base `develop` |
| [#3466](#3466) |
`3426a19c` | attested local pass at the merged head, base `develop` |
| [#3469](#3469) |
`ea4b0bb7` | attested local pass at the merged head, base `develop` |
| [#3462](#3462) |
`236eceab` | attested local pass at the merged head, base `develop` |
| [#3470](#3470) |
`3545ba46` | attested local pass at the merged head, base `develop` |
| [#3473](#3473) |
`46010530` | attested local pass at the merged head, base `develop`, and
a Copilot table on an earlier head naming exactly the changed files |

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants