Skip to content

Read per-repository settings from .github/code-review/ on the default branch - #69

Merged
polewskm merged 23 commits into
mainfrom
issue-43
Oct 31, 2026
Merged

polewskm merged 23 commits into
mainfrom
issue-43

Conversation

@polewskm

@polewskm polewskm commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

gather() now reads .github/code-review/ from the base checkout only. It reads config.yml (sensitive:, skip:, policies:, tiers:), guidance.md and reviewers/*.md, so a pull request that edits them is reviewed with the default branch's copy.
Policies are added to the conventions with their path, within a 40,000-character budget of their own. Changed files that skip: leaves out are listed in the check run as "skipped by config". Project reviewers join the roster as project:<name> and never replace an engine reviewer; a paths: frontmatter adds one to the floor. Settings the engine can't use become check-run warnings, and the review runs with the defaults instead of failing.
sensitive: is stored on the gathered context for #42; it doesn't change the review yet. The trace gets a settings block. README has a new "Repository settings" section with an Identity example. Tests are in test/settings.test.ts.

Decisions

  • No new dependency. A small hand-written parser reads the documented YAML subset and refuses anchors, tags, multi-line strings and deeper nesting. A glob that starts with * must be quoted, as real YAML also requires.
  • Any schema error, such as an unknown key, a wrong type or a bad tier, discards the whole config.yml for the defaults, with one warning. Unsafe or missing policy paths are dropped one entry at a time, with a warning for each.
  • Policy paths must be relative and contain no .., backslash or control characters. They must resolve inside the real path of the base checkout and be regular files. The same check applies to config.yml, guidance.md and the reviewers folder, so a symlinked .github/code-review isn't followed.
  • Precedence for tiers:: the engine defaults, then the repository's tiers:, then CODE_REVIEW_TIERS.
  • Limits: at most 10 project reviewers; names must be lowercase kebab case; each settings file is read up to 100,000 characters.
  • Skipped files stay out of the diff and the floor. Reviewers still see them in the file list, tagged as skipped by config.

Closes #43

This branch now also carries #78 (deep reviews for sensitive paths and a light path for documentation-only pull requests), which was merged into it.

Closes #42

… branch

gather() reads config.yml (sensitive, skip, policies, tiers), guidance.md and reviewers/*.md from the base checkout only, never the pull request's head. config.yml is a small YAML subset checked by a hand-written schema; anything unusable is a warning in the check run and the defaults, not a failed run. Policies join the conventions within their own budget and must resolve inside the checkout; skipped files are listed in the check run; project reviewers join the roster as project:<name> and their paths: frontmatter adds them to the floor.
@polewskm
polewskm deployed to code-review October 31, 2026 04:17 — with GitHub Actions Active
Comment thread src/settings.ts Fixed

@ncode-review ncode-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 NCode Review at a589e4c: 16 new findings on the changed lines.

Comment thread src/settings.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread src/results.ts Resolved
Comment thread src/gather.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread CHANGELOG.md Resolved
Comment thread src/settings.ts Resolved
@ncode-review

ncode-review Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

🤖 Review complete

Verdict: 💬 Comment — This PR reads per-repository review settings (config.yml, guidance.md, reviewers/*.md) from the default branch's checkout under .github/code-review/. They feed skip, sensitive paths, policies, model tiers and project reviewers into gather and review. Reading only from the base checkout, confining reads to that checkout after resolving symlinks, and degrading to a warning when a setting fails are all sound choices, and the tests cover them well. The main design cost is a hand-written YAML subset parser that sits beside the existing lenient frontmatter reader. Everything else raised is cleanup: duplicated checks, unnamed numbers and test structure.

7 reviewers run — 7 findings (run details)

Reviewer Model Findings Top
synthesis opus-5-5 7 🟡
architecture sonnet-5-5 1 🟡
completeness sonnet-5-5 0 🟢
conventions sonnet-5-5 0 🟢
correctness sonnet-5-5 0 🟢
hygiene haiku-5-5 3 🟡
rule-check sonnet-5-5 5 🟠
security sonnet-5-5 0 🟢

Skipped: api-compat, data-model, infrastructure, intent, performance, tests

Findings
  • 🟡 F1 src/settings.ts:262: Should a rejected settings entry drop every configured control without failing?
  • 🟡 F2 src/settings.ts:21: Hand-written YAML subset parser sits beside the lenient frontmatter reader
  • 🟡 F3 src/settings.ts:371: Path-item validation is written twice
  • 🟡 F4 src/review.ts:186: Reviewed-file filter repeated where gather already computes it
  • 🟡 F5 src/results.ts:148: Unnamed display limits repeated as literals
  • 🟡 F6 src/gather.ts:263: ruleFile takes five parameters
  • 🟡 F7 test/settings.test.ts:367: Tests: Results fixtures are cast, and one gather test covers every setting

Trial: findings are posted as review threads and in the check run. $0.81 on Amazon Bedrock, 88s, at 45c6c29. (how to interact)

Settings reads go through one open file and turn any fs error into a warning, so an unreadable file can't fail the review or change between its checks and its read. A project reviewer's frontmatter is validated (paths: only, a list of globs or one comma-separated string), shares the frontmatter split and glob list with the instruction files, and its warning loses a stray colon. parseYaml refuses a __proto__ key. A tiers: name matching no reviewer is a check-run warning, and a pull request whose every changed file is skipped by config runs no model. Drop the ?? fallbacks on fields gather always sets, cite issue #42 as the convention asks, and test the size limits, truncation, unreadable files, a reviewers path that isn't a folder, projectRoster and fileList.
@polewskm

Copy link
Copy Markdown
Contributor Author

/review

@ncode-review ncode-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 NCode Review at dc5a450: 9 new findings on the changed lines.
5 more were already posted on this pull request.

Not on a changed line:

  • 🟡 F4 Nit: Is haiku the intended default tier for project reviewers? src/review.ts:392. Project reviewers aren't in defaultReviewerTiers, so tiers[reviewer] ?? 'haiku' runs every project:&lt;name> on the cheapest model. Every engine reviewer defaults to sonnet, and neither the README nor the CHANGELOG says this. Fix: Default project reviewers to sonnet, or document the haiku default in the README's tiers: bullet.

Comment thread src/review.ts Resolved
Comment thread src/gather.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread src/gather.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread test/settings.test.ts Resolved
Comment thread src/review.ts Resolved
@polewskm

Copy link
Copy Markdown
Contributor Author

Autopilot: answering F4 in #69 (review) (also listed in #69 (comment)): fixed in 45c6c29. Haiku is the intended default. It is the engine's default for any reviewer not in defaultReviewerTiers (hygiene and tests run on it too), so not every engine reviewer defaults to sonnet. The README's tiers: bullet now says a project reviewer runs on haiku unless tiers: names it.

@polewskm

Copy link
Copy Markdown
Contributor Author

Autopilot: Round 2 answered. No blocker or should-fix is left open; the remaining threads are nits, answered above. Recommend merging once you've resolved them.

@polewskm
polewskm marked this pull request as ready for review October 31, 2026 06:39
@polewskm
polewskm deployed to code-review October 31, 2026 06:39 — with GitHub Actions Active
@polewskm
polewskm enabled auto-merge October 31, 2026 06:39

@ncode-review ncode-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 NCode Review at 45c6c29: 4 new findings on the changed lines.
2 more were already posted on this pull request.

Not on a changed line:

  • 🟡 F4 Nit: Reviewed-file filter repeated where gather already computes it src/review.ts:186. projectFloor recomputes the non-generated paths with files.filter((f) => !f.generated).map(...), the same list gather.ts already computes as reviewed. Fix: Expose the reviewed paths on Gathered, or share one helper for the filter.

Comment thread src/settings.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread src/settings.ts Resolved
Comment thread test/settings.test.ts Resolved
@polewskm

Copy link
Copy Markdown
Contributor Author

Autopilot: answering F4 in #69 (review) (also listed in #69 (comment)): declined: nit; the review-round limit (3) is reached. projectFloor takes the changed files as its input so it stays a pure function with its own test, and the one-line filter it repeats is the same rule gather uses; exposing the reviewed paths on Gathered could be a follow-up. F5 (unnamed display limits) and F6 (ruleFile parameters) in that summary are repeats, answered at #69 (comment) and #69 (comment).

@polewskm

Copy link
Copy Markdown
Contributor Author

Autopilot: Round 3 answered. The review-round limit (3) is reached; no blocker or should-fix is left open, and the remaining threads are nits or repeats, declined above with pointers to the earlier answers. The branch now has origin/main merged in (1a656ff: the run-ceiling work from #61, with its test fixture given the new settings fields). Recommend merging once you've resolved the threads; the YAML-library and shared path-validation ideas could be follow-up issues.

…-only pull requests

A change to a file config.yml's sensitive: matches runs correctness and security on Opus for the parts of the diff with it, and verification on Opus for its findings. A documentation-only pull request, or a template update of template-owned files, runs hygiene alone on Haiku with no context, selection or synthesis. CODE_REVIEW_TIERS still overrides any reviewer, and the check run's reviewer table shows each reviewer's models.

Closes #42
A file with a code or script extension, or a git hook, never counts as documentation or as a template-owned file a template update can change, so a pull request touching one gets the standard review even in docs/ or alongside .template-version. Also tests a deep review whose floor already chose security, and fixes a stray colon in the partTiers comment.
Deep reviews for sensitive paths and a light review for documentation-only pull requests
# Conflicts:
#	CHANGELOG.md
#	src/main.ts
#	src/results.ts
#	src/review.ts
#	src/verify.ts
#	test/verify.test.ts
The depth tests' fake agent returns records, and the fallback test reviews a non-documentation file so its review isn't a light one. A reviewer's models leave out a call refused for credits and name a model once across providers.
@polewskm

Copy link
Copy Markdown
Contributor Author

Autopilot: merged origin/main (#63 suggestions, #64 Bedrock rerun) into this branch in 9b9d410, with follow-up 7078ba1.

  • Retry on Bedrock when the API credits run out in the middle of a review #64 changed AgentResult.record to records: CallRecord[]. This PR's context, selection, synthesis and deep-verification calls now push ...records and go through withoutTools(run). A deep verification call that throws names Bedrock once the run has switched, still on Opus.
  • Reviewer models: Retry on Bedrock when the API credits run out in the middle of a review #64 showed the model of a reviewer's last call (lastModel), but this PR lists every model a reviewer ran on (callModels), because a deep review runs some parts on Opus. The merge keeps callModels and changes it so it leaves out a call the Claude API refused for credits. shortModel now names each model once, so a reviewer that ran on both providers shows opus-5-5, not opus-5-5, opus-5-5. Synthesis still uses lastModel.
  • test/fallback.test.ts reviewed a README-only typo fix. With this PR, that is a light review with no selection or synthesis call, so the fixture now changes config/greeting.json instead. The test's assertions are unchanged.
  • npm run typecheck and npm test pass (245 passed, 1 skipped).

# Conflicts:
#	README.md
#	src/context.ts
#	src/gather.ts
#	src/main.ts
#	src/results.ts
#	src/review.ts
#	test/verify.test.ts
… claims guard added, and test the light review's guard and a re-review's tiers
@polewskm

Copy link
Copy Markdown
Contributor Author

Autopilot: merged main (#68 guidance scoping, #71 changed-file list and claims guard, #77 re-reviews) into this branch in f6a60a2, with test fixes in 27a2ae3. What changed:

  • reviewJobs now takes the reviewers and models per part, so a deep review's Opus correctness/security on sensitive parts still works with main's per-part rule scoping. fileList keeps main's "diff in another part" marks and still lists the files skip: leaves out.
  • The light (documentation-only) review still skips synthesis, but now applies main's claims guard. A hygiene finding that says a changed file isn't changed is dropped and listed, as in every other review. README says so.
  • guidance.md's applyTo: and the instruction files share main's applyToGlobs/matchesAnyFile. The learned guidance keeps main's per-bullet scoping.
  • A re-review (Re-review only what changed since the last review, and say when it's done #77) runs correctness alone and is never deep or light. It now takes the model from the repository's tiers: as well as CODE_REVIEW_TIERS. Its results report depth: standard and no sensitive files. README's Re-reviews section says this. A re-review of a sensitive path therefore doesn't use Opus. That is worth a follow-up if you want it to.

# Conflicts:
#	CHANGELOG.md
#	src/results.ts
#	src/review.ts
…les, so the acknowledgement and finish scripts load no npm package

This branch's results.ts imported depthLabel from review.ts and creditsRanOutReason from provider.ts as values, which since main's acknowledgement (#74) made acknowledge.ts load the agent SDK through comments.ts and results.ts.
# Conflicts:
#	CHANGELOG.md
#	src/config.ts
#	src/gather.ts
#	src/main.ts
#	src/results.ts
#	src/review.ts
#	test/rereview.test.ts
@polewskm

Copy link
Copy Markdown
Contributor Author

Autopilot: merged origin/main (#70 template ownership, #75 explain/teach, #76 shadow mode) into this branch in beaa856. A few parts of this PR changed to fit main:

  • Project charters and shadow mode. Charters.reviewers is now Map<string, Charter>. readReviewers reads each project charter's text and shadow flag with the shared parseCharter, so shadow: true in a .github/code-review/reviewers/*.md frontmatter makes project:<name> a shadow reviewer. charterPaths now accepts shadow: (true or false, anything else is a warning). The trace's settings.projectReviewers notes shadow. The README and tests cover it.
  • Results. The results.json fields (skippedByConfig, warnings, depth, sensitiveFiles, and per-reviewer models from callModels) are now built in main's buildResults (src/review-results.ts). The check run's reviewer table is shared with the shadow section, so the shadow table also has the Model column.
  • Template ownership. Main's templateOf sees every changed file, including ones skip: leaves out, so skipping .copier-answers.yml doesn't hide an adoption or update. Template-owned files no longer add project reviewers through paths:, the same as main's selectionFloor. Repository and project guidance rules now apply only to the project's own files. On a light review, hygiene's findings on template-owned files below blocker go to the upstream list too.
  • Overlap to check. documentationOnly (light review) still uses this PR's own list of files a template update owns. Main's src/template.ts has the full list, but it only applies to writers' adoptions or updates of NCodeGroup/project-template. I kept both as they were.

npm run typecheck and npm test pass: 489 tests, 488 passed, 1 skipped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants