Skip to content

docs(github-project): read effective branch rules, not just classic protection - #156

Merged
CybotTM merged 1 commit into
mainfrom
docs/effective-branch-rules
Aug 10, 2026
Merged

docs(github-project): read effective branch rules, not just classic protection#156
CybotTM merged 1 commit into
mainfrom
docs/effective-branch-rules

Conversation

@CybotTM

@CybotTM CybotTM commented Aug 10, 2026

Copy link
Copy Markdown
Member

The defect

SKILL.md's "Security & Compliance Quick Checks" and checkpoint GH-31 both read branch rules from gh api repos/OWNER/REPO/branches/main/protection. That endpoint is blind to rulesets, and its two failure shapes are asymmetric — one looks like an absence, the other looks like an answer:

Classic endpoint says Reads as Reality on the same repo
required_status_checks: null nothing is required 23 required contexts
required_approving_review_count: 0 no review required 1 required approval

Measured on netresearch/t3x-nr-llm, where classic protection and two active rulesets coexist. Acting on the classic reading alone, I wrote into that repository's OpenSSF compliance document that it required neither an approving review nor any status check. Both statements were false, and nothing in the API response hints that a second source exists.

Changes

references/security-config.md gains an "Effective Branch Rules" section. Read both sources; GitHub composes them and applies the most restrictive result. Bypass actors live on the ruleset, not on the branch, and the REST API returns actor_id as an unresolved number — so "repository role id 5", not a guessed "admin".

GH-31 becomes a command checkpoint. It passes when either classic protection or a pull_request ruleset requires thread resolution. As a gh_api check against the classic endpoint it fails a correctly configured ruleset-only repository at severity error — a false negative on the checkpoint that exists to catch unresolved bot findings.

GH-30 keeps its classic read (enforce_admins is genuinely a classic field) but its description now says so and points at the ruleset's own bypass_actors.

GH-32's llm_review step 2 checked the same field from the same blind endpoint; it now checks both.

Two audit recipes, same origin

  • A required context that is skipped on every commit enforces nothing. Observed: fuzz / Fuzz Tests was required and always skipped, because the job producing it passed no inputs to its reusable workflow. The job that actually ran the suite was a different context and was not required at all.
  • Check-run names are prefixed by the calling JOB name, not the workflow. Two jobs on the same reusable produce fuzz / X and fuzz-mutation / X — different contexts that look interchangeable. The section gives the run→jobs loop to measure it instead of inferring.

A note on the word budget

SKILL.md is at 499 of its 500-word limit on main, so the file cannot accept a net addition — my first push failed Skill Validation at 593 words.

This PR therefore adds nothing to it. The quick-check gains the rules/branches call and loses the bypass_pull_request_allowances call, which reads the same blind endpoint and whose correct replacement (ruleset bypass_actors) is documented in the reference. Net 496 words.

That budget is also why the material went into the existing security-config.md instead of a new reference file: a new file needs a new pointer row in the table, and there is no room for one. Worth knowing as a maintainer — at 499/500 the next contributor hits the same wall, and the only ways through are trimming existing prose or raising the cap.

Verification

OK: SKILL.md is 496 words (under 500 limit)
Errors:   0
  • validate-skill.sh (the script CI runs) reports 0 errors locally.
  • yaml.safe_load parses checkpoints.yaml; 29 mechanical checkpoints.
  • GH-31's target passes bash -n, and both of its branches evaluated against a live repository (classic=true ruleset=true).

The 10 warnings the validator reports (README template sections, untested scripts, runnable commands in llm_reviews) are pre-existing on main and untouched here.

One limit worth stating: I have no ruleset-only repository to demonstrate the old checkpoint's false negative end to end. The blindness itself is shown by the table above — the classic endpoint reporting 0 while a ruleset requires 1 on the same branch.

Copilot AI lite review requested due to automatic review settings August 10, 2026 17:45
@github-actions

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added documentation Improvements or additions to documentation skill labels Aug 10, 2026
@CybotTM
CybotTM force-pushed the docs/effective-branch-rules branch from 88b5230 to 9221e73 Compare August 10, 2026 20:30
@CybotTM
CybotTM requested a lite review from Copilot August 10, 2026 20:34
…rotection

The skill taught `gh api repos/OWNER/REPO/branches/main/protection` for its
security quick-check, and GH-31 asserted on that endpoint. It is blind to
rulesets, and the two ways it fails are asymmetric:

  required_status_checks: null              -> reads as "nothing is required"
  required_approving_review_count: 0        -> reads as a real answer

Measured on a repository where both sources are configured: the classic
endpoint reports 0 approvals and no required checks while active rulesets
require 1 approval and 23 status contexts. A compliance document was written
off that reading and attested the repository required neither review nor
checks. Nothing in the response hints at it.

- security-config.md gains an "Effective Branch Rules" section: read both
  sources and take the more restrictive result; bypass actors live on the
  ruleset, not on the branch, and the REST API returns an unresolved numeric
  actor_id.
- GH-31 becomes a `command` checkpoint that passes when EITHER classic
  protection OR a `pull_request` ruleset requires thread resolution. As a
  gh_api check against the classic endpoint it failed a correctly configured
  ruleset-only repository at severity error.
- GH-30's description says its enforce_admins read covers classic protection
  only, and points at the ruleset bypass_actors list.
- The GH-32 llm_review step that checked the same field now checks both.

Two required-check audit recipes found the same way are in the same section: a
required context that is `skipped` on every commit enforces nothing, and
check-run names are prefixed by the calling JOB name, so two jobs on the same
reusable workflow produce two different contexts.

SKILL.md sits at 499 of its 500-word budget on main, so this adds nothing to
it: the quick-check gains the `rules/branches` call and loses the
`bypass_pull_request_allowances` call, which reads the same blind endpoint and
whose replacement is the ruleset bypass_actors list documented in the
reference. 496 words. That budget is also why the material went into the
existing security-config.md rather than a new reference — a new file needs a
new pointer row, and there is no room for one.

Verified: validate-skill.sh reports 0 errors, YAML parses (29 mechanical
checkpoints), GH-31's shell passes `bash -n`, and both of its branches evaluate
against a live repository.

Signed-off-by: Sebastian Mendel <sebastian.mendel@netresearch.de>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@sonarqubecloud

Copy link
Copy Markdown

@CybotTM
CybotTM merged commit 9525a7b into main Aug 10, 2026
20 checks passed
@CybotTM
CybotTM deleted the docs/effective-branch-rules branch August 10, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants