Skip to content

Promote the Prose Gate's Digit, Second, and Non-Shell Heredoc and Backtick Fixes to Main - #3438

Merged
ptr727 merged 3 commits into
mainfrom
develop
Oct 25, 2026
Merged

ptr727 merged 3 commits into
mainfrom
develop

Conversation

@ptr727

@ptr727 ptr727 commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

Summary

Promotes the pull requests merged into develop since release 2.0.1163.

  • #3437: the prose gate reads a heredoc label opening on a digit where its terminator line follows, queues a second heredoc on one opener line, skips the run-block heredoc split for a step whose shell: is neither bash nor sh, and numbers inline Markdown comments with escapes, HTML comments, and tags taking their backticks as CommonMark reads them, using markdown-it-py's own tag grammar. The gate's size ceiling moves from 1840 to 1862 lines, as the maintainer decided this round.
  • #3439: fixes the two findings this promotion's review raised. Each of two heredoc bodies on one opener line is lexed alone, and a URI autolink keeps its backticks out of the code-span mask, using markdown-it-py's own autolink character classes. Three walrus rewrites hold the gate at the 1862-line ceiling.
  • #3440: rewords the comment above HEREDOC to say the last, unclosed branch is the one that drops a digit label, after this promotion's review read that branch as the second-opener conditional's else branch a second time.

Closes #3428
Closes #3429
Closes #3430
Closes #3435

Constituent coverage

Each change this promotion carries had coverage on its own pull request into develop, at its merged head:

Pull request Merged head Coverage
#3437 2e271e22 attested local pass, and a Copilot table naming exactly the changed files, coverage=table
#3440 7bbd7780 attested local pass, and a Copilot table naming exactly the changed file, coverage=table
#3439 a53ee15a attested local pass, and a Copilot table naming exactly the changed files, coverage=table

🤖 Generated with Claude Code

…nd Pair Backticks as CommonMark Does (#3437)

Closes four prose-gate misreads in
`.github/actions/prose-gate/prose_lint.py`, each with its issue's
constructed case as a test that fails before and passes after.

- **A heredoc label opening on a digit** (`<<123`) is now read, but only
where its terminator line follows. Bash reads `<<2` as a shift inside
`(( ))`, `$[ ]`, and an array subscript, so accepting any `\w` label, as
the issue suggested, would have opened a false heredoc at every such
shift, the existing `(( x << 2 ))` test among them. Bash does open a
heredoc on `let x=1<<2`. This rule reads it as one only where a `2` line
follows, which is the case the issue asked to keep closed.
- **A second heredoc on one opener line** (`<<A 3<<B`) queues its body
after the first's terminator. A `<<word` after a quote or a `#` on that
line is not read as a second opener, so a `<<EOF` inside a string cannot
run the body on to a later heredoc's `EOF`.
- **A `run:` step whose `shell:` is neither bash nor sh** is no longer
split at its heredocs. The new `step_shell` reads the step's own
`shell:` key, past a comment line, and takes the program by its base
name, as `repo_gate.py`'s `step_dialect` does. A job-level or
workflow-level `defaults: run: shell:` is not read, so such a step is
split as it was on the base.
- **A stray backtick** (escaped, inside an HTML comment, or in a tag
attribute) no longer pairs with a later one in the inline comment
search. An escape, a comment, or a tag opening first keeps its
backticks, which is CommonMark's leftmost precedence. The tag grammar
comes from markdown-it-py's own `HTML_TAG_RE`, the parser this search
lines up with. `mask_code_and_quotes` keeps `CODE_SPAN` unchanged, since
it reads code comments rather than Markdown.

Raises the gate's size ceiling from 1840 to 1862 lines on the
maintainer's decision this round. The growth is parser logic rather than
vocabulary, and a scan for compaction found only cosmetic rewrites in
unrelated functions.

Closes on promotion: #3430
Closes on promotion: #3435
Closes on promotion: #3428
Closes on promotion: #3429

Local review, declined with reasons:
- An escaped backtick directly followed by a code span still leaves that
span unblanked, since `CODE_SPAN`'s lookbehind sees the escaped
backtick. It needs that adjacency plus a code span holding a later
identical comment's text, and its failure is a comment numbered one line
off, so it is declined per `GOVERNANCE.md` "Trust Boundaries and
Hardening Effort".
- A flow-mapping step (`- {shell: python, run: "..."}`) is not read by
`step_shell`, and is split as on the base.
- `MARKDOWN_SPAN` relies on markdown-it-py's `HTML_TAG_RE` holding no
capturing group, so `\1` stays the backtick run. Version 4.2.0 holds
none, and a release that added one fails the existing code-span tests in
CI, which installs the floating dependency, so no guard is added.

This change writes a comment line, so it carries the `comments` label.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings October 24, 2026 22:57
@coderabbitai

coderabbitai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Resolve the multiple-heredoc lexer-span issue and preserve autolink contents when masking Markdown code spans.

2 open issues
What changed in this PR

This pull request promotes prose-gate parser fixes from develop to main, covering heredoc detection, shell-aware YAML handling, and Markdown inline-comment positioning.

Changes:

  • Supports digit-labelled and multiple heredocs.
  • Skips heredoc splitting for non-Bash and non-sh steps.
  • Improves Markdown-aware masking for escapes, HTML, comments, and code spans.
  • Adds regression coverage and raises the size ceiling to 1862 lines.
File Summary
tests/​test_prose_lint.py Adds regression tests for heredoc, shell-step, and Markdown parsing behavior and updates the size ceiling.
.github/​actions/​prose-gate/​prose_lint.py Updates heredoc parsing, shell detection, and Markdown span masking; unresolved issues remain with separate lexer spans for multiple heredoc bodies and CommonMark autolinks.

Review effort: Lite

@codecov

codecov Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.76%. Comparing base (4c77db2) to head (687ccd9).
⚠️ Report is 418 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3438   +/-   ##
=======================================
  Coverage   80.76%   80.76%           
=======================================
  Files          23       23           
  Lines        8839     8839           
=======================================
  Hits         7139     7139           
  Misses       1700     1700           
Flag Coverage Δ
python-3.13 80.76% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…de-Span Mask (#3439)

Fixes the two findings the promotion review raised on #3438, each with a
test that fails before and passes after.

- **Each of two heredoc bodies on one opener line is lexed alone.**
`split_heredocs` took one span per heredoc token, and since #3437 a
token can hold two bodies (`<<A 3<<B`), so an apostrophe in the first
body could pair with a quote in the second and hide the second body's
comment. An empty group after the first terminator, numbered after every
existing group so no backreference moves, marks where the split cuts the
token.
- **A URI autolink keeps its backticks out of the code-span mask.**
#3429 named autolinks among the stray-backtick forms, and #3437 left
them out. The alternative copies markdown-it-py's `AUTOLINK_RE`
character classes. markdown-it's `validateLink` rejection of protocols
such as `file:` is a call rather than a pattern, so a rejected-protocol
autolink and an email autolink are not covered, both declined as needing
a backtick in the link plus two identical comments to misnumber one.
- Three single-use assignments become walrus conditions, which holds the
gate at its existing 1862-line ceiling.

Refs #3428, #3429

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The reviewed parser still has one critical numeric-heredoc defect and two moderate heredoc/shell-detection issues.

1 open issue
2 resolved since last review

Review effort: Lite

Rewords the comment above the prose gate's `HEREDOC` rule to say which
branch drops a heredoc label opening on a digit: the last one, for a
body that never closes.

The hosted reviewer twice read that final `(?(3).*|(?!))` as the else
branch of the second-opener conditional `(?(7)...)` and concluded that a
terminated `<<123` never matches, on #3437 and on the promotion #3438.
It is the outer group's second alternative, the regex does match a
terminated digit label, and
`test_a_delimiter_opening_on_a_digit_ends_on_its_terminator` passes.
Since the same misreading came back against correct code, the comment
now states the structure rather than leaving it to be re-derived. No
code changes, and the gate stays at 1862 lines.

This change edits a comment line, so it carries the `comments` label.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Shell parsing still mishandles unrecognized dynamic shells and executable-qualified shell tokens, risking incorrect heredoc classification.

No open issues 🎉

1 resolved since last review

Review effort: Lite

@ptr727

ptr727 commented Oct 25, 2026

Copy link
Copy Markdown
Owner Author

On the round's headline, which counts no finding: both cases are declined. A dynamic shell such as shell: ${{ matrix.shell }} has no value a static read can know, so step_shell returns "" and the step is split as bash, exactly as on the base and matching GitHub's default shell. A path-qualified program such as /usr/bin/python3 {0} or pwsh.exe is classified by the base name of its first word, the rule repo_gate.py's step_dialect already uses, and test_a_non_shell_run_step_takes_no_heredoc_split covers the quoted and absolute-path forms. A Windows backslash path to bash or an env bash shell reads as non-shell and skips the split. That needs such a shell, a heredoc whose body holds an apostrophe, and a later comment, and its failure is one comment left unlinted, so it is declined per GOVERNANCE.md "Trust Boundaries and Hardening Effort".

@ptr727
ptr727 merged commit 63cf1c9 into main Oct 25, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants