Skip to content

Read Digit-Labelled and Second Heredocs, Skip Non-Shell Run Splits, and Pair Backticks as CommonMark Does - #3437

Merged
ptr727 merged 2 commits into
developfrom
feature/prose-gate-heredoc-followups
Oct 24, 2026
Merged

ptr727 merged 2 commits into
developfrom
feature/prose-gate-heredoc-followups

Conversation

@ptr727

@ptr727 ptr727 commented Oct 24, 2026

Copy link
Copy Markdown
Owner

Closes four prose-gate misreads in .github/actions/prose-gate/prose_lint.py, each with its issue's constructed case as a test that fails before and passes after.

  • A heredoc label opening on a digit (<<123) is now read, but only where its terminator line follows. Bash reads <<2 as a shift inside (( )), $[ ], and an array subscript, so accepting any \w label, as the issue suggested, would have opened a false heredoc at every such shift, the existing (( x << 2 )) test among them. Bash does open a heredoc on let x=1<<2. This rule reads it as one only where a 2 line follows, which is the case the issue asked to keep closed.
  • A second heredoc on one opener line (<<A 3<<B) queues its body after the first's terminator. A <<word after a quote or a # on that line is not read as a second opener, so a <<EOF inside a string cannot run the body on to a later heredoc's EOF.
  • A run: step whose shell: is neither bash nor sh is no longer split at its heredocs. The new step_shell reads the step's own shell: key, past a comment line, and takes the program by its base name, as repo_gate.py's step_dialect does. A job-level or workflow-level defaults: run: shell: is not read, so such a step is split as it was on the base.
  • A stray backtick (escaped, inside an HTML comment, or in a tag attribute) no longer pairs with a later one in the inline comment search. An escape, a comment, or a tag opening first keeps its backticks, which is CommonMark's leftmost precedence. The tag grammar comes from markdown-it-py's own HTML_TAG_RE, the parser this search lines up with. mask_code_and_quotes keeps CODE_SPAN unchanged, since it reads code comments rather than Markdown.

Raises the gate's size ceiling from 1840 to 1862 lines on the maintainer's decision this round. The growth is parser logic rather than vocabulary, and a scan for compaction found only cosmetic rewrites in unrelated functions.

Closes on promotion: #3430
Closes on promotion: #3435
Closes on promotion: #3428
Closes on promotion: #3429

Local review, declined with reasons:

  • An escaped backtick directly followed by a code span still leaves that span unblanked, since CODE_SPAN's lookbehind sees the escaped backtick. It needs that adjacency plus a code span holding a later identical comment's text, and its failure is a comment numbered one line off, so it is declined per GOVERNANCE.md "Trust Boundaries and Hardening Effort".
  • A flow-mapping step (- {shell: python, run: "..."}) is not read by step_shell, and is split as on the base.
  • MARKDOWN_SPAN relies on markdown-it-py's HTML_TAG_RE holding no capturing group, so \1 stays the backtick run. Version 4.2.0 holds none, and a release that added one fails the existing code-span tests in CI, which installs the floating dependency, so no guard is added.

This change writes a comment line, so it carries the comments label.

🤖 Generated with Claude Code

ptr727 and others added 2 commits October 24, 2026 15:32
…nd Pair Backticks as CommonMark Does

A heredoc label opening on a digit now opens a body only where its terminator line follows, since
bash reads `<<2` in arithmetic as a shift. A second opener on one line queues its body after the
first's terminator. A `run:` step whose `shell:` is neither bash nor sh is no longer split at its
heredocs. The inline Markdown comment search lets an escape, an HTML comment, or a tag opening first
keep its backtick, so a stray one no longer pairs with a later one. Raises the gate's size ceiling
from 1840 to 1862 lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…c Past a Quote

A second heredoc opener after a quote or a `#` on the opener line no longer queues a body, so a
`<<EOF` inside a string cannot run the body on to a later heredoc's terminator. `step_shell` reads
a step's mapping past a comment line, takes a quoted or path-qualified program by its base name as
`repo_gate.py`'s `step_dialect` does, and reuses the caller's line index. The inline comment mask
takes its tag grammar from markdown-it-py's own `HTML_TAG_RE` rather than a hand-rolled copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ptr727 ptr727 added the comments Permits the comment lines the pull request adds or edits, which the prose gate otherwise refuses label Oct 24, 2026
Copilot AI lite review requested due to automatic review settings October 24, 2026 22:46
@ptr727

ptr727 commented Oct 24, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head 2e271e22cf5a353274bb777a4fc241dd8f5ed638, the content this pull request carries at that commit against develop, and it recorded 1 finding.

@coderabbitai

coderabbitai Bot commented Oct 24, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two moderate issues remain: shell lookup can cross YAML step boundaries, and digit-labelled single heredocs are not recognized.

1 open issue
What changed in this PR

This pull request fixes prose-gate parsing issues involving Bash heredocs, non-shell workflow steps, and CommonMark-style inline code handling, with regression tests for the affected cases.

Changes:

  • Supports digit-labelled and multiple heredocs while avoiding shift-expression false positives.
  • Skips heredoc splitting for non-Bash and non-sh-compatible run: steps.
  • Applies CommonMark-style precedence for escaped backticks, HTML tags, comments, and code spans.
  • Raises the prose-gate size ceiling and adds regression coverage.
File Description
.github/​actions/​prose-gate/​prose_lint.py Extends heredoc parsing, workflow shell detection, and Markdown span masking.
tests/​test_prose_lint.py Adds regression tests for the new parsing behaviors and updates the size ceiling.

Review effort: Lite

@codecov

codecov Bot commented Oct 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.76%. Comparing base (4c77db2) to head (2e271e2).

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #3437   +/-   ##
========================================
  Coverage    80.76%   80.76%           
========================================
  Files           23       23           
  Lines         8839     8839           
========================================
  Hits          7139     7139           
  Misses        1700     1700           
Flag Coverage Δ
python-3.13 80.76% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ptr727
ptr727 merged commit 33d4763 into develop Oct 24, 2026
14 checks passed
@ptr727
ptr727 deleted the feature/prose-gate-heredoc-followups branch October 24, 2026 22:57
ptr727 added a commit that referenced this pull request Oct 24, 2026
…de-Span Mask (#3439)

Fixes the two findings the promotion review raised on #3438, each with a
test that fails before and passes after.

- **Each of two heredoc bodies on one opener line is lexed alone.**
`split_heredocs` took one span per heredoc token, and since #3437 a
token can hold two bodies (`<<A 3<<B`), so an apostrophe in the first
body could pair with a quote in the second and hide the second body's
comment. An empty group after the first terminator, numbered after every
existing group so no backreference moves, marks where the split cuts the
token.
- **A URI autolink keeps its backticks out of the code-span mask.**
#3429 named autolinks among the stray-backtick forms, and #3437 left
them out. The alternative copies markdown-it-py's `AUTOLINK_RE`
character classes. markdown-it's `validateLink` rejection of protocols
such as `file:` is a call rather than a pattern, so a rejected-protocol
autolink and an email autolink are not covered, both declined as needing
a backtick in the link plus two identical comments to misnumber one.
- Three single-use assignments become walrus conditions, which holds the
gate at its existing 1862-line ceiling.

Refs #3428, #3429

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
ptr727 added a commit that referenced this pull request Oct 25, 2026
Rewords the comment above the prose gate's `HEREDOC` rule to say which
branch drops a heredoc label opening on a digit: the last one, for a
body that never closes.

The hosted reviewer twice read that final `(?(3).*|(?!))` as the else
branch of the second-opener conditional `(?(7)...)` and concluded that a
terminated `<<123` never matches, on #3437 and on the promotion #3438.
It is the outer group's second alternative, the regex does match a
terminated digit label, and
`test_a_delimiter_opening_on_a_digit_ends_on_its_terminator` passes.
Since the same misreading came back against correct code, the comment
now states the structure rather than leaving it to be re-derived. No
code changes, and the gate stays at 1862 lines.

This change edits a comment line, so it carries the `comments` label.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
ptr727 added a commit that referenced this pull request Oct 25, 2026
…ktick Fixes to Main (#3438)

## Summary

Promotes the pull requests merged into develop since release 2.0.1163.

- [#3437](#3437): the
prose gate reads a heredoc label opening on a digit where its terminator
line follows, queues a second heredoc on one opener line, skips the
run-block heredoc split for a step whose `shell:` is neither bash nor
sh, and numbers inline Markdown comments with escapes, HTML comments,
and tags taking their backticks as CommonMark reads them, using
markdown-it-py's own tag grammar. The gate's size ceiling moves from
1840 to 1862 lines, as the maintainer decided this round.
- [#3439](#3439): fixes
the two findings this promotion's review raised. Each of two heredoc
bodies on one opener line is lexed alone, and a URI autolink keeps its
backticks out of the code-span mask, using markdown-it-py's own autolink
character classes. Three walrus rewrites hold the gate at the 1862-line
ceiling.
- [#3440](#3440): rewords
the comment above `HEREDOC` to say the last, unclosed branch is the one
that drops a digit label, after this promotion's review read that branch
as the second-opener conditional's else branch a second time.

Closes #3428
Closes #3429
Closes #3430
Closes #3435

## Constituent coverage

Each change this promotion carries had coverage on its own pull request
into develop, at its merged head:

| Pull request | Merged head | Coverage |
| --- | --- | --- |
| [#3437](#3437) |
`2e271e22` | attested local pass, and a Copilot table naming exactly the
changed files, `coverage=table` |
| [#3440](#3440) |
`7bbd7780` | attested local pass, and a Copilot table naming exactly the
changed file, `coverage=table` |
| [#3439](#3439) |
`a53ee15a` | attested local pass, and a Copilot table naming exactly the
changed files, `coverage=table` |

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comments Permits the comment lines the pull request adds or edits, which the prose gate otherwise refuses

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants