Skip to content

Promote Develop: Read validate.py's Markdown and the README Link Definitions Through markdown-it-py - #3636

Merged
ptr727 merged 2 commits into
mainfrom
develop
Oct 30, 2026
Merged

ptr727 merged 2 commits into
mainfrom
develop

Conversation

@ptr727

@ptr727 ptr727 commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

Promotes the #3426 markdown-it-py link reading from develop to main.

  • #3635 Reads spec/validate.py's link targets, sections, and headings, and the README audit's reference definitions, through spec/fidelity_text.py and markdown-it-py. A definition resolves first-wins with folded lookup, as GitHub renders it, and code is content rather than links. The validator now runs under uv run --no-project --script.
  • #3637 Reads backslash escapes in a definition's label and angle destination, so an escaped > no longer cuts a destination short and a label ending in an escaped backslash no longer raises. It answers both findings Copilot raised on this promotion.

Closes #3426

🤖 Generated with Claude Code

…arkdown-it-py (#3635)

## Summary

Moves the last two hand-written Markdown readers onto markdown-it-py,
per #3426.

- **`spec/validate.py`** reads link targets, the verbatim-section
region, and the section-declaration headings through
`spec/fidelity_text.py`. So a code span, a code block, and a fenced `##
` line are content rather than links or headings, and an autolink counts
as a link. The import stays lazy, held by a new import-probe test, so
`spec/resolve_description.py` still imports the module under the plain
`python3` that `repo-config/configure.sh` finds. The validator gains a
PEP 723 header, and CI and `OPERATIONS.md` run it under `uv run
--no-project --script`.
- **`spec/audit_markdown.py`** reads reference definitions from the
parser for the link-naming, shield, and 3rd Party Tools checks,
replacing the `_LINK_DEF` line scan. A definition-shaped line inside a
paragraph is no longer a definition. Of two definitions whose labels
fold together, the first wins, as GitHub renders it. Every reference
lookup is folded the same way, so `![x][Docs]` resolves to `[docs]`.
- **`spec/fidelity_text.py`**'s cached block parse now emits definition
tokens. It gains `link_definitions` (the token's label as written, the
destination re-read from the source lines), `link_targets` (links,
images, definitions, and duplicates, all decoded), `reference_key`, and
a `fenced_only` switch on `code_lines`.

The `comments` label is for the PEP 723 header in `spec/validate.py`.
#3634 tracks exempting such a block from `comment-added`.

## Before and After

- **`spec/validate.py`** prints the same output on the base and at every
commit: 22 cataloged repos classify cleanly.
- **The full audit sweep** (`uv run --no-project --script
spec/audit.py`, every cataloged repo) ran on the base, on the first
commit, on 97e63e9, and on 7ed8ecf, the last commit touching code the
audit reads. Every finding line matched, except where a live repository
moved between runs. Two `main` heads advanced. ESPHome-Config gained a
branch-divergence DRIFT, which the base code also reports when run
against that repository now.
- **One test fixture changed with the behavior.** `DOCKER_SHIELD` in
`tests/test_audit_helpers.py` placed `[e]: ...` directly under a
paragraph line, which GitHub renders as text. Its definition now sits
with the others.
- **New cases fail on the old code**: the paragraph-line definition, the
case-variant lookup, the autolink to the template repository, and the
fenced `## Rule`.

## Local Review

Seven recorded `local-strict-review` passes ran, the last at ff63df6
with no findings. Every `introduced` finding the earlier passes raised
was fixed on this branch:

- A `functools.cache` that a Python 3.8 `configure.sh` host lacks, and
mixed decoded and raw link targets.
- An empty `<>` destination read as `None`, a wrapped label dropped, and
a label re-parsed by regex rather than taken from the parser's
definition token.
- Duplicate definitions unread by `link_targets`, and a fenced-sample
tool test that first-wins had made unable to fail.
- A duplicated fenced-lines set, a second block parser, and docstrings
and an `OPERATIONS.md` bullet that no longer matched the code.

Declined with evidence:

- **Merging `audit_markdown.references()` into the new definition
readers.** `references()` feeds `strip_md_links`, which needs
markdown-it's env shape with normalized hrefs. The naming checks need
labels and destinations as written, and both read the same parser table.
- **Sharing one inline parser between `fidelity_text._LINKS` and
`audit_markdown._MD`.** `_MD` disables `text_join` because
`strip_md_links` rebuilds a line from its source tokens, a requirement
sharing would move into `fidelity_text`.
- **`validate.fidelity_text_module` as a copy of the `carry.py` and
`build_dist.py` helpers.** Those also put `spec/` on `sys.path`. This
one is a two-line wrapper.

Closes #3426

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings October 30, 2026 03:08
@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A moderate active issue remains because escaped characters in angle-bracket destinations can be truncated, causing valid links to be classified differently from GitHub’s rendering.

1 open issue
What changed in this PR

Promotes the Markdown parsing migration from develop to main, replacing handwritten Markdown readers with markdown-it-py for validator and README audit behavior aligned with GitHub rendering.

Changes:

  • Centralizes parsing of links, reference definitions, sections, headings, and fenced code in spec/fidelity_text.py.
  • Updates validation and README audits to use folded, first-wins reference resolution.
  • Adds parser-focused tests and updates the uv-based validator invocation in CI and documentation.

Review findings:

  • Angle-bracket destinations containing escaped > characters may be truncated.
  • Reference labels ending in an escaped backslash may cause definition parsing to fail.
File Summary
tests/​test_spec_validate.py Tests parser-backed validator behavior, carried links, fenced headings, and lazy imports.
tests/​test_fidelity_text.py Tests link targets, definitions, folded labels, headings, and fenced code handling.
tests/​test_audit_helpers.py Tests README reference resolution and first-wins behavior.
spec/​validate.py Uses shared Markdown parsing for links, sections, and headings with lazy loading.
spec/​fidelity_text.py Adds shared parser-backed link-target and reference-definition readers.
spec/​audit_markdown.py Uses parser-backed definitions and folded reference lookup for README checks.
OPERATIONS.md Documents the uv validator invocation and expanded parser responsibilities.
.github/​actions/​validate/​action.yml Sets up uv and runs the validator with its inline dependencies.

Review effort: Lite

Comment thread spec/fidelity_text.py Resolved
@codecov

codecov Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.59155% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 81.69%. Comparing base (e15559f) to head (19548f4).
⚠️ Report is 447 commits behind head on main.

Files with missing lines Patch % Lines
spec/validate.py 87.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3636      +/-   ##
==========================================
+ Coverage   81.63%   81.69%   +0.06%     
==========================================
  Files          26       26              
  Lines        9081     9111      +30     
==========================================
+ Hits         7413     7443      +30     
  Misses       1668     1668              
Flag Coverage Δ
python-3.13 81.69% <98.59%> (+0.06%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…3637)

## Summary

Fixes the finding Copilot raised on promotion #3636 against #3635's
`spec/fidelity_text.py` `_DESTINATION`, and the second one its review
body named. Both come from one root cause: the regex re-reading a
definition's as-written destination did not honor backslash escapes.

- An angle-bracket destination holding an escaped `>` was cut at it, so
`[b]: <https://example.test/a\>b>` read as `https://example.test/a\`.
- A label ending in an escaped backslash matched nothing, so `[a\\]:
one.md` raised `AttributeError` in `link_definitions`.

The regex now anchors on the label's opening `[` and honors escapes on
both sides, including a label the parser lets wrap after a backslash.
`test_an_escape_in_a_label_or_an_angle_destination_is_read_through` pins
all three shapes. The first two fail on the old regex, and the third on
this branch's first commit.

## Verification

- The full unittest suite ran 3103 tests, OK (skipped=7).
- `spec/validate.py` prints the same output as before #3635.
- Two recorded `local-strict-review` passes ran. The first raised the
wrapped-after-backslash label, fixed in 149f883. The second raised
nothing.

## Dropped findings

- `spec/fidelity_text.py` `_DESTINATION`, `pre-existing`: `[ \t>]*`
swallows a leading `>` of a bare destination, an angle destination
holding a backslash-newline falls back to `\S+`, and `\S+` stops at a
non-breaking space the parser keeps. The old regex behaved the same on
each, and none is a realistic README shape.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@ptr727

ptr727 commented Oct 30, 2026

Copy link
Copy Markdown
Owner Author

Answering the finding Copilot's review of 04bd8ae names in its body with no thread (1): #3636 (review)

  • spec/fidelity_text.py:35 _DESTINATION, "Reference labels ending in an escaped backslash may cause definition parsing to fail." Fixed in 19548f4 (Read Backslash Escapes in a Definition's Label and Angle Destination #3637). The regex now pairs each backslash with the character after it inside the label, so [a\\]: one.md matches and reads one.md rather than raising AttributeError in link_definitions. The same test pins it.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain, so the parser migration and its related validation, audit, documentation, and regression-test changes are ready for approval.

No open issues 🎉

1 resolved since last review

Review effort: Lite

@ptr727
ptr727 merged commit 5525f3f into main Oct 30, 2026
14 checks passed
ptr727 added a commit that referenced this pull request Oct 30, 2026
…Session Prompt Rules (#3667)

Promotes the changes merged into `develop` since promotion #3636 to
`main`.

- [#3640](#3640) Pins the
validator's `uvx ruff` to a Dependabot-managed manifest.
- [#3642](#3642) Counts
Copilot findings under the unresolved bullet and the file table.
- [#3644](#3644) Aligns
the split comment copies and drops the stale counts.
- [#3646](#3646) Sends a
command meant for the session's input line in a plain message, never in
a blocking prompt.
- [#3648](#3648) Opens the
attended session by cloning a hub checkout of the session's own.
- [#3650](#3650) Reads a
held package as installed in both Linux host scripts.
- [#3652](#3652) States
the PowerShell `-Help` and `-h` form in the command-line argument rules.
- [#3654](#3654) Counts
sentences in Markdown table cells for the sentence-length check.
- [#3658](#3658) Splits a
table row only at an unescaped pipe in the semicolon rule.
- [#3660](#3660) Finds
table cell boundaries before blanking code spans and quotations.
- [#3662](#3662) Names a
stale job record on a completed run in `pr_review.py`.
- [#3665](#3665) Keeps a
line holding code after a block comment's closer out of `comment-added`.
Its squash commit on `develop` carries the pull request's original
title, which names a heredoc reading. That reading was withdrawn before
the merge on decision #3666, so a workflow `run:` heredoc body still
reads as code.
- [#3669](#3669) Counts
each `[Pn]` marker in a Copilot Findings cell, so a second finding in
one cell is no longer dropped from triage. It answers one of the two
findings Copilot raised on this promotion. The other is declined with a
parser probe in its thread.

Closes #3616
Closes #3610
Closes #3622
Closes #3638
Closes #3604
Closes #3511
Closes #3602
Closes #3456
Closes #3655
Closes #3657
Closes #3362
Closes #1664

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read validate.py's Markdown and the README Link Definitions Through markdown-it-py

2 participants