Repository navigation
🐛 sphinx-codelinks: report a marked-rst block on the row of its start marker - #2021
Merged
Merged
Conversation
… marker extract_rst took the row of `@rst` as `len(text[:start_idx].splitlines())`, which counts a trailing partial line too. Any text before the marker on its own line -- a one-line `// @rst ... @endrst`, or a doxygen ` * ` prefix -- put the block one row too low. Count newlines instead. Three snapshots carried the wrong row and are corrected, and test_extract_rst now asserts the row it already listed. Closes useblocks#1982
Documentation build overview
46 files changed ·
|
chrisjsewell
approved these changes
Oct 7, 2026
chrisjsewell
left a comment
Member
There was a problem hiding this comment.
Reviewed and approved; thanks for the fix.
What I checked:
- The fix itself:
row_offset = text[:start_idx].count(UNIX_NEWLINE)is the right quantity.splitlines()counted the partial line before the marker as a line, so any text before@rston its own row pushed the block down by one; counting newlines gives the marker's row in every case (// @rst …→ 0,/*\n@rst→ 1,* @rstafter/**\n→ 1).UNIX_NEWLINEis already imported in the module. test_extract_rstnow assertsrow_offset(it listed the value before but never checked it); the corrected expected value for the existing case is right ("\n"precedes the marker there), and the new// @rst … @endrstcase fails without the fix.- The three snapshot rows each move by exactly one, and each new row matches its source line:
dummy_1.cpp13 → 12, the single-line fixture 2 → 1, the doxygen fixture 3 → 2. - On a clean rebase onto master
5868bdd1:poe test-codelinks -rs -n 4→ 662 passed, 59 snapshots passed, 0 skipped; withutils.pyreverted to master the new case and the twomarked_rstsnapshots fail, so the tests pin the fix. ubCode carries nomarked_rstfixture, so nothing there to match. - The changelog entry describes the user-visible effect accurately (rows in
marked_content.jsonand the blob links built from them).
One note, not a blocker: extract_rst is about to be reworked as part of the marked-blocks design (#1885, #1898), so this function has a short remaining life; merging now still closes #1982 and keeps the row semantics the new extractor will inherit.
5 tasks done
chrisjsewell
added a commit
that referenced
this pull request
Oct 8, 2026
…scope association (moved from sphinx-codelinks#100) (#2128) ## What `packages/sphinx-codelinks`: moves useblocks/sphinx-codelinks#100 into the monorepo with the recipe from #1871 (11 commits, authorship kept), and addresses that PR's review on top. The declarative extraction fixtures (`tests/data/extraction/*.yaml`) now snapshot **the real production output**: exactly what `SourceAnalyse.dump_marked_content()` writes, as the flat sorted list of `Metadata.to_dict()` items. Before, they held a test-only projection. Warnings are a separate `[warnings].json` artefact per case. Three deviations from production are documented in the harness and the README: - `filepath` is relative to the case root. - One additive, test-only `tagged_scope_type` key is added. - The warnings are sorted (production reports them in tree-sitter capture order, which differs between runs). New `scope.yaml` (25 cases) covers marker scope association: above a function, inside a body, no scope, the Python docstring path, YAML's structure finder, and C++ class and struct bodies. ## Why Scope association (`find_associated_scope`) had no fixture coverage at all. Snapshotting the real shape also covers `filepath`, `source_map` columns and end positions, `type` and `marker`, which the projection dropped. Review of useblocks/sphinx-codelinks#100, addressed here: 1. **Warnings snapshots were order-unstable.** `_build_warnings` sorts the records again (`file_path, lineno, type, sub_type, msg`). Production reports them in tree-sitter capture order, which differs between two runs of the same input once several extractors are on, so it cannot be pinned. 2. **`SCOPE_NODE_TYPES[cpp]` named `class_definition`,** a kind tree-sitter-cpp never emits, so a marker in a class body had no scope. It now names `class_specifier` and `struct_specifier`, pinned by `scope.yaml`'s `cpp_class_body` / `cpp_struct_body` (🐛, changelog entry). 3. **The field-count wording** was already fixed on master by #2092 ("minimum is" / "maximum is"). Also from the review: - `test_extraction_is_crlf_insensitive` was imperative. It becomes a declarative `line_endings` key on the case schema, plus `line_endings.yaml` (CRLF and a lone CR); such a case also runs with LF endings and must match. - The extraction corpus is shared with ubCode, which copies the YAML files it also has (today 11 of the 14) byte for byte and takes these snapshots as the expected output of its parity test; the README and the harness header now say so, and useblocks/ubcode#3928 tracks how ubCode's sync script takes up this production shape. The LF pin moves to the root `.gitattributes` with that reason, and yamlfmt no longer formats the corpus (closes #2133). Production fixes the fixtures surfaced (physical columns, the need-ref span, docstring tags, a block comment's `*/` as an id) are in #2129, stacked on this one, so this stays reviewable as a move. ## Review of this PR, addressed on the branch (taken over 2026-10-08 with Marco's go) Four commits on top of Marco's, no source, corpus or snapshot change: - The README and the harness header state that the corpus and snapshots are shared with ubCode (the first bullet above). - The LF pin for the corpus and snapshots lives in the root `.gitattributes`, one commented block like the three conformance corpora, with the real reason: ubCode copies the bytes. The tests do not depend on it (YAML normalises line breaks, the harness writes each case's source as exact bytes, syrupy reads snapshots in text mode). The package-level `.gitattributes` is gone, and yamlfmt's exclude covers the corpus (closes #2133). - `_assert_portable_path` is removed: its input is `relative_to(...).as_posix()`, which cannot fail its checks. - The harness's contract comment and the README are cut to the facts; the README lists the warnings sort among the deviations. Found in that review, pre-existing and not changed here: production orders two marked-content entries on one row by tree-sitter's capture order, which differs between runs (#2150; no case here puts two entries on one row), and the scope walk binds a marker above a `typedef struct`, a union or a namespace to a struct further down the file now that `struct_specifier` is a scope kind (#2151). ## Tests The fixture harness and every `test_extraction_fixture` snapshot are converted to the production shape. Additions: - `scope.yaml` (25 cases) and `line_endings.yaml` (2 cases). - Re-captures after #2092 (warning wording, the moved `too_few_fields` case) and #2021 (marked-rst rows); every one of those snapshot diffs is a master change. `uv run poe test-codelinks`: **689 passed, 0 skipped**, stable across reruns (also under xdist and from the repository root); the same at the takeover tip, with lint, typecheck, check-workspace, the CLI suite under `GITHUB_ACTIONS=1` and `docs-codelinks` (0 warnings) green, and every corpus and snapshot byte unchanged. ## Checklist - [x] Every line of this change has been read: Marco's review of useblocks/sphinx-codelinks#100, the review of 2026-10-08, and the takeover's line-by-line pass; it was not generated automatically from an issue. - [x] I ran the package's tests (`uv run poe test-<package>`) and they pass. - [x] Documentation is updated where behaviour or options change. - [x] The package's `docs/changelog.rst` has an entry under *Unreleased*. - [x] `uv run poe lint` and `uv run poe typecheck` pass. Closes #2133 --------- Co-authored-by: Chris Sewell <chrisj_sewell@hotmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1982
extract_rsttook the row of@rstaslen(text[:start_idx].splitlines()).splitlines()counts a trailing partial line as a line, so any text before the start marker on its own line pushed the block one row down. This PR counts newlines instead:text[:start_idx].count("\n").The bug was wider than the issue describes. It hit any
@rstwith text before it on the same line, not only one-line blocks:@rst// @rst … @endrst//* @rston its own line/**\n *(after stripping)@rststarting its own line/*\nChanges
analyse/utils.py: the one-line fix.test_extract_rstlisted the expectedrow_offsetbut never asserted it, and its existing one-line case expected0when@rstsits on row 1. The test now asserts the row, that case is corrected, and a// @rst … @endrstcase is added. The new case fails before the fix.test_analyse[src_dir0-src_paths0].anchors.json:dummy_1.cpprow 13 → 12, as the issue predicted.test_extraction_fixture[marked_rst-single_line_rst]:/* @rst … @endrst */on line 1, reported line 2 → 1.test_extraction_fixture[marked_rst-multi_line_rst_doxygen]:* @rston line 2, reported line 3 → 2.test_analyse_long_source.py: removed the comment that described this bug as a separate problem.marked_content.json, and blob links built from them, move up one row for such blocks.Checks
uv run poe test-codelinks: 404 passed, 58 snapshots passeduv run poe lint: cleanuv run poe typecheck: cleanuv run poe docs-codelinks: build succeeded