Skip to content

🐛 sphinx-codelinks: report a marked-rst block on the row of its start marker - #2021

Merged
chrisjsewell merged 3 commits into
useblocks:masterfrom
patdhlk:fix/1982-one-line-rst-row
Oct 7, 2026
Merged

chrisjsewell merged 3 commits into
useblocks:masterfrom
patdhlk:fix/1982-one-line-rst-row

Conversation

@patdhlk

@patdhlk patdhlk commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Closes #1982

extract_rst took the row of @rst as len(text[:start_idx].splitlines()). splitlines() counts a trailing partial line as a line, so any text before the start marker on its own line pushed the block one row down. This PR counts newlines instead: text[:start_idx].count("\n").

The bug was wider than the issue describes. It hit any @rst with text before it on the same line, not only one-line blocks:

case text before @rst before after
one-line block, // @rst … @endrst // 1 0
doxygen block, * @rst on its own line /**\n * (after stripping) 2 1
@rst starting its own line /*\n 1 1 (unchanged)

Changes

  • analyse/utils.py: the one-line fix.
  • test_extract_rst listed the expected row_offset but never asserted it, and its existing one-line case expected 0 when @rst sits on row 1. The test now asserts the row, that case is corrected, and a // @rst … @endrst case is added. The new case fails before the fix.
  • Three snapshots had the wrong row baked in, and each corrected row matches its source:
    • test_analyse[src_dir0-src_paths0].anchors.json: dummy_1.cpp row 13 → 12, as the issue predicted.
    • test_extraction_fixture[marked_rst-single_line_rst]: /* @rst … @endrst */ on line 1, reported line 2 → 1.
    • test_extraction_fixture[marked_rst-multi_line_rst_doxygen]: * @rst on line 2, reported line 3 → 2.
  • test_analyse_long_source.py: removed the comment that described this bug as a separate problem.
  • Changelog entry under Unreleased. Users will see source maps in marked_content.json, and blob links built from them, move up one row for such blocks.

Checks

  • uv run poe test-codelinks: 404 passed, 58 snapshots passed
  • uv run poe lint: clean
  • uv run poe typecheck: clean
  • uv run poe docs-codelinks: build succeeded

… marker

extract_rst took the row of `@rst` as `len(text[:start_idx].splitlines())`,
which counts a trailing partial line too. Any text before the marker on its
own line -- a one-line `// @rst ... @endrst`, or a doxygen ` * ` prefix --
put the block one row too low. Count newlines instead.

Three snapshots carried the wrong row and are corrected, and
test_extract_rst now asserts the row it already listed.

Closes useblocks#1982
@github-actions github-actions Bot added the pkg: sphinx-codelinks Concerns the sphinx-codelinks package (packages/sphinx-codelinks) label Oct 2, 2026

@chrisjsewell chrisjsewell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and approved; thanks for the fix.

What I checked:

  • The fix itself: row_offset = text[:start_idx].count(UNIX_NEWLINE) is the right quantity. splitlines() counted the partial line before the marker as a line, so any text before @rst on its own row pushed the block down by one; counting newlines gives the marker's row in every case (// @rst … → 0, /*\n@rst → 1, * @rst after /**\n → 1). UNIX_NEWLINE is already imported in the module.
  • test_extract_rst now asserts row_offset (it listed the value before but never checked it); the corrected expected value for the existing case is right ("\n" precedes the marker there), and the new // @rst … @endrst case fails without the fix.
  • The three snapshot rows each move by exactly one, and each new row matches its source line: dummy_1.cpp 13 → 12, the single-line fixture 2 → 1, the doxygen fixture 3 → 2.
  • On a clean rebase onto master 5868bdd1: poe test-codelinks -rs -n 4 → 662 passed, 59 snapshots passed, 0 skipped; with utils.py reverted to master the new case and the two marked_rst snapshots fail, so the tests pin the fix. ubCode carries no marked_rst fixture, so nothing there to match.
  • The changelog entry describes the user-visible effect accurately (rows in marked_content.json and the blob links built from them).

One note, not a blocker: extract_rst is about to be reworked as part of the marked-blocks design (#1885, #1898), so this function has a short remaining life; merging now still closes #1982 and keeps the row semantics the new extractor will inherit.

@chrisjsewell
chrisjsewell merged commit abab51f into useblocks:master Oct 7, 2026
36 checks passed
chrisjsewell added a commit that referenced this pull request Oct 8, 2026
…scope association (moved from sphinx-codelinks#100) (#2128)

## What

`packages/sphinx-codelinks`: moves useblocks/sphinx-codelinks#100 into
the monorepo with the
recipe from #1871 (11 commits, authorship kept), and addresses that PR's
review on top.

The declarative extraction fixtures (`tests/data/extraction/*.yaml`) now
snapshot **the real
production output**: exactly what `SourceAnalyse.dump_marked_content()`
writes, as the flat sorted
list of `Metadata.to_dict()` items. Before, they held a test-only
projection. Warnings are a separate
`[warnings].json` artefact per case. Three deviations from production
are documented in the harness
and the README:
- `filepath` is relative to the case root.
- One additive, test-only `tagged_scope_type` key is added.
- The warnings are sorted (production reports them in tree-sitter
capture order, which differs between runs).

New `scope.yaml` (25 cases) covers marker scope association: above a
function, inside a body,
no scope, the Python docstring path, YAML's structure finder, and C++
class and struct bodies.

## Why

Scope association (`find_associated_scope`) had no fixture coverage at
all. Snapshotting the real
shape also covers `filepath`, `source_map` columns and end positions,
`type` and `marker`, which the
projection dropped.

Review of useblocks/sphinx-codelinks#100, addressed here:

1. **Warnings snapshots were order-unstable.** `_build_warnings` sorts
the records again
(`file_path, lineno, type, sub_type, msg`). Production reports them in
tree-sitter capture order,
which differs between two runs of the same input once several extractors
are on, so it cannot be
   pinned.
2. **`SCOPE_NODE_TYPES[cpp]` named `class_definition`,** a kind
tree-sitter-cpp never emits, so a
marker in a class body had no scope. It now names `class_specifier` and
`struct_specifier`, pinned by
`scope.yaml`'s `cpp_class_body` / `cpp_struct_body` (🐛, changelog
entry).
3. **The field-count wording** was already fixed on master by #2092
("minimum is" / "maximum is").

Also from the review:
- `test_extraction_is_crlf_insensitive` was imperative. It becomes a
declarative `line_endings` key
on the case schema, plus `line_endings.yaml` (CRLF and a lone CR); such
a case also runs with LF
  endings and must match.
- The extraction corpus is shared with ubCode, which copies the YAML
files it also has (today 11 of the 14) byte for byte and takes these
snapshots as the expected output of its parity test; the README and the
harness header now say so, and useblocks/ubcode#3928 tracks how ubCode's
sync script takes up this production shape. The LF pin moves to the root
`.gitattributes` with that reason, and yamlfmt no longer formats the
corpus (closes #2133).

Production fixes the fixtures surfaced (physical columns, the need-ref
span, docstring tags, a block
comment's `*/` as an id) are in #2129, stacked on this one, so this
stays reviewable as a move.

## Review of this PR, addressed on the branch (taken over 2026-10-08
with Marco's go)

Four commits on top of Marco's, no source, corpus or snapshot change:
- The README and the harness header state that the corpus and snapshots
are shared with ubCode (the first bullet above).
- The LF pin for the corpus and snapshots lives in the root
`.gitattributes`, one commented block like the three conformance
corpora, with the real reason: ubCode copies the bytes. The tests do not
depend on it (YAML normalises line breaks, the harness writes each
case's source as exact bytes, syrupy reads snapshots in text mode). The
package-level `.gitattributes` is gone, and yamlfmt's exclude covers the
corpus (closes #2133).
- `_assert_portable_path` is removed: its input is
`relative_to(...).as_posix()`, which cannot fail its checks.
- The harness's contract comment and the README are cut to the facts;
the README lists the warnings sort among the deviations.

Found in that review, pre-existing and not changed here: production
orders two marked-content entries on one row by tree-sitter's capture
order, which differs between runs (#2150; no case here puts two entries
on one row), and the scope walk binds a marker above a `typedef struct`,
a union or a namespace to a struct further down the file now that
`struct_specifier` is a scope kind (#2151).

## Tests

The fixture harness and every `test_extraction_fixture` snapshot are
converted to the production
shape. Additions:
- `scope.yaml` (25 cases) and `line_endings.yaml` (2 cases).
- Re-captures after #2092 (warning wording, the moved `too_few_fields`
case) and #2021 (marked-rst
  rows); every one of those snapshot diffs is a master change.

`uv run poe test-codelinks`: **689 passed, 0 skipped**, stable across
reruns (also under xdist and
from the repository root); the same at the takeover tip, with lint,
typecheck, check-workspace, the CLI suite under `GITHUB_ACTIONS=1` and
`docs-codelinks` (0 warnings) green, and every corpus and snapshot byte
unchanged.

## Checklist

- [x] Every line of this change has been read: Marco's review of
useblocks/sphinx-codelinks#100, the review of 2026-10-08, and the
takeover's line-by-line pass; it was not generated automatically from an
issue.
- [x] I ran the package's tests (`uv run poe test-<package>`) and they
pass.
- [x] Documentation is updated where behaviour or options change.
- [x] The package's `docs/changelog.rst` has an entry under
*Unreleased*.
- [x] `uv run poe lint` and `uv run poe typecheck` pass.

Closes #2133

---------

Co-authored-by: Chris Sewell <chrisj_sewell@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pkg: sphinx-codelinks Concerns the sphinx-codelinks package (packages/sphinx-codelinks)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 A one-line marked-rst block is reported one row too low

2 participants