Repository navigation
fix(analyse): anchor newline-terminated one-line markers to comment start (#88) - #89
Conversation
…tart (#88) The one-line marker scanner located the start sequence with ``str.find``, matching it anywhere in the comment. When the sequence (default ``@``) appeared inside free-form prose followed by a comma, the text was mis-parsed as ``title, id, ...`` and the bogus id (containing spaces/specials) later failed the sphinx-needs id regex with ``InvalidNeedException``. Anchor markers whose ``end_sequence`` is the newline (i.e. they extend to the end of the line) to the start of the comment content: only comment decoration (``//``, ``#``, ``*`` ...) and whitespace may precede the start sequence. An alphanumeric char before it means the sequence is prose and the line is ignored. Explicitly-bounded markers (e.g. ``[[ ... ]]``) are self-delimiting and stay position-independent.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #89 +/- ##
==========================================
+ Coverage 91.40% 91.44% +0.04%
==========================================
Files 34 34
Lines 3012 3027 +15
Branches 322 323 +1
==========================================
+ Hits 2753 2768 +15
Misses 160 160
Partials 99 99 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Correct, well-tested, and merge-ready. One backwards-incompatible behavior change worth a changelog note; the rest are minor. The guard is well-placed — right after Backwards-compat (intended, but please note in the release notes)For newline-terminated styles (the default The drop is silent — Correctness — verified
Minor
No outstanding review comments to address. |
Follow-up to #97. Adds the traceability artifacts for the substantial features that landed since 1.3.0 without any. ## Audit Every merged PR in the `1.3.0` → `1.4.0` window, checked against `features.rst` and the `@` markers in `src/`: | PR | Substantial feature? | Traceability | |---|---|---| | #92 Bash language support | yes | ✅ `FE_BASH` + `FAULT_BASH_1/2` + `IMPL_BASH_1/2/3` + added to `IMPL_LANG_1` | | #85 Preprocessor-aware C/C++ engine (libclang) | yes | ❌ **none** — no feature need, no markers | | #89 Marker anchoring fix | no (fix to existing traced code, `IMPL_OLP_1`) | n/a | | #83 Declarative fixture tests | no (test infra, no `src/` change) | n/a | | #84 / #86 Dependency caps | no | n/a | So the libclang engine was the one gap: 4 new modules and 353 lines in `analyse/preproc/`, plus wiring in `analyse.py` / `config.py` / `cmd.py` / `src_trace.py`, with zero traceability. ## Changes **`docs/source/components/features.rst`** — new `FE_PREPROC` feature need, placed after the language-support block, with three `fault` children: - `FAULT_PREPROC_1` — objects in inactive branches are extracted (false positives, the defect the engine exists to prevent) - `FAULT_PREPROC_2` — objects in active branches are dropped (false negatives) - `FAULT_PREPROC_3` — flags resolved from the wrong compilation database entry, so the wrong branches evaluate as active **Six `@` markers**, one per capability listed in the need, so the mapping is 1:1 and reviewable: | Marker | Site | Capability | |---|---|---| | `IMPL_PREPROC_1` | `analyse.py::create_src_objects_libclang` | engine entry / opt-in | | `IMPL_PREPROC_2` | `preproc/libclang_parser.py::extract_active_comments` | drop inactive-branch comments | | `IMPL_PREPROC_3` | `preproc/compile_db.py::find_compile_db` | walk-up database discovery | | `IMPL_PREPROC_4` | `preproc/compile_db.py::load_flags_map` | per-file compiler flags | | `IMPL_PREPROC_5` | `preproc/compile_db.py::defines_to_args` | standalone header parsing | | `IMPL_PREPROC_6` | `preproc/loader.py::load_clang_cindex` | optional-dependency guard | Comments only — no logic touched. ## Verification A green `-nW` build alone would not prove the markers work: a marker that fails to parse produces no need and no error. So the built output was checked directly. ``` $ tox -e docs-clean # sphinx-build -nW --keep-going build succeeded. $ grep -o 'id="IMPL_PREPROC_[0-9]"' docs/_build/html/development/traceability.html | sort -u id="IMPL_PREPROC_1" … id="IMPL_PREPROC_6" # all six needs materialized $ grep -o 'href="[^"]*#FE_PREPROC"[^>]*title="IMPL_PREPROC_[0-9]"' … | sort -u IMPL_PREPROC_1 … IMPL_PREPROC_6 # all six link to the feature ``` - `tox -e py312-sphinx8-needs5` — 345 passed, 1 skipped, 58 snapshots passed - `tox -e mypy` — no issues in 20 source files - `pre-commit run --files …` — all hooks pass ## Deliberately out of scope Pre-existing gaps, all predating the 1.3.0 → 1.4.0 window this PR was scoped to. Flagging rather than fixing, since each is a separate judgment call: - **`FE_CSHARP` is an orphan.** It has no incoming `impl` link — it is missing from the `IMPL_LANG_1` list in `analyse/utils.py:110` even though `init_tree_sitter` handles `CommentType.cs` (utils.py:122). C# is the only supported language whose implementation is untraced. One-word fix if you want it here. - **No feature need for the `src-trace` directive, nor for source-link URL generation** (`local-url` / `remote-url`), while the CLI has three (`FE_CLI_*`). This asymmetry may be intentional — capability-level needs covering *what* is traced regardless of interface — so changing it is a feature-model decision, not a gap fix. - **The test-needs layer is empty.** `docs/ubproject.toml` defines a `test` type and `development/traceability.rst` runs a `src-trace` over the `tests` project, but `tests/` carries no `@` markers, so zero `TEST_` needs exist. ## Note on sequencing No changelog entry here, deliberately: this PR is based on `main`, where the `Unreleased` heading still holds the Bash entry that #97 converts into the dated `1.4.0` section. Editing it would conflict with the open release PR. Since these artifacts document a feature already listed in the 1.4.0 notes, either fold a line into #97 before merging it, or let the next release's notes pick this up — your call on the order.
Prepares the **1.4.0** release. ## Release changes - ⬆️ Bump version `1.3.0` → `1.4.0` in `pyproject.toml`. - 📝 Finalize the changelog: convert the `Unreleased` heading into a dated `1.4.0` section (released 30.07.2026). Only 1 of the 6 PRs merged since 1.3.0 (#92) had added a changelog entry, so this also documents the 5 that were missing one, plus the traceability artifacts from #98. - 🗺️ No roadmap changes — none of the merged PRs completed a listed roadmap item. Minor bump: two new user-facing features, no removals or config-breaking changes. ## Changelog for 1.4.0 **New and Improved** - ✨ Added Bash language support for the `analyse` module (#92) - ✨ Added an opt-in preprocessor-aware C/C++ extraction engine, powered by libclang (#85) - 📚 Traced the preprocessor-aware C/C++ engine in the feature documentation (#98) - 🧪 Added a declarative fixture and snapshot test layer for marker extraction (#83) **Fixes** - 🐛 Anchor newline-terminated one-line markers to the start of the comment (#89, fixes #88) - 🐛 Pinned `typer` and `sphinxcontrib-typer` to keep the documentation build working (#84, #86) ## Behavior change worth calling out #89 is released as a fix, but it does narrow marker matching: a newline-terminated one-line marker (default `@…`) is now only recognized when nothing but comment decoration and whitespace precedes the start sequence. A project that deliberately embedded such markers *after* prose on the same line will stop seeing them — the fix is what stops `@` in prose from producing bogus need IDs and `InvalidNeedException`. Explicitly bounded markers (e.g. `[[ … ]]`) are unaffected. ## Verification Rebased onto `main` at #98, so this branch carries the full 1.4.0 content. - `tox -e py312-sphinx8-needs5` — 345 passed, 1 skipped (58 snapshots passed). - `tox -e docs-clean` (`sphinx-build -nW --keep-going`) — build succeeded, 0 warnings. - `pre-commit run --files docs/source/development/change_log.rst pyproject.toml` — all hooks pass. ## Not included Pre-existing traceability gaps are left for a follow-up, since all predate 1.3.0: `FE_CSHARP` has no incoming `impl` link; the five enum-site markers carrying a trailing `;` (`IMPL_BASH_1`, `IMPL_GO_1`, `IMPL_RUST_1`, `IMPL_RUST_2`, `IMPL_JSONC_1`) are inert and produce no needs, so file-extension discovery is untraced for every language; and `IMPL_C_1` / `IMPL_PY_1` / `IMPL_GO_2` are anchored on `COMMENT_MARKERS`, which has no readers. See the discussion in #98. ## After merge - Tag `1.4.0` on the merge commit (matching the existing `1.0.0`–`1.3.0` tags). The `Release` workflow triggers on that tag pattern and publishes to PyPI.
…tart (useblocks/sphinx-codelinks#88) (useblocks/sphinx-codelinks#89) ## Problem Fixes useblocks/sphinx-codelinks#88. The one-line marker scanner locates the start sequence with `str.find`, so it matches the sequence (default `@`) **anywhere** in a comment. When `@` appears inside free-form prose followed by a comma, the text is mis-parsed as `title, id, …` and the bogus id (with spaces/specials) later fails the sphinx-needs id regex with `InvalidNeedException`: ``` // See @author, check the example → id = "check the example" → InvalidNeedException // We parse things matching @pattern, then act // Some prose that mentions @@another-tag(item_1, item_2): more text ``` ## Fix Anchor markers whose `end_sequence` is the newline (i.e. they run to end-of-line) to the **start of the comment content**: only comment decoration (`//`, `#`, `*`, `///`, `//!`, …) and whitespace may precede the start sequence. A word character before it means the sequence is prose, and the line is ignored. Explicitly-bounded markers (e.g. `[[ … ]]`) are self-delimiting, so they stay position-independent and may still be embedded after prose — this is what keeps the recommended bracket-style workaround (and the existing `supercharge.cpp` fixture) working. ## Tests - New: the three issue-useblocks/sphinx-codelinks#88 prose lines are ignored; decoration-prefixed markers (line-comment, indented, block `*`, `///`, `//!`) still parse; a bounded `[[ … ]]` marker embedded after prose still parses. - Full suite green (271 passed), mypy clean. ## Workaround (no upgrade required) Switch the affected project to a self-delimiting bracket style, which does not collide with `@` in prose: ```toml [codelinks.projects.<name>.analyse.oneline_comment_style] start_sequence = "[[" end_sequence = "]]" ```
…cks/sphinx-codelinks#98) Follow-up to useblocks/sphinx-codelinks#97. Adds the traceability artifacts for the substantial features that landed since 1.3.0 without any. ## Audit Every merged PR in the `1.3.0` → `1.4.0` window, checked against `features.rst` and the `@` markers in `src/`: | PR | Substantial feature? | Traceability | |---|---|---| | useblocks/sphinx-codelinks#92 Bash language support | yes | ✅ `FE_BASH` + `FAULT_BASH_1/2` + `IMPL_BASH_1/2/3` + added to `IMPL_LANG_1` | | useblocks/sphinx-codelinks#85 Preprocessor-aware C/C++ engine (libclang) | yes | ❌ **none** — no feature need, no markers | | useblocks/sphinx-codelinks#89 Marker anchoring fix | no (fix to existing traced code, `IMPL_OLP_1`) | n/a | | useblocks/sphinx-codelinks#83 Declarative fixture tests | no (test infra, no `src/` change) | n/a | | useblocks/sphinx-codelinks#84 / useblocks/sphinx-codelinks#86 Dependency caps | no | n/a | So the libclang engine was the one gap: 4 new modules and 353 lines in `analyse/preproc/`, plus wiring in `analyse.py` / `config.py` / `cmd.py` / `src_trace.py`, with zero traceability. ## Changes **`docs/source/components/features.rst`** — new `FE_PREPROC` feature need, placed after the language-support block, with three `fault` children: - `FAULT_PREPROC_1` — objects in inactive branches are extracted (false positives, the defect the engine exists to prevent) - `FAULT_PREPROC_2` — objects in active branches are dropped (false negatives) - `FAULT_PREPROC_3` — flags resolved from the wrong compilation database entry, so the wrong branches evaluate as active **Six `@` markers**, one per capability listed in the need, so the mapping is 1:1 and reviewable: | Marker | Site | Capability | |---|---|---| | `IMPL_PREPROC_1` | `analyse.py::create_src_objects_libclang` | engine entry / opt-in | | `IMPL_PREPROC_2` | `preproc/libclang_parser.py::extract_active_comments` | drop inactive-branch comments | | `IMPL_PREPROC_3` | `preproc/compile_db.py::find_compile_db` | walk-up database discovery | | `IMPL_PREPROC_4` | `preproc/compile_db.py::load_flags_map` | per-file compiler flags | | `IMPL_PREPROC_5` | `preproc/compile_db.py::defines_to_args` | standalone header parsing | | `IMPL_PREPROC_6` | `preproc/loader.py::load_clang_cindex` | optional-dependency guard | Comments only — no logic touched. ## Verification A green `-nW` build alone would not prove the markers work: a marker that fails to parse produces no need and no error. So the built output was checked directly. ``` $ tox -e docs-clean # sphinx-build -nW --keep-going build succeeded. $ grep -o 'id="IMPL_PREPROC_[0-9]"' docs/_build/html/development/traceability.html | sort -u id="IMPL_PREPROC_1" … id="IMPL_PREPROC_6" # all six needs materialized $ grep -o 'href="[^"]*#FE_PREPROC"[^>]*title="IMPL_PREPROC_[0-9]"' … | sort -u IMPL_PREPROC_1 … IMPL_PREPROC_6 # all six link to the feature ``` - `tox -e py312-sphinx8-needs5` — 345 passed, 1 skipped, 58 snapshots passed - `tox -e mypy` — no issues in 20 source files - `pre-commit run --files …` — all hooks pass ## Deliberately out of scope Pre-existing gaps, all predating the 1.3.0 → 1.4.0 window this PR was scoped to. Flagging rather than fixing, since each is a separate judgment call: - **`FE_CSHARP` is an orphan.** It has no incoming `impl` link — it is missing from the `IMPL_LANG_1` list in `analyse/utils.py:110` even though `init_tree_sitter` handles `CommentType.cs` (utils.py:122). C# is the only supported language whose implementation is untraced. One-word fix if you want it here. - **No feature need for the `src-trace` directive, nor for source-link URL generation** (`local-url` / `remote-url`), while the CLI has three (`FE_CLI_*`). This asymmetry may be intentional — capability-level needs covering *what* is traced regardless of interface — so changing it is a feature-model decision, not a gap fix. - **The test-needs layer is empty.** `docs/ubproject.toml` defines a `test` type and `development/traceability.rst` runs a `src-trace` over the `tests` project, but `tests/` carries no `@` markers, so zero `TEST_` needs exist. ## Note on sequencing No changelog entry here, deliberately: this PR is based on `main`, where the `Unreleased` heading still holds the Bash entry that useblocks/sphinx-codelinks#97 converts into the dated `1.4.0` section. Editing it would conflict with the open release PR. Since these artifacts document a feature already listed in the 1.4.0 notes, either fold a line into useblocks/sphinx-codelinks#97 before merging it, or let the next release's notes pick this up — your call on the order.
Prepares the **1.4.0** release. ## Release changes - ⬆️ Bump version `1.3.0` → `1.4.0` in `pyproject.toml`. - 📝 Finalize the changelog: convert the `Unreleased` heading into a dated `1.4.0` section (released 30.07.2026). Only 1 of the 6 PRs merged since 1.3.0 (useblocks/sphinx-codelinks#92) had added a changelog entry, so this also documents the 5 that were missing one, plus the traceability artifacts from useblocks/sphinx-codelinks#98. - 🗺️ No roadmap changes — none of the merged PRs completed a listed roadmap item. Minor bump: two new user-facing features, no removals or config-breaking changes. ## Changelog for 1.4.0 **New and Improved** - ✨ Added Bash language support for the `analyse` module (useblocks/sphinx-codelinks#92) - ✨ Added an opt-in preprocessor-aware C/C++ extraction engine, powered by libclang (useblocks/sphinx-codelinks#85) - 📚 Traced the preprocessor-aware C/C++ engine in the feature documentation (useblocks/sphinx-codelinks#98) - 🧪 Added a declarative fixture and snapshot test layer for marker extraction (useblocks/sphinx-codelinks#83) **Fixes** - 🐛 Anchor newline-terminated one-line markers to the start of the comment (useblocks/sphinx-codelinks#89, fixes useblocks/sphinx-codelinks#88) - 🐛 Pinned `typer` and `sphinxcontrib-typer` to keep the documentation build working (useblocks/sphinx-codelinks#84, useblocks/sphinx-codelinks#86) ## Behavior change worth calling out useblocks/sphinx-codelinks#89 is released as a fix, but it does narrow marker matching: a newline-terminated one-line marker (default `@…`) is now only recognized when nothing but comment decoration and whitespace precedes the start sequence. A project that deliberately embedded such markers *after* prose on the same line will stop seeing them — the fix is what stops `@` in prose from producing bogus need IDs and `InvalidNeedException`. Explicitly bounded markers (e.g. `[[ … ]]`) are unaffected. ## Verification Rebased onto `main` at useblocks/sphinx-codelinks#98, so this branch carries the full 1.4.0 content. - `tox -e py312-sphinx8-needs5` — 345 passed, 1 skipped (58 snapshots passed). - `tox -e docs-clean` (`sphinx-build -nW --keep-going`) — build succeeded, 0 warnings. - `pre-commit run --files docs/source/development/change_log.rst pyproject.toml` — all hooks pass. ## Not included Pre-existing traceability gaps are left for a follow-up, since all predate 1.3.0: `FE_CSHARP` has no incoming `impl` link; the five enum-site markers carrying a trailing `;` (`IMPL_BASH_1`, `IMPL_GO_1`, `IMPL_RUST_1`, `IMPL_RUST_2`, `IMPL_JSONC_1`) are inert and produce no needs, so file-extension discovery is untraced for every language; and `IMPL_C_1` / `IMPL_PY_1` / `IMPL_GO_2` are anchored on `COMMENT_MARKERS`, which has no readers. See the discussion in useblocks/sphinx-codelinks#98. ## After merge - Tag `1.4.0` on the merge commit (matching the existing `1.0.0`–`1.3.0` tags). The `Release` workflow triggers on that tag pattern and publishes to PyPI.
Problem
Fixes #88.
The one-line marker scanner locates the start sequence with
str.find, so itmatches the sequence (default
@) anywhere in a comment. When@appearsinside free-form prose followed by a comma, the text is mis-parsed as
title, id, …and the bogus id (with spaces/specials) later fails thesphinx-needs id regex with
InvalidNeedException:Fix
Anchor markers whose
end_sequenceis the newline (i.e. they run toend-of-line) to the start of the comment content: only comment decoration
(
//,#,*,///,//!, …) and whitespace may precede the start sequence.A word character before it means the sequence is prose, and the line is ignored.
Explicitly-bounded markers (e.g.
[[ … ]]) are self-delimiting, so they stayposition-independent and may still be embedded after prose — this is what keeps
the recommended bracket-style workaround (and the existing
supercharge.cppfixture) working.
Tests
(line-comment, indented, block
*,///,//!) still parse; a bounded[[ … ]]marker embedded after prose still parses.Workaround (no upgrade required)
Switch the affected project to a self-delimiting bracket style, which does not
collide with
@in prose: