Skip to content

✨ sphinx-codelinks: VHDL comment type - #2077

Draft
lennard-ub wants to merge 1 commit into
useblocks:masterfrom
lennard-ub:feat/codelinks-vhdl
Draft

lennard-ub wants to merge 1 commit into
useblocks:masterfrom
lennard-ub:feat/codelinks-vhdl

Conversation

@lennard-ub

@lennard-ub lennard-ub commented Oct 6, 2026 •

Copy link
Copy Markdown

Adds VHDL as a sphinx-codelinks comment type. Rebased onto master e29e35bc, so VHDL now uses the scope snapshots (#2128), physical columns (#2129), multi-line needs (#2152) and the TypeScript work (#2166). The ubCode counterpart is useblocks/ubcode#3875; both run the same shared corpus.

What

  • comment_type = "vhdl" analyses .vhd and .vhdl files: -- line comments and VHDL-2008 /* */ block comments. The grammar is tree-sitter-vhdl>=1.4.0, and the query reuses RUST_QUERY, since VHDL names its two comment kinds the same way.
  • Scope: a VHDL branch of find_associated_scope, find_vhdl_associated_scope, implements the rule from the review.
    • A trailing comment documents the innermost design element ending on its row, a port included.
    • A leading comment documents the next non-comment sibling. For one of the six wrappers, it documents the wrapper's first item, and in a design_unit that is the first item after the context clauses.
    • Otherwise the comment documents the enclosing design element, or nothing at file level. The walk never passes a sibling.
    • The 37 design-element kinds are in SCOPE_NODE_TYPES[CommentType.vhdl]. find_next_scope is unchanged.
  • Multi-line needs: -- and --! join LINE_DELIMITERS. --! is to -- what //! is to //, so a Doxygen run forms a block too.
  • Doxygen: a --! comment line that starts with a Doxygen command is a doxygen_tag warning, never a need. The rule is the narrowed jsdoc_tag rule: only under the start sequence @, and only where the one-line parser would have read the line. The command end is a space, a tab, [, { or the line end, so @param[in] counts.
  • The rule text and the 197 command names live in the header of the new shared corpus file doxygen.yaml, so A Doxygen tag line with a comma in a C/C++, C# or Go /** */ comment becomes a need with a garbage id #2171 only needs to add its /**, /*! and /// forms.

Review points

  1. Scope: implemented as specified. Each row of the review's table is a vhdl_* case in scope.yaml, pinned by tagged_scope_type. More cases cover the design_unit / interface_list wrappers, a sequential statement (binds its process) and the file level (no scope).
  2. doxygen_tag: implemented, with the shared corpus header described above.
  3. Rebase:
  4. Fixtures: the VHDL cases sit in scope, positions (end column of the newline-inclusive node, no newline at EOF, a comment after code), line_endings (CRLF), multiline_needs (-- and --! runs), need_refs, doxygen and extra_languages. The shared discover_fixtures.json gains a vhdl_comment_type case. The result in text unit tests of the first version are not ported.
  5. Floor: >=1.4.0. The whole suite passes under 1.4.0 and under the locked 2.0.3.

No doc_test project: Bash and TypeScript have none, and the corpus covers extraction.

Checked locally

  • uv run poe test-codelinks: 1103 passed, none skipped. The suite passes again with tree-sitter-vhdl==1.4.0.
  • uv run poe lint, uv run poe typecheck and uv run poe docs-codelinks (-nW) pass.
  • End to end: one project with one ubproject.toml, built with Sphinx and with ubCode's ubc build html, gives the same one-line and multi-line needs and the same doxygen_tag warning.

@github-actions github-actions Bot added the pkg: sphinx-codelinks Concerns the sphinx-codelinks package (packages/sphinx-codelinks) label Oct 6, 2026

@ubmarco ubmarco left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed together with its ubCode counterpart, useblocks/ubcode#3875. The extraction itself is careful. Master has gained more than 100 commits since 6434aef0, including the scope snapshots (#2128), physical columns (#2129), multi-line needs (#2152) and TypeScript (#2166). So the rebase re-plugs VHDL into those seams. Two behaviour decisions come first; they apply to both repositories.

1. Scope association: a VHDL rule instead of the generic walk

find_next_scope binds a marker to the next scope-kind sibling anywhere further down. VHDL code is mostly declarations and concurrent statements, so that puts markers on the wrong construct. Measured with a standalone probe on 72 VHDL samples (82 markers); tree-sitter-vhdl 1.4.0 and 2.0.3 agree on every one.

Marker binds with this PR's tables should bind
above u_fifo : entity work.fifo … the next process the instantiation
above or trailing led_o <= …; the next process the assignment
above or trailing a signal or constant a later function, or the architecture the declaration
trailing a port (clk : in std_logic; --! …) the entity the port
above a process inside for … generate the generate the process
above the first procedure of a process the process the procedure
above a function in a block's declarations the block the function

The rule, as a VHDL branch of find_associated_scope (YAML and JSONC have their own):

  1. Trailing comment (a preceding non-comment sibling ends on the comment's row): the innermost design element ending on that row, which reaches a port inside interface_list. If there is none, the nearest enclosing design element. Comment siblings are skipped when looking back, because a VHDL line_comment holds its newline, so the comment on the row above "ends" on this row.
  2. Leading comment: the next non-comment sibling, if it is a design element.
    • If that sibling is a wrapper (design_unit, process_head, block_head, generate_head, generate_block, interface_list), its first non-comment item instead. In a design_unit, that is the first item after the library, use and context clauses.
    • The walk never passes a sibling. This is the first fix candidate of #2151, so VHDL starts where the other languages are heading.
  3. Otherwise the nearest enclosing design element, or none at file level.

Design elements, for SCOPE_NODE_TYPES[CommentType.vhdl], are 37 kinds:

  • Design units: entity_declaration, architecture_definition, package_declaration, package_definition, package_instantiation_declaration, configuration_declaration, context_declaration.
  • Subprograms: subprogram_declaration, subprogram_definition, subprogram_instantiation_declaration.
  • Declarations: type_declaration, subtype_declaration, component_declaration, constant_declaration, signal_declaration, variable_declaration, file_declaration, alias_declaration.
  • Generics, ports and parameters: interface_declaration and the seven interface_*_declaration kinds.
  • Concurrent statements: process_statement, block_statement, the three *_generate_statement kinds, component_instantiation_statement, the three concurrent_*_signal_assignment kinds, concurrent_assertion_statement, concurrent_procedure_call_statement.

Sequential statements are not design elements: a marker among them documents its process or subprogram. Each of the six wrappers is needed by at least one probe marker. Under this rule, 36 of the 82 markers bind differently than with this PR's tables. The hard-coded {"block", "design_unit"} in the shared find_next_scope goes. FE_VHDL's "Accurate scope detection for nested structures" then holds; today a process inside a generate binds to the generate.

2. Doxygen --! lines become needs with garbage ids

--! is Doxygen's VHDL comment. Through the shared one-line parser, sphinx-codelinks 2.0.0 turns these lines into needs, with no warning:

  • --! @brief Counter with enable, reset and load becomes a need with the id reset and load.
  • --! @param clk Clock input, rising edge active becomes a need with the id rising edge active.
  • The tag lines without a comma are dropped silently.

This is #2171 for VHDL, whose scope (/**, /*!, ///) misses --!. Implement doxygen_tag here for --! lines and write the shared corpus file's header, with the command list and the rule text, as jsdoc.yaml has. #2171 then only adds its three comment forms.

3. Rebase

  • LINE_DELIMITERS in analyse/multiline_parser.py lacks --. _classify therefore returns "unknown" for -- @need …: no run forms, the lines are read unstripped, and open_rest never sees the open word.
  • The analyse.py hunk goes: that code is gone. _row_count and _line_comment_line already handle a comment node that ends in its newline. Rust's /// node does too, contrary to the description.
  • VHDL_QUERY is word-for-word RUST_QUERY; reuse it.
  • COMMENT_MARKERS in config.py has no readers on master.
  • The extraction snapshot is in the pre-#2128 format and gets regenerated.
  • In uv.lock, keep only the semantic hunk; most of the diff is fork-marker churn.
  • The description's "@rst blocks work" is obsolete since #2152.

4. Declarative fixtures

The 9 scope cases in test_analyse_utils.py assert result in text, which also passes on the enclosing node. Three of them would still pass if process_statement or subprogram_definition were missing from the table. The corpus's tagged_scope_type exists for exactly this, so the VHDL cases belong there:

  • scope.yaml: one case per row of the table in point 1.
  • positions.yaml: the newline-inclusive node's end column, no newline at EOF, and a trailing comment after code.
  • line_endings.yaml: with CRLF the node keeps \r and drops \n; with LF it keeps \n.
  • multiline_needs.yaml, need_refs.yaml and the doxygen_tag cases.

5. Grammar floor

The floor is >=1.5.0 and the lock resolves 2.0.3, but the grammar's crates.io binding stops at 1.4.0. 1.4.0 and 2.0.3 agreed on every probe marker. Unless 1.5.0 is needed for something specific, >=1.4.0 keeps a Rust consumer of the same corpus inside the supported range.

@lennard-ub lennard-ub changed the title ✨ sphinx-codelinks: add VHDL language support ✨ sphinx-codelinks: VHDL comment type Oct 20, 2026
`comment_type = "vhdl"` analyses `.vhd` and `.vhdl` files, their `--` and
VHDL-2008 `/* */` comments, with the tree-sitter-vhdl grammar.

- Scope: a VHDL branch of `find_associated_scope` binds a marker to the
  design element next to it (37 kinds in `SCOPE_NODE_TYPES`): a trailing
  marker to the innermost element ending on its row, a leading one to the
  next sibling or a wrapper's first item, never past a sibling, and
  otherwise to the enclosing element.
- Multi-line needs: `--` and `--!` join `LINE_DELIMITERS`.
- Doxygen: a command line in a `--!` comment is a `doxygen_tag` warning,
  not a need, under the narrowed `jsdoc_tag` rule. The command list and
  the rule live in the new shared corpus file `doxygen.yaml`.
- Corpus: VHDL cases in scope, positions, line_endings, multiline_needs,
  need_refs and extra_languages.
@lennard-ub

Copy link
Copy Markdown
Author

How the review findings were addressed

Everything since your review (2026-10-19) is in ee8e0517, one commit on master e29e35bc.
The branch was not rebased hunk by hunk. It was re-ported onto master's seams (#2128, #2129, #2152, #2166), because the first version predated all four.
Numbering follows your review.

1. Scope: a VHDL rule instead of the generic walk

  • find_vhdl_associated_scope (analyse/utils.py) is a VHDL branch of find_associated_scope, implementing your rule:
    • Trailing: the previous non-comment sibling ends on the comment's row → the innermost design element ending on that row, via _vhdl_innermost_on_row, which follows the last non-comment child that ends on the row. If there is none, the enclosing design element.
    • Leading: the next non-comment sibling, if it is a design element. For one of the six wrappers (VHDL_SCOPE_WRAPPER_TYPES), its first non-comment item instead; in a design_unit, the first item after library / use / context_reference (VHDL_CONTEXT_CLAUSE_TYPES). The walk never passes a sibling.
    • Otherwise: find_enclosing_scope, or None at file level.
    • Comment siblings are skipped in both directions.
  • The 37 kinds are in SCOPE_NODE_TYPES[CommentType.vhdl], grouped as in your list. find_next_scope is unchanged against master: the hard-coded {"block", "design_unit"} is gone.
  • Pinned in scope.yaml with tagged_scope_type, one case per row of your table:
    • vhdl_above_instantiation
    • vhdl_signal_assignment (above and trailing)
    • vhdl_signal_and_constant
    • vhdl_trailing_port
    • vhdl_process_in_generate
    • vhdl_procedure_in_process
    • vhdl_function_in_block
  • More cases cover the remaining branches:
    • vhdl_entity_after_context_clauses: the design_unit and interface_list wrappers.
    • vhdl_sequential_statement: enclosing process.
    • vhdl_no_scope: file level.
  • Not pinned: generate_head has no case of its own.

2. Doxygen --! lines: doxygen_tag

  • WarningSubTypeEnum.doxygen_tag, with DOXYGEN_COMMANDS and doxygen_tag() in analyse/oneline_parser.py.
  • The gate in extract_oneline_need fires only under comment_type = "vhdl", for a comment whose node text starts with --!, and only under the start sequence @. A command line warns only where oneline_parser would have read it. That is jsdoc_tag's narrowed rule: a line it would ignore (--! @brief Counter) stays silent.
  • The command ends at a space, a tab, [ (@param[in]), { (@code{.vhd}) or the line end. JSDoc's } is not included.
  • The command list is 197 names: Doxygen's special commands that are words. The symbol commands (\f$, \\, …) and the internal i* commands are left out. Please check this list.
  • The shared corpus file doxygen.yaml carries the rule text and the full command list in its header, plus 6 cases:
    • doxygen_tags
    • doxygen_command_end
    • doxygen_command_is_case_sensitive
    • not_a_doxygen_comment (a -- comment is no Doxygen comment)
    • doxygen_need_ids
    • doxygen_with_other_start_sequence
  • A Doxygen tag line with a comma in a C/C++, C# or Go /** */ comment becomes a need with a garbage id #2171 then only has to widen the gate to its /**, /*! and /// forms.
  • test_doxygen_tag is the matcher's unit table, the same table as ubCode's.

3. Rebase

  • LINE_DELIMITERS gains -- and also --!. That goes beyond the finding: it mirrors //!, so a Doxygen --! run forms a block (vhdl_doxygen_run). It is easy to drop if unwanted.
  • The analyse.py hunk is gone.
  • VHDL reuses RUST_QUERY.
  • COMMENT_MARKERS is untouched.
  • The extraction snapshot is regenerated in the 🧪 sphinx-codelinks: snapshot the real extraction output, with marker scope association (moved from sphinx-codelinks#100) #2128 format.
  • uv.lock holds only the semantic hunk (+18 lines). The first version's fork-marker churn is gone. uv lock --check passes with the hook's uv 0.12.18.
  • The description is rewritten; the "@rst blocks work" claim is gone.

4. Declarative fixtures

  • The result in text scope tests in test_analyse_utils.py are not ported. All VHDL behaviour sits in the shared corpus:

    File Cases
    scope 10
    positions vhdl_line_comment_end (own line and after code), vhdl_no_newline_at_eof
    line_endings vhdl_crlf
    multiline_needs vhdl_dash_run, vhdl_doxygen_run
    need_refs vhdl_marker: -- and VHDL-2008 /* */
    doxygen 6
    extra_languages default_oneliner_vhdl
  • The shared discover_fixtures.json gains a vhdl_comment_type case (.vhd / .vhdl), so discovery is pinned too.

  • Also dropped from the first version: doc_test/vhdl_basic, tests/data/vhdl/demo.vhd and its test_analyse.py case. Bash and TypeScript have none either.

5. Grammar floor

tree-sitter-vhdl>=1.4.0. The whole suite (1103 tests, the shared corpus included) passes under 1.4.0 and under the locked 2.0.3.

Verified

  • uv run poe test-codelinks: 1103 passed, none skipped.
  • poe lint, poe typecheck and poe docs-codelinks (-nW) pass. CI: 36/36.
  • End to end: one project with one ubproject.toml, built with Sphinx + sphinx-codelinks and with ubc build html, gives the same one-line and multi-line needs and the same doxygen_tag warning on the same line.
  • ubCode side: useblocks/ubcode#3875 syncs this commit's snapshots. Its parity gate is green on all 242 shared cases, and goes red, naming the VHDL cases, when the Rust VHDL rule or the --! gate is broken on purpose.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pkg: sphinx-codelinks Concerns the sphinx-codelinks package (packages/sphinx-codelinks)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants