Repository navigation
Close the Walk-Arounds in the Block Scanner's Whitespace-Predicate Guard #1618
Description
scripts/tests/test_release_guards.py carries test_no_bare_whitespace_predicate_in_the_block_scanner, added because six silent defects in the AUDIT.md fence scanner were one habit: a Python whitespace or line predicate taking a wider class than a renderer does, each written while fixing the previous one. The guard reads the scanner's own source and fails on such a predicate. A review of it found three ways it does not hold, the first of them verified by a mutation that changes behaviour and passes the whole suite.
The guard does not reach the module-level pattern
inspect.getsource(function) returns a function body, so the module-level FENCE_LINE regex, which holds the scanner's only whitespace-class decision, is never read. Verified: "FENCE_LINE = " in scanned_source is False.
Changing that pattern's indent group from (?P<indent> {0,3}) to (?P<indent>\s{0,3}) passes the entire suite, and the mutant is not equivalent. On "\t```bash\nA=1\n\t```\n" it returns a block where a reference CommonMark implementation returns none, since a tab-indented line is an indented code block rather than a fence. That is exactly the defect class the guard exists to stop, three lines above the function it guards.
The set of functions it reads is hard-coded and unverified
The tuple of functions is a literal, and mutating it to read only fenced_blocks passes the suite. A predicate moved into shell_blocks, or into a new module-level helper, is then caught only by whichever behaviour case happens to cover it. Three such walk-arounds were tried and all three were caught, but by the refusal cases over tab, no-break space and form feed body lines rather than by the guard. The docstring's claim that every function the scanner is built from is read is false in both directions, since module-level code is read in neither.
The wide-class half is decorative
The guard also looks for \s written into a pattern, and deleting that half passes the suite: no case in the file contains one for it to find.
Smaller, in the same file
- "Every comparison here names the characters it means" is false by design, since the label split carries an inline exemption and the test asserts exactly one such exemption exists.
- The docstring's refusal rule reads as though four leading spaces would be read, where the indent is capped at three, and it omits the refusal of a backtick or tilde after the run. The module comment states the rule correctly, so the docstring is the surface that drifted.
- "dedented out of the list item holding it" describes one case, where a top-level fence indented one to three spaces is dedented by the same rule.
- The refusal loop's
lineshadows the scan loop'sline. Harmless as written, and the kind of footing several fix rounds in this file have slipped on, since a statement added after it would read the clobbered name and look correct.
Why this is filed rather than fixed
The branch that added the guard answered its own decisive question, that the scanner cannot silently drop a labelled block or hand back a body that changes what bash -n decides, and these are gaps in the guard rather than defects in the scanner. Reading module-level source and verifying the function set are a change to the guard's own design and deserve their own review rather than a fourteenth commit on that branch.
Activity
ptr727 commentedon Sep 14, 2026
Two more from later reviews of the same guard, both the same shape as the three above: the guard reaches less than its prose implies.
The wide-class half reaches one spelling of the class.
WIDE_CLASSisre.compile(r"\\s"), so\Sand a set built from either, such as[^\S], pass the detector. Under the defaultre.UNICODEthose are the same class, so a blank test written that way treats a no-break space, a form feed and U+2028 as blank, which is the sixth defect's shape exactly. Verified:bare_whitespace_predicates(' if re.search(r"\\S", line):')returns no offender. Rewriting either existing call site that way is still caught by the behaviour cases, so this is a gap in the guard rather than a live defect, and the guard is the layer meant to catch a newly written comparison before its behaviour case exists."Every comparison here names the characters it means" is false by design, and two docstrings in the file disagree about it. The scanner's docstring states that universal; the guard's own docstring calls the label split "a comparison needing a wider class than the characters it names" and the test asserts exactly one such exemption exists. One surface states the rule and the other states the exception, using the same word for both.
A consequence worth recording, since it will surprise whoever hits it: the escape hatch cannot be used for a prose mention. Adding a sentence to the scanner's docstring that contains the parenthesised spelling of a bare call fails the guard, and marking that line with the exemption marker then fails the count assertion instead. A documentation edit must avoid the spelling or the rule needs a notion of a mention as distinct from a call.
Diagnostics lost in the refactor. Lifting the detector into a function dropped the line number from the offender tuple, so a failure names the offending text without saying where it is, and two identical offending lines in one function report once. That mutation survives the suite.
Recording these here rather than fixing them on #1619, whose scanner the reviews confirmed cannot silently drop a labelled block or alter a body, because each is a change to the guard's own design.
Say What the Guard Reads and What the Dedent Uses, Rather Than More T…
Pin the Refusals a Document Would Actually Reach, and Say What the De…
Read Every Fence Shape With One Scanner, and Hold the Rule That Six S…
Promote the AUDIT.md Section 6 Fixes and the Fence Scanner to Main (#…
ptr727 commentedon Oct 8, 2026
Declined per #2517, approved by the maintainer, under GOVERNANCE.md "Trust Boundaries and Hardening Effort": It hardens a test that guards a scanner, against an input nothing produces.