Repository navigation
Session Handoff [auto-1910]: Context-Tracking Fallback Tokenizer for the Write Guard #1957
Description
Activity
Parked: waiting on decision #1960
Done. Branch
feature/auto-1910is pushed at 63f3417, with no pull request yet. It replaces the guard's comment strip, its_COMMENT_SCAN_BAILskip list, and its union with the per-line base tokens with_context_lex, a context-tracking scan inhost-setup/agent-safety/claude/gh-write-guard.py. Every constructed example in #1910 and its comments is a selftest case, and the multi-line quote no longer draws a false deny. A token-level case list,_CONTEXT_LEX_CASES, covers the scan directly, and every targeted mutation of the scan fails the selftest. ruff, mypy, and the pre-commit gates pass.Why it parked. Three local strict review rounds found hiding regressions against develop, where develop denies, the branch allows, and bash runs the command. The rounds found 4, then 8, then 3. Two rounds of fixes closed the first twelve, each with a selftest case that fails on the previous commit. The two-round edit budget is spent with three still open:
- A
#right after a subshell)or an arithmetic))is read as text. The fix is to allow it only after a)that closes$(...), both at the top level (thecmd[i - 1 : i] != ")"test in_context_lex) and in_skip_group. caseandesacare counted in_skip_groupwherever they start a word, when they should count only in command position._closes_as_arithmeticis quadratic on a long run of(.
Remaining. Follow the answer on #1960. Under option 1, restore a union with develop's per-line fallback tokens in
_shell_tokensafter_context_lexsucceeds, fix item 3, adjust the one wait case that expectsallowfor the multi-line quote, then drive the PR withCloses on promotion: #1910removed, since #1910 stays open for the false deny. Under option 2, fix items 1 to 3 and run another review. Under option 3, delete the branch.Worktree
/home/pieter/repos/worktrees/ProjectTemplate-auto-1910is left standing, clean, at the pushed head.Filed along the way: #1958 and #1959, pre-existing gaps in the same fallback. #1836's three cases already deny on develop. #1843's reproduction denies on this branch. Neither issue was closed.
- A
Decision #1960 answered by the maintainer: option 1, keep the new scan and restore the union with develop's fallback tokens, fix the quadratic scan, and drive the pull request; #1910 stays open for the multi-line-quote false deny. The
blockedlabel is removed, so this lane can resume with that answer.Parked: waiting on decision #2023
Done. This round followed the answer on #1960 (option 1). Branch
feature/auto-1910is pushed at 751d058, and there is no pull request yet. Develop is merged in at 0766531._shell_tokensreturns develop's fallback reading,_line_fallback_tokens, followed by the context scan's tokens._check_unbounded_waitjudges the command on two heredoc strips, one by the scan's_heredoc_openerand one by_line_heredoc_opener, which is develop's reading._closes_as_arithmeticis linear, through_paren_closers, built once per scan.- The multi-line-quote wait case now expects
deny, and Read a Multi-Line Quote Whole in the Guard's Token Fallback #1910 stays open for that false deny. - The selftest passes, and a mutation of each mechanism fails a case. ruff, mypy, and the commit hooks pass. A differential fuzz against develop found no deny-to-allow regression on this head.
Why it parked. This push's two-round budget is spent. Pass 1 found 3 introduced findings and pass 2 found 1, and all of those are fixed. Pass 3 found 1 more introduced finding, which is still open:
_redirects_stdincan read the"\n"that joins the two readings as a dangling redirect's target. #2023 has the constructed command and the one-line fix. The receipt for pass 3 is recorded at the pushed head.Remaining. Follow the answer on #2023.
- Option 1: refuse a shell-operator token as a redirect target in
_redirects_stdinand add Grant One More Review Round for the Guard's Context Scan Branch #2023's command as a wait case. Then run a fourth pass, open the pull request withCloses on promotion: #1910left out, and drive it to develop. - Option 2: judge each reading separately in every rule, rather than one joined token list.
- Option 3: delete the branch.
Worktree.
/home/pieter/repos/worktrees/ProjectTemplate-auto-1910is left standing and clean at the pushed head.Filed along the way: #2023. No lesson was filed.
Next steps, in priority order
host-setup/agent-safety/claude/gh-write-guard.py. Replace the shell-token fallback's line-by-line re-tokenizing and its_COMMENT_SCAN_BAILskip list with a context-tracking scan that carries quote state across lines, handles backslash escapes and the$(,${,$'and$"forms directly, finds a comment only outside quotes, and treats#as text inside a heredoc, arithmetic, an extglob, and a[[ =~ ]]operand. Done looks like: every constructed example in Read a Multi-Line Quote Whole in the Guard's Token Fallback #1910's body and comments is tested, the multi-line quote no longer draws a false deny, the three loop shapes in the comments are denied, the existing guard tests stay green, and a mutation of the new scan fails a test. Drive the pull request to develop withCloses on promotion: #1910.External blockers
None known at creation.
Internal dependencies
State
Re-derive each of these at resume.
defaultandpython-contractare the maintainer's attended lanes.The parked decision queue
No open issue carries
decisionat creation.What the last round did
Nothing. This link was created by an unattended-handoff picker from the open backlog.
What not to repeat
#Comment Defeats Requirement 7 Entirely #1635 found a context the list missed, per Read a Multi-Line Quote Whole in the Guard's Token Fallback #1910's body. Track context instead.New learnings
None yet.