Skip to content

pre-commit fails or rewrites files outside a change set, so clean PRs need --no-verify or drag in other experiments #60

Closed

Description

@jwm4

From Bill Murdock, with assistance from Claude Code.

What happens

On current main (2026-10-06), the pre-commit configuration does not pass on the existing tree, and two of its hooks act on files outside the change being committed:

  • ty check is configured with pass_filenames: false, so it type-checks the whole repository on every commit. It currently reports 8 errors in pre-existing code (0008-guardrails-eval/eval-model-armor.py, 0024-codeowners-app-matrix/run_matrix.py, 0025-subagent-process-isolation/.fullsend/scripts/post-emit-cost.py, 0026-eval-statistical-significance/significance.py and its test). A commit that adds only a Markdown file therefore fails the hook and can only land with --no-verify.
  • pre-commit run --all-files, the command AGENTS.md gives for running all hooks manually, reformats 11 Python files in experiments 0025, 0026 and agent-eval-mlflow-otel via ruff format, and surfaces 10 bandit findings in the same areas. A contributor who runs that command before committing, then stages their work, ends up with a PR that rewrites other people's experiments.

Observed while opening #59, whose first push carried 11 unrelated reformatted files for exactly this reason; it took a force-push to remove them.

What should happen

A contributor who follows AGENTS.md and touches only their own experiment should get a passing pre-commit run and a PR containing only their files.

Context

The experiment-specific lints (lint-experiment-numbers, -frontmatter, -index) and the secret scanners pass on the current tree; the failures are confined to the Python formatting and type-check hooks and to code that predates them. Related: #38 (tests for the lint scripts).

Activity

  1. ifireball commented on Oct 15, 2026

    @ifireball
    Member

    /fs-triage

  2. fullsend-ai-triage commented on Oct 15, 2026

    @fullsend-ai-triage

    🤖 Finished Triage · ✅ Success · Started 6:13 AM UTC · Completed 6:16 AM UTC

    Commit: 3f1614f · View workflow run →

    Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $0.60

  3. fullsend-ai-triage commented on Oct 15, 2026

    @fullsend-ai-triage

    Triage summary

    The repository config matches your description. The local ty hook passes the hard-coded target . with pass_filenames: false, so it ignores which files are staged. AGENTS.md also points contributors to --all-files, which makes ruff, ruff-format and bandit run across every experiment.

    One small correction: the ty hook declares types: [python], so pre-commit should skip it for a commit that stages only Markdown files. The real failure is that a commit staging any Python file type-checks the whole repo and fails on the 8 pre-existing errors. The fix is the same either way. The ty errors and formatting drift could not be reproduced here because the triage sandbox cannot download hook environments, so the counts in the issue are taken as reported.

    Severity: medium. Workarounds exist (--no-verify, or manually unstaging unrelated files), but both weaken the hooks, and every contributor who touches Python is affected.

    Recommended fix

    1. .pre-commit-config.yaml: scope the ty hook to staged files. Remove the . target from the entry and set pass_filenames: true, keeping types: [python].
    2. AGENTS.md (Linting section): recommend pre-commit run (staged files) or pre-commit run --files <paths> instead of --all-files, and note that --all-files will reformat and flag pre-existing code in other experiments.
    3. Optional follow-up for maintainers: a separate one-time cleanup (ruff format, fix the ty errors, handle the bandit findings) so --all-files passes again. It touches other people's experiments, so it is better kept out of the main fix.

    No CI workflow changes are needed, since no workflow runs pre-commit.

    Proposed test case

    In a clean clone of main:
    (a) Stage only a new, type-correct .py file plus README/index updates in a new
        experiment directory; run `pre-commit run`.
        Expect: all hooks pass; no files modified outside the staged set.
    (b) Stage a .py file with a deliberate type error; run `pre-commit run`.
        Expect: ty fails and reports only that file.
    (c) Stage only a Markdown change; run `pre-commit run`.
        Expect: ty is skipped; all hooks pass.
    Edge cases: a staged .py importing a sibling module in the same experiment
    (imports still resolve); a staged file under a dotted dir like .fullsend/scripts/.
    

    Related


    Next steps:

    • /fs-code — agent creates a PR to implement this issue
    • /fs-code <your instruction> — agent implements with your specific guidance
  4. fullsend-ai-coder commented on Oct 15, 2026

    @fullsend-ai-coder
    Contributor

    🤖 Finished Code · ✅ Success · Started 6:18 AM UTC · Completed 6:22 AM UTC

    Commit: 3f1614f · View workflow run →

    Runtime: claude · Model: opus → claude-opus-5-5 · Effort: high · Cost: $0.66

  5. added a commit that references this issue on Nov 15, 2026
    fe218d1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingready-to-codeTriggers code agent dispatch

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions