chore: update dependencies, fix nix builds, speed up mutation gate - #119
Merged
Merged
Conversation
Rust: tree-sitter 0.26 -> 0.27, tree-sitter-language-pack 1.14 -> 1.20
(the pack's 1.19+ line requires tree-sitter ^0.27). Declared MSRV
1.85 -> 1.90 to match tree-sitter 0.27's floor; the raised MSRV
unlocked clippy's msrv-gated collapsible_if, so two nested ifs became
let-chains (classify.rs, main.rs) — semantics unchanged, mutation score
stays 118/118 (113 caught, 5 unviable, 0 missed). flake.nix parser-sources
pin moved to v1.20.0 with the matching sha256.
npm: effect + @effect/platform-node 4.0.0-rc.112 -> rc.118 (peer-matched
pair), oxlint 1.86.0, turbo 2.10.12 -> 2.11.5, @systemfsoftware/tsconfig
1.3.3 -> 2.0.1, tsdown 0.23.0. rc.118 moved effect/unstable/{process,cli}
to effect/process + effect/cli and renamed Flag.string -> Flag.String, so
the launcher imports were updated. turbo 2.11.5 appends its managed
agent-guidance block to AGENTS.md; kept committed per turbo's guidance.
Deno: scripts/ and hooks/ imports refreshed (effect/platform-deno rc.118,
@std/*, arktype 2.2.5, @libs/diff 4.0.1) with regenerated lockfiles.
Tracked as a none changeset — no observable behaviour change.
…ed nix leak flake.nix: tree-sitter-language-pack 1.20.0's build.rs hard-requires a `<bundle>.sha256` sidecar, so the previous commit's bare fetchurl bundle failed to build. Serve the bundle from a runCommand dir with a sidecar written from the same pinned digest (tslpVersion/tslpSha256 are now the single source). Also export HOME inside the build: cargo defaulted CARGO_HOME to nix's HOME=/homeless-shelter, which without a sandbox is the real host path, so every rebuild after the first failed nix's purity check (and with it direnv and the repo's PostToolUse hook). flake.lock: nixpkgs and rust-overlay updated to latest. hook_commands tests: pass XDG_CACHE_HOME through to the sandboxed deno. The warm-cache probe runs with the full environment, so on hosts with XDG_CACHE_HOME set it checked a different cache than the tests used.
@effect/tsgo 0.47.0 was inside pnpm's minimum release age during the previous refresh; it is now installable. hooks/run.ts imports only @std/io and arktype; @std/fs and @std/path were declared in hooks/deno.jsonc but never used. Removed, and deno.lock regenerated from run.ts's real import graph (deno cache --frozen passes).
Mutation concurrency lives in the root `mutants` script: -j comes from CARGO_MUTANTS_JOBS, defaulting to half of nproc (getconf fallback for macOS). CI sets CARGO_MUTANTS_JOBS=$(nproc). turbo passes the variable through without hashing it, so CI and local runs share cached verdicts. AGENTS.md now names `pnpm mutants` as the gate so direct runs get the policy instead of cargo-mutants' single-job default. devShell: stdenv exports LD_FOR_BUILD, and Deno refuses to spawn under a scoped --allow-run while any LD_* var is set, so every scripts/tools/*.ts that shells out (lint-workflows, plan-release, ...) failed inside the dev shell. Unset it in shellHook.
turbo runs tasks in strict env mode, so //#mutants dropped XDG_CACHE_HOME and DENO_DIR. On a host that sets either, deno then looked for hooks/run.ts's cached deps under $HOME/.cache/deno, missed the warmed cache, and the unmutated baseline failed seven hook_commands tests before any mutant ran. Neither variable changes a verdict, so pass them through unhashed.
…command tree-sitter-language-pack 1.15.0+ never finds its own OUT_DIR parser cache when TSLP_LANGUAGES is set: the probe checks `_parsers/<lang>/src/parser.c` while the bundle extracts to `_parsers/parsers/<lang>/...`. Every build script run therefore re-downloads and re-extracts the 36 MB bundle, and the extraction bumps the mtime of `_parsers/patches/`, which the script watches with rerun-if-changed. So every cargo command recompiled all 37 grammars (~22-26s even for a no-op build) and needed network access, and each mutant paid it twice. 1.14.3 had the same probe but did not watch patches/; staying on it would ship the python/yaml/markdown scanner heap-overflow fixes that 1.15+ patches in. Setting PROJECT_ROOT at a pre-extracted bundle makes build.rs take its workspace path instead. It applies the patches once (they are idempotent) and then stays fresh: no-op cargo commands drop to ~50ms. - devShell: extract the flake's hash-pinned bundle once into $XDG_CACHE_HOME/comment-checker/tslp-<version> and export PROJECT_ROOT, only when Cargo.lock names the pinned version; a stale root would compile another release's grammar sources silently. - CI: .github/actions/tslp-project-root derives the version from Cargo.lock, verifies the bundle against the release's .sha256 sidecar (as build.rs does), and exports PROJECT_ROOT. Used by rust-gate here and by mutation in the next commit.
- [profile.mutants] (debug = "none"), selected in .cargo/mutants.toml: debug info only slows mutant builds, and a caught mutant never needs a backtrace. Full gate 144s -> 115s. - CPU policy: cargo-mutants' jobserver already allows one compile task per CPU across all -j jobs, so `-j nproc/2` never capped CPU. --jobserver-tasks now does: half the CPUs locally, all of them in CI (CARGO_MUTANTS_JOBSERVER_TASKS). -j defaults to 2 everywhere: under a 4-token cap, -j2 ran the full gate in 102s against 114s for -j4. - CI: cargo-mutants is pinned and built with --locked, and the binary is cached by version instead of compiled from source on every run. - mutation.yml provisions PROJECT_ROOT (previous commit); turbo passes it and the jobserver variable through unhashed, and hashes .cargo/mutants.toml.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rust, npm, and Deno dependencies move to their latest mutually compatible versions, and the classifier mutation gate drops from over 17 minutes to under 2. The gate was slow because
tree-sitter-language-packrecompiled every grammar on every cargo command. With that fixed, no-opcargocommands take about 50ms instead of about 25s. Two Nix problems are fixed along the way: flake builds on hosts without a Nix sandbox no longer fail on every rebuild, and the Deno tools underscripts/tools/now run inside the dev shell.Version decisions
tree-sittertree-sitter-language-pack1.19+tree-sitter-language-packrust-version)tree-sitter0.27 declares 1.90effect,@effect/platform-nodelatesttag is 3.22.2@systemfsoftware/tsconfigtsc -bpasses unchangedturbo@effect/tsgo,tsdown,oxlint,@types/node, the Deno imports inscripts/andhooks/, andflake.lock(nixpkgs, rust-overlay) move to latest as well.@std/fsand@std/pathare dropped fromhooks/deno.jsonc: nothing inhooks/run.tsimports them.Why not stay on
tree-sitter-language-pack1.14.3Every release from 1.15.0 on has the rebuild loop described below, so staying on 1.14.3 would avoid it without any workaround. It would also keep three known heap-buffer overflows in grammars this binary compiles and runs on every agent write. The
python,yaml, andmarkdownexternal scanners write past tree-sitter's fixed serialization buffer on deeply nested or indented input, and the only bounds check compiles away in release builds. 1.15.0+ patches all three.The rebuild loop
With
TSLP_LANGUAGESset, the crate's build script looks for its cache atOUT_DIR/_parsers/<lang>/src/parser.c, but the bundle extracts to_parsers/parsers/<lang>/…. The check never matches, so every build-script run re-downloads the 36 MB bundle and re-extracts it. The re-extraction bumps the mtime of_parsers/patches/, which the script watches withrerun-if-changed, so the next cargo command runs the build script again. This is the same class of bug upstream fixed in xberg-io/tree-sitter-language-pack#158.Pointing
PROJECT_ROOTat a pre-extracted bundle sends the build script down its workspace path instead. It applies the grammar patches once (they are idempotent) and then stays fresh.$XDG_CACHE_HOME/comment-checker/tslp-<version>and exportsPROJECT_ROOT, but only whenCargo.locknames the pinned version. A stale root would compile another release's grammar sources without any error..github/actions/tslp-project-rootaction reads the version fromCargo.lock, verifies the bundle against the release's.sha256sidecar the same waybuild.rsdoes, and exportsPROJECT_ROOT.rust-gateandmutationuse it.PROJECT_ROOT(for example, the release builds inplatform.yml), builds fall back to the old path: slow, but correct.Mutation setup
[profile.mutants]withdebug = "none"-j 2under a 4-task jobserver cap, instead of-j 4-jjobs, so the earlier-j nproc/2never capped CPU at all.--jobserver-tasksnow does: half the CPUs locally, all of them in CI (CARGO_MUTANTS_JOBSERVER_TASKS).-jdefaults to 2 everywhere, because more parallel mutants measured slower.--locked, and the binary is cached by version, instead of being compiled from source on every run.XDG_CACHE_HOMEandDENO_DIR. On any host that sets either, sevenhook_commandstests failed in the unmutated baseline, so the gate never tested a mutant. turbo now passes those, the jobserver variable, andPROJECT_ROOTthrough without hashing them. It hashes.cargo/mutants.toml.Measured and not adopted: the mold linker (125s against 115s without it), nextest (the suite runs in about 0s), and CI sharding (a full run is now under 2 minutes).
Code the updates forced
effect/unstable/processandeffect/unstable/clitoeffect/processandeffect/cli, and renamedFlag.stringtoFlag.String. The launcher's imports follow.collapsible_iflint, which fails two nestedifs under-D warnings. They become let-chains inclassify.rsandmain.rs, with identical control flow.tree-sitter-language-pack1.20.0 refuses a parser-source bundle with no.sha256file beside it.flake.nixnow writes one from the same pinned digest.Nix fixes
HOME=/homeless-shelteras a real directory. Every later build then failed with "home directory /homeless-shelter exists", which also broke direnv and the repo's PostToolUse hook. The build now sets a privateHOME. This fix is visible to consumers, so it carries apatchchangeset.LD_FOR_BUILD, and Deno refuses to spawn a subprocess under a scoped--allow-runwhile anyLD_*variable is set.shellHooknow unsets it.AGENTS.mdTwo edits to this locked file, both at the maintainer's direction. turbo 2.11.5 appends its managed agent-guidance block and re-adds it on every agent run, so the block is committed. The mutation gate is now documented as
pnpm mutants, and the concurrency rules describe the jobserver policy above.Release
.changeset/update-deps.mdisnone, and.changeset/flake-build-home.mdispatch. Merging opens a release PR for 0.3.6.Validation
All of the following ran against the pushed head (
95f1bc6d1), inside the dev shell:cargo fmt --check,cargo clippy --all-targets -- -D warnings, andcargo test --all-targets(128 tests) pass.pnpm gate:mutantstested 118 mutants: 113 caught, 5 unviable, 0 missed, in 104s.pnpm lint,pnpm build, andpnpm typecheckpass with turbo's cache bypassed, andpnpm install --frozen-lockfilepasses. The built launcher prints--helpand names the missing platform package.deno lint,deno check tools/*.ts,check-matrix,lint-workflows, anddeno cache --frozenfor the hook all pass.nix build .#comment-checkerbuilds and leaves no/homeless-shelter. The built binary flags# increment iabovei += 1as a restatement.nix flake check --no-buildpasses.PROJECT_ROOT.cargo install --locked cargo-mutants@27.1.0succeeds.Not run: the Nix build on aarch64 or Darwin, and the CI workflows themselves, including the cached-install path.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.