Repository navigation
fix(tests): seed tempfile's name generator per process - #4693
Conversation
tempfile draws names from fastrand, whose thread-local generator seeds from the time and the thread id alone (smol-rs/fastrand#130). Nextest runs each test in its own process, so processes that seed in the same clock tick generate the same names. A duplicate that hits an entry another process just deleted fails on Windows with PermissionDenied (os error 5), which tempfile does not retry. A stress run on windows-2025 with TMP=D:\tmp generated duplicate names in ~0.3% of creations across short-lived processes (0 of 30,000 on Linux), and all 27 Access-is-denied failures were on duplicated names. worktrunk::testing::{tempdir, named_tempfile} and test_tempdir now prefix names with the process id, every test call site uses them, and clippy's disallowed-methods keeps tempfile::tempdir, TempDir::new and NamedTempFile::new out of new tests. Closes #4692
|
@worktrunk-bot
|
format_migration_diff is production code, so it goes back to tempfile::tempdir() with a call-site expect on the disallowed-methods rule, which only targets the test suite. Co-Authored-By: Claude <noreply@anthropic.com>
|
I narrowed the rule and left the production code alone. In 3bceb98, None of the PR's other |
worktrunk-bot
left a comment
There was a problem hiding this comment.
HOST_MARKERS in tests/integration_tests/snapshot_formatting_guard.rs still matches tempfile names with \.tmp[A-Za-z0-9]{6}. The new names have the form .tmp<pid>-XXXXXX, so the pattern matches only when the pid has 6 or more digits. test_no_host_specific_paths_in_snapshots therefore stops catching an unredacted temp path in a committed snapshot whenever the generating process had a pid under 100000. That covers essentially every macOS run (pids top out at 99998) and most Windows runs. The file is outside the diff, so I'm pushing the fix: the pattern becomes \.tmp(?:\d+-)?[A-Za-z0-9]{6}, and its doc comment is updated to match.
…uard
worktrunk::testing's temp helpers now name entries .tmp<pid>-XXXXXX, which
\.tmp[A-Za-z0-9]{6} only matched for pids of six or more digits.
|
The old Any name that includes the pid has to fit in roughly the old 10 characters. Every way I found to do that gives up something:
Each one trades the collision fix against the documented socket budget, which is a call for a maintainer, so I haven't pushed any of them. The snapshot-guard fix from my review is in 4743ec0. Its own test passes locally. |
…g the pid The `.tmp<pid>-XXXXXX` names pushed macOS's fsmonitor socket test past sun_path (108 bytes, 103 allowed). Seeding fastrand's thread-local generator from RandomState, which draws OS randomness per process, separates names across processes while keeping tempfile's `.tmpXXXXXX` length. This reverts the snapshot guard change that matched the pid-prefixed form. Refs #4692
|
@worktrunk-bot Current-head
|
worktrunk-bot
left a comment
There was a problem hiding this comment.
Two of this head's checks fail because of the reseeding commit, and both are one-line fixes. I'm pushing them.
fast-checksfails inpre-merge project:doc:tempdir's public doc comment links to the privatetemp_builder, and-D warningsturnsrustdoc::private_intra_doc_linksinto an error.minimal-versionsfails atcargo update -Z direct-minimal-versions:fastrand = "2"resolves to 2.0.0, below the^2.1.1that tempfile 3.27 requires, so resolution fails.fastrand::seedexists in every 2.x, so the floor only needs to match the version that's locked.
minimal-versions resolved `fastrand = "2"` to 2.0.0, below tempfile's own 2.1.1 floor. Refs #4692
|
I fixed the doc link in 2f94714 by replacing the link with plain text. Locally, |
On Windows CI, temp entries under
D:\tmpfailed withAccess is deniedbecause concurrent test processes were generating the same names.tempfiletakes its names fromfastrand, and fastrand seeds its thread-local generator only fromInstant::now()and the thread id (smol-rs/fastrand#130). Under nextest each test is a separate process whose thread ids start from the same values, so two processes that seed in the same clock tick generate the same names. If the duplicate name is held by a live entry,tempfilegetsAlreadyExistsand retries. If another process has just deleted that entry, Windows returnsPermissionDenied(os error 5), whichtempfilepasses to the caller. That is the failure in #3613, #3619 and theTempDir::new()panic in #4692.The suite's temp helpers now reseed that generator per process:
worktrunk::testing::tempdir()andnamed_tempfile()replacetempfile::tempdir(),TempDir::new()andNamedTempFile::new()at every test call site, includingwt-perf.test_tempdir()and the fixture-template scratch directory go through the same builder. On a thread's first use, the builder callsfastrand::seedwith a value fromRandomState::new(), which std keys from OS randomness, so the seed differs between processes even when the clock reading matches. A probe confirmed that seeding fastrand fixes the next tempfile name, so both crates share the one generator. Names keep tempfile's.tmpXXXXXXlength, which matters because a pid prefix pushed the macOS fsmonitor socket path pastsun_path.clippy.tomlusesdisallowed-methodsto reject those three constructors in new code. The one production caller,format_migration_diff, keepstempfile::tempdir()with a call-site#[expect].fastrandbecomes a direct dependency. It resolves to the same 2.5.0 thattempfilealready uses.Windows reproduction
I ran a temporary workflow on
windows-2025withTMP=D:\tmp, set up the wayci.yamldoes it. Batches of short-lived processes each createdTempDirs andNamedTempFiles withtempfile3.27.0 and fastrand 2.5.0, and the parent recorded every name and error:Access is deniedAll 27 denials hit a name that another process had also generated. The same program on Linux generated no duplicates in 30,000 names.
This PR doesn't rerun that experiment against the new helpers either. The Windows
testleg on this PR is the check, and the recurrence rate (#3619 measured 4 failures in 80 loop runs) means one green leg is weak evidence.Locally, clippy is clean across the workspace, and the lib tests covering the directive helpers,
cache::and the template claim pass. An earlier fullcargo nextest run --all-featuresof the pid-prefix version passed 5473 of 5480, and the 7 failures fail the same way onmainin this sandbox. The sandbox ran out of disk before the full suite could run on the final version, so CI is the check for it.Not changed: production
wtalso createstempfilenames in shared directories, so concurrentwtprocesses on Windows share the same weak seed. The temp-index path already falls back to the Git directory on any error.Closes #4692 — automated triage