Skip to content

fix(evals): support cross-drive scaffold imports on Windows - #459

Open
legend91019 wants to merge 1 commit into
Tencent:mainfrom
legend91019:fix/windows-cross-drive-eval-scaffold
Open

legend91019 wants to merge 1 commit into
Tencent:mainfrom
legend91019:fix/windows-cross-drive-eval-scaffold

Conversation

@legend91019

@legend91019 legend91019 commented Oct 17, 2026 •

Copy link
Copy Markdown

Why

Fixes #458.

On Windows, path.relative() returns an absolute path when the generated badcase directory and the repository helper are on different drives. The scaffold code treated that result as a relative path and prefixed ./, so generated fixtures could not import fixtures.mjs.

This is an eval-only path-resolution fix and does not change BrowserSkill runtime behavior.

What changed

  • Resolve the target helper once.
  • Preserve the existing relative specifier on the same filesystem root.
  • Fall back to pathToFileURL() when the relative result is absolute, which covers cross-drive Windows paths with a valid ESM specifier.

Verification

  • On the unmodified d9a78ae baseline with the repository on D: and %TEMP% on C:, the focused test failed (0 passed, 1 failed); the full browser-eval suite reported 22 passed, 1 failed.
  • Moving TEMP/TMP to the repository drive made the same unmodified focused test pass (1/1).
  • On this branch, Node v22.22.2 focused test: 1 passed, 0 failed.
  • Full browser eval: 23 passed, 0 failed; 10 cases and 19 fixture routes validated.
  • corepack pnpm exec biome check evals/browser/lib/scaffold-case.mjs
  • node --check evals/browser/lib/scaffold-case.mjs
  • git diff --check

No screenshot is included because this is a deterministic Node/Windows path-resolution failure with no UI change. The complete Chinese evidence, independent re-verification, and full before/after code are maintained in the local workflow artifact D:\Desktop\open_coding\pr_reviews\Tencent__BrowserSkill__dogfood__windows-cross-drive-eval-scaffold.md and can be provided to maintainers on request.

@NianJiuZst NianJiuZst left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, approving.

The fix is correct. On Windows, path.relative() gives back an absolute path when the two paths are on different drives. Detecting that with isAbsolute() and falling back to pathToFileURL(targetPath).href produces a valid ESM import. When both paths are on the same drive, the generated specifier is the same relative path as before, so cases scaffolded inside the repo don't change.

What I checked:

  • Ran the specifier logic with path.win32. A cross-drive pair (C:\…\Temp\… → D:\repo\…\fixtures.mjs) gives an absolute result and file:///D:/repo/evals/browser/lib/fixtures.mjs. A same-drive pair still gives ../../../lib/fixtures.mjs.
  • node --test evals/browser/tests/eval.test.mjs on this branch: 15/15 pass (Linux, Node 22).
  • Biome is clean on the changed file, and there are no runtime/product changes.

Non-blocking: CI runs the eval tests only on Ubuntu, so the cross-drive path has no automated test. If we want to guard it, moduleSpecifier could take the path functions as a parameter, so a test can call it with path.win32 and a cross-drive pair. Fine as a follow-up.

@legend91019

Copy link
Copy Markdown
Author

你是马哥吗

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Windows: badcase scaffold fails when TEMP and the repository are on different drives

2 participants