Repository navigation
Make the Test Suite Pass or Skip Cleanly on Windows #2488
Copy link
Copy link
Closed
Description
What happens
python3 -m unittest discover -s tests on a Windows 11 host, at develop e96cb58, reports Ran 2192 tests, FAILED (failures=47, errors=32, skipped=66). Nothing runs the suite on Windows, so none of these show up in CI, which runs only on ubuntu-latest.
This is the measurement behind #1143's open question, whether the fleet's tooling gets Windows coverage, now taken over the whole suite instead of the two tools that question started from.
By module
| Module | Failing |
|---|---|
test_prose_lint |
20 |
test_release_guards |
14 |
test_local_review |
14 |
test_build_dist |
10 |
test_bootstrap |
6 |
test_spec_validate |
3 |
test_host_gate |
3 |
test_docker_lint |
3 |
test_skills_install |
2 |
test_publish_plan |
2 |
test_carry |
2 |
By cause, where the error names one
- Symlink creation needs a privilege, 16.
OSError: [WinError 1314] A required privilege is not held by the client. Without Developer Mode or an elevated shell, Windows refusesos.symlink, and every test that builds a symlink fixture errors instead of skipping. These are spread across several modules,test_carry's two remaining errors among them. - A script run as an executable, 6.
OSError: [WinError 193]. These are the sixtest_bootstrapinstaller tests, fixed by #2484 for Skip the Linux Installer Tests in test_bootstrap Where install-tools.sh Cannot Execute #2473. - Git Bash's bash rejects
read -awith no array name, 8./bin/bash: line 1: read: -a: option requires an argument, in tests that run a workflow step's shell through bash. - Workflow-step harnesses returning a different exit or output, the rest. For example
('0', 'labeled=true') != ('2', '')intest_prose_lint's label tests, andsemver2parsing intest_release_guards. These need reading one by one, since each may be a Windows defect in the tool or a test assuming a POSIX shell. test_carry's registration and change-line failures are fixed by #2485 for scripts/carry.py's worktree-registration check can never match on Windows #1117.
Done means
- Each failing test either passes on Windows, or skips there with a reason that names the platform boundary, the way
test_bootstrap's Linux-only classes already do. - The symlink class is settled once, for example a shared helper that skips when
os.symlinkraisesWinError 1314, rather than per test. - Whether CI gains a Windows test job stays Fleet Tooling Assumes a POSIX Host, So It Fails on the Windows Hosts the Fleet Supports #1143's decision, and this issue supplies its cost.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
Test the Windows bash Selection on Any Host, and Name an Unusable bas…
Probe Each PATH Directory for bash.exe, and Record the Skipped Link S…
Take the Link Case Last in the Two Mixed build_dist Tests
Let shutil.which Answer for bash Off Windows
Join a Stub Directory Onto PATH With os.pathsep in Two configure Tests
Resolve the jq Stub's Directory in bash Before Prepending It
Skip a Symlink Test Where Windows Withholds the Privilege, and Run ba…
Promote develop to main: Skip or Run the Test Suite Cleanly on Windows (
Absorb a Windows Closed Pipe and Remove Read-Only Staged Objects in l…
Pin validate.py's UTF-8 Output on Every Host, and Keep the Remedy Ass…
Write spec/validate.py Output as UTF-8, and Fix Four Windows-Shaped T…
Promote develop to main: Make the Test Suite Pass on Windows (#2528)
Run the Test Suite on windows-latest in the Pull Request Gate (#2555)