Skip to content

Make the Test Suite Pass or Skip Cleanly on Windows #2488

Closed

Description

@ptr727

What happens

python3 -m unittest discover -s tests on a Windows 11 host, at develop e96cb58, reports Ran 2192 tests, FAILED (failures=47, errors=32, skipped=66). Nothing runs the suite on Windows, so none of these show up in CI, which runs only on ubuntu-latest.

This is the measurement behind #1143's open question, whether the fleet's tooling gets Windows coverage, now taken over the whole suite instead of the two tools that question started from.

By module

Module Failing
test_prose_lint 20
test_release_guards 14
test_local_review 14
test_build_dist 10
test_bootstrap 6
test_spec_validate 3
test_host_gate 3
test_docker_lint 3
test_skills_install 2
test_publish_plan 2
test_carry 2

By cause, where the error names one

  • Symlink creation needs a privilege, 16. OSError: [WinError 1314] A required privilege is not held by the client. Without Developer Mode or an elevated shell, Windows refuses os.symlink, and every test that builds a symlink fixture errors instead of skipping. These are spread across several modules, test_carry's two remaining errors among them.
  • A script run as an executable, 6. OSError: [WinError 193]. These are the six test_bootstrap installer tests, fixed by #2484 for Skip the Linux Installer Tests in test_bootstrap Where install-tools.sh Cannot Execute #2473.
  • Git Bash's bash rejects read -a with no array name, 8. /bin/bash: line 1: read: -a: option requires an argument, in tests that run a workflow step's shell through bash.
  • Workflow-step harnesses returning a different exit or output, the rest. For example ('0', 'labeled=true') != ('2', '') in test_prose_lint's label tests, and semver2 parsing in test_release_guards. These need reading one by one, since each may be a Windows defect in the tool or a test assuming a POSIX shell.
  • test_carry's registration and change-line failures are fixed by #2485 for scripts/carry.py's worktree-registration check can never match on Windows #1117.

Done means

  • Each failing test either passes on Windows, or skips there with a reason that names the platform boundary, the way test_bootstrap's Linux-only classes already do.
  • The symlink class is settled once, for example a shared helper that skips when os.symlink raises WinError 1314, rather than per test.
  • Whether CI gains a Windows test job stays Fleet Tooling Assumes a POSIX Host, So It Fails on the Windows Hosts the Fleet Supports #1143's decision, and this issue supplies its cost.

Activity

  1. added 2 commits that reference this issue on Oct 8, 2026
    9eda5a1
    31c4bbd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions