Skip to content

Flaky: ExecutionTimeTests.Completes_Within_Unhappy* exceed 1s wall-clock bound on macOS CI #7159

Closed

Description

@thomhurst

What happened

ExecutionTimeTests.Completes_Within_Unhappy and Completes_Within_Unhappy2 failed on the modularpipeline (macos-latest) job of PR #7139. That PR only changes assertion source generators, so it did not cause the failure.

failed Completes_Within_Unhappy2 (2s 361ms)
  AssertionException: Expected to be less than 00:00:01
  but received 00:00:01.3215153
  at Assert.That(duration).IsLessThan(TimeSpan.FromSeconds(1))
failed Completes_Within_Unhappy (2s 246ms)
  but received 00:00:01.1886545

Why it is flaky

Each test sets a CompletesWithin(250ms) timeout. It then asserts that the whole wall-clock duration, measured with a Stopwatch, stays under 1s. That duration also includes the throw/catch path and the message-regex assertion. On a loaded macOS arm64 runner it reached 1.19s and 1.32s, even though the 250ms timeout fired correctly.

Reproduce

Run TUnit.Assertions.Tests on a busy or slow machine, for example the macOS GitHub runner, using --treenode-filter "/*/*/ExecutionTimeTests/*".

Expected

The tests should check that CompletesWithin stops waiting well before the 10s action finishes. They should not depend on a tight 1s wall-clock bound. A bound such as < 5s still proves the timeout fired early and leaves room for CI jitter.

Activity

  1. thomhurst commented on Oct 20, 2026

    @thomhurst
    OwnerAuthor

    Seen again on PR #7154 macOS: Completes_Within_Unhappy2 (2s 609ms, Expected to be less than 00:00:01). https://github.com/thomhurst/TUnit/actions/runs/39356018996/job/118411284592

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions