HttpStress execution failure on Windows

HttpStress tests are failing on Windows since 02.17:

The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.

https://dev.azure.com/dnceng-public/public/_build/results?buildId=180524&view=logs&j=0bc77094-9fcd-5c38-f6e4-27d2ae131589&t=4fd93899-d50b-5dc1-f9eb-66ab1b883c22

Weirdly, this happens after the actual run is complete.

@dotnet/dnceng was there any change around the 17th that might have led to this?

Windows images were updated on the 24th (dotnet/arcade#12486) but nothing around the 17th.

@ilyas1974

Do you know what Docker version runs in the Nano server @ilyas1974 ?

@mthalman

Do you know VMs the docker nano server container is running on? Otherwise, you can check dotnet-buildtools-prereqs-docker to see what version of docker is installed on the docker containers hosting nano.

/cc: @mthalman

From what I can tell, it's not running this in a container from dotnet-buildtools-prereqs-docker. It's running compose directly from the build agent which is from the 1es-windows-2022-open pool.

I don't know if we can make a helix repro vm directly from the 1es-windows-2022-open pool, but if this job is running docker compose, I wonder if you could just do a docker version to figure out this question. I don't know what is baked into those images, as they come to us from 1es

@antonfirsov

maybe draft PR @antonfirsov ? I think we can then adjust the version in docker-compose.yml. It seems like we now ask for version '3'.

It also seems like the recommendation is to use both major and minor version:
https://docs.docker.com/compose/compose-file/compose-versioning/

Some quick points:

For any 1es-* image, the version can "float" even when we're not doing rollouts. Thus, when folks notice differences it's best to check out the image version found inside the "Initialize Job" phase. Relevant snippets:

From your failed run:

Image: 1es-windows-2022-open
Image Version: 51.0.0
Pool Name: NetCore-Public
Current image version: '20230214.1'

From the pipeline's last successful run:

Image: 1es-windows-2022-open
Image Version: 50.0.0
Pool Name: NetCore-Public
Current image version: '20230129.1'

... this shows us that indeed the image version being updated is a prime suspect for what broke you.

For these 1es-windows-2022 images, the "starting point" is https://github.com/actions/runner-images/blob/main/images/win/Windows2022-Readme.md . This is the best place to go when something updated and broke you with these guys.
Reviewing the PR that changed this doc on the image version noted, indeed Docker Compose gets updated from 2.15.1 -> 2.16.0
Finally, for pedantry's sake, 1es-windows-2022-open is an image name, not a pool (the pool is NetCore-Public)

I think we are hitting actions/runner-images#7080 which is still happening, despite the issue being closed. We can try the workarounds recommended in the discussion, or wait for an upstream fix.

I think we are hitting actions/runner-images#7080 which is still happening, despite the issue being closed. We can try the workarounds recommended in the discussion, or wait for an upstream fix.

If transitioning to windows.vs2022.amd64 / windows.vs2022.amd64.open works, that's a fine workaround for as long as you like.

Navigation Menu

Repository navigation

HttpStress execution failure on Windows #82528

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

HttpStress execution failure on Windows #82528

Description

Activity

antonfirsov commented on Feb 24, 2023

ilyas1974 commented on Feb 24, 2023

wfurt commented on Feb 24, 2023

ilyas1974 commented on Feb 24, 2023

mthalman commented on Feb 24, 2023

michellemcdaniel commented on Feb 24, 2023

wfurt commented on Feb 24, 2023

MattGal commented on Feb 27, 2023

antonfirsov commented on Feb 27, 2023

MattGal commented on Feb 27, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions