ENH: implement support for build-details.json (PEP 739) (continued)

[mgorny]

This a continuation of #779. For a start, I just did a dumb rebase to rerun the tests.

[rgommers]

I closed gh-779. Copying over the main comment from that PR discussion:

Good point. Yes, there isn't any reason that build-details.json cannot be used for a native build. So we should support that as well. That said, there isn't a benefit from the user passing in such a file explicitly; we can start reading it automatically for Python >=3.14, but I'm a bit hesitant to do that right away since it's so new. We can start doing that at any point in the future if it helps simplify our internal code, since it's a backwards-compatible change.

So for now in this PR we can distinguish three cases, with the user passing:

1. Only a cross file (or multiple cross files)
2. Only a build-details.json file
3. Both a cross file and a build-details.json file

I focused on (3) first and have a TODO in here already to add (1); I should add (2) as well.

[dnicolodi]

I had a quick look at this. Found some possible improvements.

[mgorny]

I'm going to rebase it now, and make changes later when I find more time.

[mgorny]

Updated and rebased now.

[dnicolodi]

I tied up some loose ends. I think this is ready to be merged now. @rgommers, do you want to take a look?

It would be nice to have an integration test for this, but I don't know how hard it is to get a suitable environment setup on GitHub Actions.

[mgorny]

Actually, I think it was in Draft because we wanted to make an integration test, but we were blocked on build-details.json problems in conda packages. I need to recheck if we resolved them all now.

[rgommers]

Sounds great! Indeed, that integration test would be nice. IIRC conda-forge/python-feedstock#858 should have solved the one issue in conda-forge that was blocking for a cross-compile test.

[dnicolodi]

This still uses _32_BIT_INTERPRETER as determined from the running interpreter and not from build-details.toml.

[mgorny]

Ok, disabled it when using build details. I suppose it's fine to assume that if you're configuring cross via build-details.json, you'd be targeting the bitness from the file.

I've also adjusted the logic for macosx, though I have to say that the logic there still relies on the running interpreter; I really don't know much about macOS, and I don't understand the idea behind versions there, but on the test box I have access to mac_ver() returns 26.x, while platform is macosx-11.0-arm64; and I don't really see a way of getting that 26 out of build-details.json.

[dnicolodi]

I've also adjusted the logic for macosx, though I have to say that the logic there still relies on the running interpreter

In this case, it is better to emit an error stating that build-details.json is not supported on macOS rathe than silently emitting wrong wheel tags. The latter would be difficult to debug.

[dnicolodi]

I really don't know much about macOS, and I don't understand the idea behind versions there, but on the test box I have access to mac_ver() returns 26.x, while platform is macosx-11.0-arm64; and I don't really see a way of getting that 26 out of build-details.json.

macOS and iOS may introduce ABI incompatible changes at each release. However, a specific ABI can be requested setting some environment variables during compilation. Therefore, the platform tag contains a system ABI version part that is derived from the current OS version, unless the relevant environment variable is set. I am not sure what is the best way to map this to the use of build-details.json. When compiling a redistributable wheel on macOS it is customary to set the env var to have the toolchain compile for the macOS ABI supported by the relevant Python interpreter, however, it is not a requirement. I think that returning an error if the detected macOS ABI version and the one in build-details.json differ may be a good starting point. For the unit test, you can just set the env variable to what the given Python version expects.

[rgommers]

There is nothing in build-details.json that has a relationship with the minimum macOS version, it's orthogonal.

For the unit test, you can just set the env variable to what the given Python version expects.

This should always be the right answer for tests. And at runtime, nothing needs to change to MACOSX_DEPLOYMENT_TARGET logic. Only the platform part of the tag (e.g., x86_64 or arm64) can come from build-details.json.