Build abi3 wheels so installs don't recompile on macOS/Windows#198
Merged
Conversation
The release workflow built macOS and Windows wheels with a bare `maturin build`, which only targets the runner's single Python interpreter (3.11). As a result PyPI only had cp311 wheels for macOS/Windows, while Linux had cp310-cp313 (built via `-i` in maturin-action). Users on other Python versions — e.g. an M1 Mac on 3.12 — found no matching wheel and fell back to compiling the Rust sources from sdist on install (issue #163). Enable PyO3's stable ABI (`abi3-py310`) so each platform ships a single `cpXX-abi3` wheel that installs on all CPython >= 3.10, including future releases. This is the same approach datafusion-python uses for the same pyo3 + arrow + datafusion-ffi stack. The manylinux jobs no longer need to enumerate every interpreter, so they build with a single `-i python3.10`. The mac/win job needs no change: maturin auto-detects abi3 from Cargo.toml. Verified with `cargo check`: the extension only uses abi3-safe APIs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DDK8JtQyRQDdwJMgqADbe5
Gate the upload-to-pypi job on `release` events so the publish workflow can be run manually (workflow_dispatch) to build and verify wheels as artifacts without performing an irreversible PyPI upload. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DDK8JtQyRQDdwJMgqADbe5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Installing on macOS (and Windows) triggers a full local Rust compilation instead of using a pre-built wheel — see #163. On an M1 Mac with Python 3.12/3.13,
uv/pipfind no matching wheel and fall back to building the whole datafusion/arrow/pyo3 stack from the sdist.Root cause: PyPI only had
cp311wheels for macOS/Windows, while Linux hadcp310–cp313:The mac/win job in
publish.ymlruns a barematurin build, which only targets the runner's single interpreter (3.11, from.python-version). The manylinux jobs avoided this by enumerating interpreters with-i python3.10 python3.11 python3.12 python3.13.Fix
Enable PyO3's stable ABI (
abi3-py310) — the same approach datafusion-python uses for this exact pyo3 + arrow + datafusion-ffi stack. Each platform now ships a singlecpXX-abi3wheel that installs on all CPython ≥ 3.10, including future versions. This closes the per-version coverage gap and collapses 12+ wheels into 5.Cargo.toml— addabi3-py310to pyo3's features. maturin auto-detects abi3 and tags wheelscp310-abi3. No change needed to the mac/win job.publish.yml— manylinux jobs no longer need to enumerate interpreters; one-i python3.10produces the abi3 wheel.publish.yml— gateupload-to-pypiongithub.event_name == 'release'so the workflow can be run viaworkflow_dispatchto build/verify wheels as artifacts without an irreversible PyPI publish. (Previously a manual dispatch would have published.)Verification
cargo checkwithabi3-py310: passes — the extension uses only abi3-safe APIs (pyclass,pymethods,PyCapsule,extract,try_iter).maturin build --release: producedxarray_sql-0.3.0-cp310-abi3-manylinux_2_34_x86_64.whl(single abi3 wheel, wascp311-cp311before).--only-binary :all:(compilation forbidden) and imported the native module successfully — proving cross-version install with no rebuild.CI (
ci-build.yml) will exercisematurin buildwith abi3 on macOS, Windows, and Linux runners.Closes #163
🤖 Generated with Claude Code
Generated by Claude Code