Skip to content

Build abi3 wheels so installs don't recompile on macOS/Windows#198

Merged
alxmrs merged 2 commits into
mainfrom
claude/mac-release-compilation-2f3gj9
Jun 29, 2026
Merged

Build abi3 wheels so installs don't recompile on macOS/Windows#198
alxmrs merged 2 commits into
mainfrom
claude/mac-release-compilation-2f3gj9

Conversation

@alxmrs

@alxmrs alxmrs commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Problem

Installing on macOS (and Windows) triggers a full local Rust compilation instead of using a pre-built wheel — see #163. On an M1 Mac with Python 3.12/3.13, uv/pip find no matching wheel and fall back to building the whole datafusion/arrow/pyo3 stack from the sdist.

Root cause: PyPI only had cp311 wheels for macOS/Windows, while Linux had cp310cp313:

cp311-cp311-macosx_11_0_arm64.whl   ← only Python 3.11
cp311-cp311-win_amd64.whl           ← only Python 3.11
cp310/311/312/313 ...manylinux...   ← all versions (built with -i in maturin-action)

The mac/win job in publish.yml runs a bare maturin build, which only targets the runner's single interpreter (3.11, from .python-version). The manylinux jobs avoided this by enumerating interpreters with -i python3.10 python3.11 python3.12 python3.13.

Fix

Enable PyO3's stable ABI (abi3-py310) — the same approach datafusion-python uses for this exact pyo3 + arrow + datafusion-ffi stack. Each platform now ships a single cpXX-abi3 wheel that installs on all CPython ≥ 3.10, including future versions. This closes the per-version coverage gap and collapses 12+ wheels into 5.

  • Cargo.toml — add abi3-py310 to pyo3's features. maturin auto-detects abi3 and tags wheels cp310-abi3. No change needed to the mac/win job.
  • publish.yml — manylinux jobs no longer need to enumerate interpreters; one -i python3.10 produces the abi3 wheel.
  • publish.yml — gate upload-to-pypi on github.event_name == 'release' so the workflow can be run via workflow_dispatch to build/verify wheels as artifacts without an irreversible PyPI publish. (Previously a manual dispatch would have published.)

Verification

  • cargo check with abi3-py310: passes — the extension uses only abi3-safe APIs (pyclass, pymethods, PyCapsule, extract, try_iter).
  • maturin build --release: produced xarray_sql-0.3.0-cp310-abi3-manylinux_2_34_x86_64.whl (single abi3 wheel, was cp311-cp311 before).
  • Installed that 3.10-tagged wheel into a clean Python 3.11 venv with --only-binary :all: (compilation forbidden) and imported the native module successfully — proving cross-version install with no rebuild.

CI (ci-build.yml) will exercise maturin build with abi3 on macOS, Windows, and Linux runners.

Closes #163

🤖 Generated with Claude Code


Generated by Claude Code

claude added 2 commits June 29, 2026 13:27
The release workflow built macOS and Windows wheels with a bare
`maturin build`, which only targets the runner's single Python
interpreter (3.11). As a result PyPI only had cp311 wheels for
macOS/Windows, while Linux had cp310-cp313 (built via `-i` in
maturin-action). Users on other Python versions — e.g. an M1 Mac on
3.12 — found no matching wheel and fell back to compiling the Rust
sources from sdist on install (issue #163).

Enable PyO3's stable ABI (`abi3-py310`) so each platform ships a single
`cpXX-abi3` wheel that installs on all CPython >= 3.10, including future
releases. This is the same approach datafusion-python uses for the same
pyo3 + arrow + datafusion-ffi stack. The manylinux jobs no longer need
to enumerate every interpreter, so they build with a single `-i
python3.10`. The mac/win job needs no change: maturin auto-detects abi3
from Cargo.toml.

Verified with `cargo check`: the extension only uses abi3-safe APIs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDK8JtQyRQDdwJMgqADbe5
Gate the upload-to-pypi job on `release` events so the publish workflow
can be run manually (workflow_dispatch) to build and verify wheels as
artifacts without performing an irreversible PyPI upload.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDK8JtQyRQDdwJMgqADbe5
@alxmrs
alxmrs merged commit 8bd0063 into main Jun 29, 2026
12 checks passed
@alxmrs
alxmrs deleted the claude/mac-release-compilation-2f3gj9 branch June 29, 2026 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xarray-sql is slow to install with uv in a new python environment on an M1 mac

2 participants