Skip to content

./uw build should verify numpy ↔ petsc4py ↔ Cython-extension ABI consistency (and rebuild on mismatch) #308

Description

@lmoresi

Problem

After the numpy≥2 dependency bump (#301/#304/#305) landed on development, import underworld3 fails on an environment whose petsc4py and compiled UW3 Cython extensions were built against numpy 1.x. The failures are cryptic and happen at import/first-use, not at build time:

# petsc4py (built against old numpy):
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
  ...  in init petsc4py.PETSc

# UW3 extension (built against old numpy), later at runtime:
ImportError: numpy.core.multiarray failed to import
  ... in init underworld3.function._dminterp_wrapper

Both the development main checkout and a worktree env hit this. ./uw build completed "successfully" yet left an unimportable install — its post-build import check aborted at petsc4py before recompiling, and pip install . did not rebuild the locally-built petsc4py binding.

What actually fixed it (manual, per-env)

  1. Rebuild the petsc4py binding against the installed numpy:
    pip install --no-build-isolation --force-reinstall --no-deps $PETSC_DIR/src/binding/petsc4py
  2. Clean-rebuild the UW3 extensions: rm -rf build/lib.* build/temp.* && ./uw build

Ask

This is an environment/build-consistency issue, not a source bug — so ./uw build (and/or ./uw doctor) should detect and handle it:

  • Detect when the installed numpy differs from the numpy petsc4py / the compiled UW3 extensions were built against (e.g. compare recorded build-time numpy ABI, or a cheap import-probe of petsc4py.PETSc after build).
  • On mismatch, rebuild petsc4py + force a clean extension rebuild, or at minimum fail loudly with an actionable message ("numpy X installed but petsc4py built against Y — run …") instead of leaving a broken install that errors cryptically at first use.
  • The post-build import check should exercise a real path (e.g. uw.function.evaluate), not just import underworld3, so extension-ABI breaks surface at build time.

Environment

  • numpy 2.4.6 (pin >=2,<2.5), PETSc 3.25 (petsc-325-uw-openmpi), locally-built petsc4py 3.25.0.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions