Support Python 3.14 and upgrade YAC to v3.18 (with DNN remapping) - #1563
Conversation
- upgrade YAC CI to v3.18.0 on Python 3.14 (numba>=0.63, py3.14 classifier, cython>=3.1 via conda for the bindings build) - fix add_average: reduction_type -> weight_type (renamed in YAC v3.18) - expose distance-nearest-neighbour (yac_method='dnn', new in YAC v3.15) + test - remove the obsolete 'pathlib' backport dependency: it shadows the stdlib and breaks tools that import pathlib on Python 3.10+ (broke YAC's Cython build)
c414783 to
e0989be
Compare
|
@MoritzHanke a quick YAC-side sanity check would be welcome — this bumps us to YAC v3.18 and exposes |
|
Hey, I'm sorry but I think u got the wrong guy... |
My bad, @nils3er can you take a look. |
|
Taking a look. Do you think the python version update will affect asv? The asv repo probably needs a refresh anyway @erogluorhan |
|
If you still need a review from me, please let me know. In the meantime I just skimmed quickly, and it looks reasonable. One tiny suggestion would be to mention #1561 in your comment on the numba>=0.63 tag in pyproject.toml, so that future contributors may find it easier to quickly understand why the version pin exists for numba but not other packages, by reading our longer discussion there. If you're otherwise ready to merge, though, don't worry about delaying just for this! |
|
Added the #1561 reference to the numba pin comment. Thanks @Sevans711. |
|
Thanks @cmdupuis3 and @Sevans711 for the reviews. |
* Honor quadrature kwargs in calculate_total_face_area calculate_total_face_area accepted quadrature_rule, order and latitude_adjusted_area but ignored them, always returning the cached default-parameter face_areas. As a result the gaussian/corrected path produced the same total as the triangular one. Keep the cached fast path for the default parameters (which also preserves the equal-area values used for HEALPix grids) and route any non-default quadrature settings through _compute_face_areas so the requested rule, order and latitude adjustment actually take effect. * Restore matplotlib backend after HoloViews matplotlib plot (#1538) * Restore matplotlib backend after HoloViews matplotlib plot plot(backend='matplotlib') calls hv.extension('matplotlib'), which switches the active matplotlib backend and clobbers the IPython inline display hook, silently breaking subsequent native matplotlib/xarray .plot() calls. Restore the original matplotlib backend right after the HoloViews extension switch; HoloViews objects still display via Store.current_backend, so this is safe. Closes #1537 * Address review: capture backend at switch, accurate docstring, effective test * Reconfigure IPython inline display hook when restoring backend mpl.use() restores the matplotlib backend name but does not re-register IPython's inline display integration that hv.extension('matplotlib') clobbers. In a Jupyter kernel without an explicit %matplotlib inline, native matplotlib/xarray .plot() calls after a uxarray matplotlib plot still failed to render. Re-run configure_inline_support when the restored backend is inline so the display hook is reinstated. See #1537. * Restore matplotlib backend via IPython shell reactivation to fix inline display --------- Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com> * Allow SCRIP reader to respect units w/r/t radians (#1433) * Allow SCRIP reader to respect units wrt to radians * Add test file and unit test for SCRIP radian coordinate handling * minor formatting changes for scrip radians fix Moves meshfiles/scrip/scrip_radians.nc to meshfiles/scrip/scrip_radians/scrip_radians_grid.nc to match style of other meshfiles naming schemes. Renames the new _scrip._convert_to_degrees() to _scrip._values_in_degrees(). It doesn't always convert; and it also does more than just converting, because it gives numpy array from DataArray. Clarified docstring. ran the following, so it should now pass ruff checks: pre-commit run --all-files --------- Co-authored-by: Sam Evans <s7evans11@gmail.com> Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com> Co-authored-by: Rajeev Jain <rajeeja@gmail.com> * Support Python 3.14 and upgrade YAC to v3.18 (with DNN remapping) (#1563) * Upgrade YAC to v3.18, expose DNN remapping, drop pathlib backport - upgrade YAC CI to v3.18.0 on Python 3.14 (numba>=0.63, py3.14 classifier, cython>=3.1 via conda for the bindings build) - fix add_average: reduction_type -> weight_type (renamed in YAC v3.18) - expose distance-nearest-neighbour (yac_method='dnn', new in YAC v3.15) + test - remove the obsolete 'pathlib' backport dependency: it shadows the stdlib and breaks tools that import pathlib on Python 3.10+ (broke YAC's Cython build) * Reference #1561 in numba version pin comment --------- Co-authored-by: Christopher Dupuis <45972964+cmdupuis3@users.noreply.github.com> * Bump actions/download-artifact in the actions group across 1 directory (#1572) Bumps the actions group with 1 update in the / directory: [actions/download-artifact](https://github.com/actions/download-artifact). Updates `actions/download-artifact` from 7 to 8 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v7...v8) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#1565) updates: - [github.com/astral-sh/ruff-pre-commit: v0.15.20 → v0.15.21](astral-sh/ruff-pre-commit@v0.15.20...v0.15.21) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Accusphere: gca_gca benchmarks and better arc sampling * Accusphere: fix sum_of_squares accuracy bug + tests * Accusphere: numba fma refactor and fix validation tautology * Accusphere: scalarized _counts_as_crossing --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: vakudo <127726617+vakudo@users.noreply.github.com> Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com> Co-authored-by: Rajeev Jain <rajeeja@gmail.com> Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com> Co-authored-by: zarzycki <colin.zarzycki@gmail.com> Co-authored-by: Sam Evans <s7evans11@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…tion, point-in-face, face bounds) (#1513) * Port AccuSphGeom EFT algorithms and add spherical geometry user guide Closes #1509 * Fix pre-commit: remove unused imports and variable * Fix pre-commit: split semicolons in notebook cells * Address Hongyu's review: port AccuSphGeom compensated arithmetic, rewrite intersections, add 241 baseline testsgit status! - most came from accusphere * Fix RTD: remove RST-invalid numbered list from gca_const_lat_intersection docstring * Fix RTD notebook kernel metadata * Address AccuSphGeom review feedback * Separate intersection kernels into three layers: numerical core, status/mask, dispatcher * Fix NaN/Inf propagation in intersection kernels for denom=0 and planar_sq<0 * Add geometry kernel benchmarks and to_raster auto-extent test - benchmarks/geometry_kernels.py: ASV micro-benchmarks for all three layers of the EFT intersection stack (_accux_gca, _try_gca_gca_intersection, gca_gca_intersection, _accux_constlat, _try_gca_const_lat_intersection, gca_const_lat_intersection) plus EFT primitives and point-in-polygon; all functions warmed before timing so results reflect steady-state cost - test/test_plot.py: add test_to_raster_auto_extent verifying that the axis limits change and the raster contains finite data * Address AccuSphGeom review: precision wording, dead code removal, correctness fixes Review comments addressed: - Remove "near-double precision" / "sufficient" overclaims; say "roughly twice as accurate" and note the robustness tier boundary clearly - Explain _lon_bounds_from_vertices is required for UXarray antimeridian encoding and cannot be removed - Add block comment before _no_extreme functions clarifying they are pre-existing edge screeners unrelated to the EFT stack - Document SoS as explicit future work in _point_in_polygon_sphere docstring - L2 pos_fin/neg_fin: replace ternary with int(); exploit neg=-pos symmetry - Label computation: drop dead local*0 term, use integer mask arithmetic - Remove vertex-lat snap from bounds: _face_location_info already captures interior arc extrema accurately via the compensated kernel - _ON_MINOR_ARC_TOL: document intentional 1e-10 vs C++ 1e-8 divergence Bug fixes: - on_minor_arc: add antipodal-endpoint guard; a x b = 0 for antipodal inputs so every point on the great circle passes the collinearity test (false pos) - bounds.py: replace mask arithmetic use_ext*z_ext + (1-use_ext)*z_edge with plain if/else; 0*NaN = NaN propagates when norm=0, if/else does not - _point_in_polygon_sphere: ray-nudge now restarts the loop from i=0 so all edges are counted with the same ray (mid-loop nudge corrupted crossing parity) Cleanup: - Remove _flip_sign, _SIGN_NEG, _SIGN_POS, _SIGN_ZERO dead code from point_in_face.py; inline literals in _counts_as_crossing - Remove _SNAP_TOL_DEG constant and snap_tol_deg parameter throughout bounds.py - Notebook: fix Grid.get_point_on_face -> get_faces_containing_point; remove incorrect geometry.py row from Section 4 table; add accucross_pair and acc_sqrt_re to Section 2 building-blocks table * Pin tornado<6.5.7 to fix Windows CI; remove dead shim; fix notebook API name - ci/environment.yml: pin tornado<6.5.7 to avoid ssl.SSLError in panel 1.9.3 on Python 3.11 Windows (conda-forge regression, 2026-06-10) - intersections.py: remove _gca_gca_intersection_cartesian shim (dead code); add comment explaining _snap_const_lat_endpoint snap_sq constant - test_intersections.py: update 4 call sites to use gca_gca_intersection directly - spherical-geometry-accuracy.ipynb: fix stale Grid.get_point_on_face -> Grid.get_faces_containing_point (2 occurrences) * Trim computing.py docstring; drop inline=always from L1 kernels * Revert tornado pin; root cause is openssl 3.6.3 on Windows * Port AccuSphGeom EFT algorithms and add spherical geometry user guide Closes #1509 * Fix pre-commit: remove unused imports and variable * Fix pre-commit: split semicolons in notebook cells * Address Hongyu's review: port AccuSphGeom compensated arithmetic, rewrite intersections, add 241 baseline testsgit status! - most came from accusphere * Fix RTD: remove RST-invalid numbered list from gca_const_lat_intersection docstring * Fix RTD notebook kernel metadata * Address AccuSphGeom review feedback * Separate intersection kernels into three layers: numerical core, status/mask, dispatcher * Fix NaN/Inf propagation in intersection kernels for denom=0 and planar_sq<0 * computing: use FMA two_prod where available (portable Veltkamp fallback) Add an LLVM fma intrinsic and route two_prod through a single fused multiply-add for its error term on hardware that supports it, selected at import time and validated to be bit-exact against the Veltkamp split. Falls back to the portable Veltkamp form otherwise, so there is no hard FMA dependency. The FMA path is ~2x faster in the compensated geometry kernels (each two_prod drops from ~17 flops to one FMADD) and is numerically identical: all 241 AccuSphGeom baseline cases pass unchanged. * intersections: add allocation-free scalar const-lat L1 kernel Add _accux_constlat_scalar, which takes the arc endpoints as six scalars and returns the candidate coordinates as scalars instead of two np.empty(3) arrays. _accux_constlat now wraps it so the array API is unchanged. Returning scalars lets Numba keep the candidates in registers, so a batch loop over many edges does no per-point heap allocation. On a 16M-point const-lat sweep this is ~2.7x faster than the array-returning path and drops the AccuX/FP64 cost ratio from ~19x to ~7x. Bit-identical results; all 241 AccuSphGeom baseline cases pass. * style: apply ruff formatting to computing.py * Clean up spherical geometry review items * Scalarize constant-latitude intersection dispatcher to remove per-call heap allocations * Import benchmark kernels inside setup to avoid asv collection failures Move the uxarray kernel imports from module level into each benchmark class's setup, so a stale or mismatched environment build only errors the affected benchmark instead of aborting collection of every benchmark in the directory. Thanks @cmdupuis3 for catching the asv import failure. * Cmd/accusphere (#1579) * Honor quadrature kwargs in calculate_total_face_area calculate_total_face_area accepted quadrature_rule, order and latitude_adjusted_area but ignored them, always returning the cached default-parameter face_areas. As a result the gaussian/corrected path produced the same total as the triangular one. Keep the cached fast path for the default parameters (which also preserves the equal-area values used for HEALPix grids) and route any non-default quadrature settings through _compute_face_areas so the requested rule, order and latitude adjustment actually take effect. * Restore matplotlib backend after HoloViews matplotlib plot (#1538) * Restore matplotlib backend after HoloViews matplotlib plot plot(backend='matplotlib') calls hv.extension('matplotlib'), which switches the active matplotlib backend and clobbers the IPython inline display hook, silently breaking subsequent native matplotlib/xarray .plot() calls. Restore the original matplotlib backend right after the HoloViews extension switch; HoloViews objects still display via Store.current_backend, so this is safe. Closes #1537 * Address review: capture backend at switch, accurate docstring, effective test * Reconfigure IPython inline display hook when restoring backend mpl.use() restores the matplotlib backend name but does not re-register IPython's inline display integration that hv.extension('matplotlib') clobbers. In a Jupyter kernel without an explicit %matplotlib inline, native matplotlib/xarray .plot() calls after a uxarray matplotlib plot still failed to render. Re-run configure_inline_support when the restored backend is inline so the display hook is reinstated. See #1537. * Restore matplotlib backend via IPython shell reactivation to fix inline display --------- Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com> * Allow SCRIP reader to respect units w/r/t radians (#1433) * Allow SCRIP reader to respect units wrt to radians * Add test file and unit test for SCRIP radian coordinate handling * minor formatting changes for scrip radians fix Moves meshfiles/scrip/scrip_radians.nc to meshfiles/scrip/scrip_radians/scrip_radians_grid.nc to match style of other meshfiles naming schemes. Renames the new _scrip._convert_to_degrees() to _scrip._values_in_degrees(). It doesn't always convert; and it also does more than just converting, because it gives numpy array from DataArray. Clarified docstring. ran the following, so it should now pass ruff checks: pre-commit run --all-files --------- Co-authored-by: Sam Evans <s7evans11@gmail.com> Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com> Co-authored-by: Rajeev Jain <rajeeja@gmail.com> * Support Python 3.14 and upgrade YAC to v3.18 (with DNN remapping) (#1563) * Upgrade YAC to v3.18, expose DNN remapping, drop pathlib backport - upgrade YAC CI to v3.18.0 on Python 3.14 (numba>=0.63, py3.14 classifier, cython>=3.1 via conda for the bindings build) - fix add_average: reduction_type -> weight_type (renamed in YAC v3.18) - expose distance-nearest-neighbour (yac_method='dnn', new in YAC v3.15) + test - remove the obsolete 'pathlib' backport dependency: it shadows the stdlib and breaks tools that import pathlib on Python 3.10+ (broke YAC's Cython build) * Reference #1561 in numba version pin comment --------- Co-authored-by: Christopher Dupuis <45972964+cmdupuis3@users.noreply.github.com> * Bump actions/download-artifact in the actions group across 1 directory (#1572) Bumps the actions group with 1 update in the / directory: [actions/download-artifact](https://github.com/actions/download-artifact). Updates `actions/download-artifact` from 7 to 8 - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@v7...v8) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: '8' dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#1565) updates: - [github.com/astral-sh/ruff-pre-commit: v0.15.20 → v0.15.21](astral-sh/ruff-pre-commit@v0.15.20...v0.15.21) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Accusphere: gca_gca benchmarks and better arc sampling * Accusphere: fix sum_of_squares accuracy bug + tests * Accusphere: numba fma refactor and fix validation tautology * Accusphere: scalarized _counts_as_crossing --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: vakudo <127726617+vakudo@users.noreply.github.com> Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com> Co-authored-by: Rajeev Jain <rajeeja@gmail.com> Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com> Co-authored-by: zarzycki <colin.zarzycki@gmail.com> Co-authored-by: Sam Evans <s7evans11@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Add thread-scaling benchmark for FP64 vs AccuX constlat dispatcher * Cap thread-scaling sweep at performance cores to avoid E-core artifact * Fix thread-scaling plot ticks to show only 1,2,4,8 * o Fix pre-commit * o doc fixes * o Remove benchmark image from repo * Split out point-in-face and lat-lon bounds to a separate PR * o computing: match AccuSphGeom acc_sqrt_re exactly (residual order, branch-free) * o intersections: compensated norm in AccuXGCA kernel, branch-free mask validity per AccuSphGeom * o arcs: branch-free on_minor_arc int mask at tol 1e-8 matching AccuSphGeom * o benchmarks: drop point-in-polygon kernel bench, moved to point-in-face PR * o benchmarks: drop thread-scaling constlat, moved to separate scaling issue * o docs: move spherical-geometry notebook out, depends on split-out point-in-face * o docs: drop spherical-geometry notebook from user-guide toctree * o test_plot: drop to_raster_auto_extent, unrelated to AccuXGCA/AccuXConstLat scope * o fix on_minor_arc antipodal false positive * o add on_minor_arc degeneracy regression tests * fix diff_of_products addition grouping to match AccuSphGeom eft.hpp * o intersections: collapse pos/neg-valid branches via XOR in gca_const_lat_intersection * arcs: keep normal hi/lo split through _normal_dot_value via compensated dot * arcs: fix comments overclaiming branch-free form and degeneracy-check robustness * arcs: stop claiming on_minor_arc mirrors AccuSphGeom, it's an approximation --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Hongyu Chen <hyvchen@ucdavis.edu> Co-authored-by: Christopher Dupuis <45972964+cmdupuis3@users.noreply.github.com> Co-authored-by: vakudo <127726617+vakudo@users.noreply.github.com> Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com> Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com> Co-authored-by: zarzycki <colin.zarzycki@gmail.com> Co-authored-by: Sam Evans <s7evans11@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Upgrades the optional YAC backend to v3.18.0 (built on Python 3.14).
numba>=0.63and py3.14 classifierInterpolationStack.add_averagearg renamedreduction_type->weight_typein YAC v3.18yac_method='dnn'(added in YAC v3.15), with test