Skip to content

Support Python 3.14 and upgrade YAC to v3.18 (with DNN remapping) - #1563

Merged
rajeeja merged 4 commits into
mainfrom
rajeeja/upgrade_yac_318
Jul 14, 2026
Merged

Support Python 3.14 and upgrade YAC to v3.18 (with DNN remapping)#1563
rajeeja merged 4 commits into
mainfrom
rajeeja/upgrade_yac_318

Conversation

@rajeeja

@rajeeja rajeeja commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Upgrades the optional YAC backend to v3.18.0 (built on Python 3.14).

  • CI: YAC v3.14 -> v3.18.0, Python 3.11 -> 3.14; numba>=0.63 and py3.14 classifier
  • fix: InterpolationStack.add_average arg renamed reduction_type -> weight_type in YAC v3.18
  • new: expose distance-nearest-neighbour via yac_method='dnn' (added in YAC v3.15), with test

- upgrade YAC CI to v3.18.0 on Python 3.14 (numba>=0.63, py3.14 classifier,
  cython>=3.1 via conda for the bindings build)
- fix add_average: reduction_type -> weight_type (renamed in YAC v3.18)
- expose distance-nearest-neighbour (yac_method='dnn', new in YAC v3.15) + test
- remove the obsolete 'pathlib' backport dependency: it shadows the stdlib and
  breaks tools that import pathlib on Python 3.10+ (broke YAC's Cython build)
@rajeeja
rajeeja force-pushed the rajeeja/upgrade_yac_318 branch from c414783 to e0989be Compare July 13, 2026 18:30
@rajeeja
rajeeja requested a review from cmdupuis3 July 13, 2026 18:55
@rajeeja

rajeeja commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

@MoritzHanke a quick YAC-side sanity check would be welcome — this bumps us to YAC v3.18 and exposes add_dnn; does our usage look right?

@MoritzHanke

Copy link
Copy Markdown

Hey, I'm sorry but I think u got the wrong guy...

@rajeeja rajeeja mentioned this pull request Jul 13, 2026
6 tasks
@rajeeja

rajeeja commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Hey, I'm sorry but I think u got the wrong guy...

My bad, @nils3er can you take a look.

@rajeeja
rajeeja requested a review from Sevans711 July 14, 2026 16:22
@rajeeja rajeeja changed the title Upgrade YAC to v3.18 and expose DNN remapping Support Python 3.14 and upgrade YAC to v3.18 (with DNN remapping) Jul 14, 2026
@cmdupuis3

Copy link
Copy Markdown
Collaborator

Taking a look. Do you think the python version update will affect asv? The asv repo probably needs a refresh anyway @erogluorhan

@Sevans711

Copy link
Copy Markdown
Collaborator

If you still need a review from me, please let me know. In the meantime I just skimmed quickly, and it looks reasonable. One tiny suggestion would be to mention #1561 in your comment on the numba>=0.63 tag in pyproject.toml, so that future contributors may find it easier to quickly understand why the version pin exists for numba but not other packages, by reading our longer discussion there. If you're otherwise ready to merge, though, don't worry about delaying just for this!

@rajeeja

rajeeja commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Added the #1561 reference to the numba pin comment. Thanks @Sevans711.

@rajeeja

rajeeja commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @cmdupuis3 and @Sevans711 for the reviews.

@rajeeja
rajeeja merged commit 51b9bc6 into main Jul 14, 2026
14 checks passed
@rajeeja
rajeeja deleted the rajeeja/upgrade_yac_318 branch July 14, 2026 19:02
rajeeja added a commit that referenced this pull request Jul 16, 2026
* Honor quadrature kwargs in calculate_total_face_area

calculate_total_face_area accepted quadrature_rule, order and
latitude_adjusted_area but ignored them, always returning the cached
default-parameter face_areas. As a result the gaussian/corrected path
produced the same total as the triangular one.

Keep the cached fast path for the default parameters (which also preserves
the equal-area values used for HEALPix grids) and route any non-default
quadrature settings through _compute_face_areas so the requested rule,
order and latitude adjustment actually take effect.

* Restore matplotlib backend after HoloViews matplotlib plot (#1538)

* Restore matplotlib backend after HoloViews matplotlib plot

plot(backend='matplotlib') calls hv.extension('matplotlib'), which switches
the active matplotlib backend and clobbers the IPython inline display hook,
silently breaking subsequent native matplotlib/xarray .plot() calls. Restore
the original matplotlib backend right after the HoloViews extension switch;
HoloViews objects still display via Store.current_backend, so this is safe.

Closes #1537

* Address review: capture backend at switch, accurate docstring, effective test

* Reconfigure IPython inline display hook when restoring backend

mpl.use() restores the matplotlib backend name but does not re-register
IPython's inline display integration that hv.extension('matplotlib')
clobbers. In a Jupyter kernel without an explicit %matplotlib inline,
native matplotlib/xarray .plot() calls after a uxarray matplotlib plot
still failed to render. Re-run configure_inline_support when the restored
backend is inline so the display hook is reinstated. See #1537.

* Restore matplotlib backend via IPython shell reactivation to fix inline display

---------

Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com>

* Allow SCRIP reader to respect units w/r/t radians (#1433)

* Allow SCRIP reader to respect units wrt to radians

* Add test file and unit test for SCRIP radian coordinate handling

* minor formatting changes for scrip radians fix

Moves meshfiles/scrip/scrip_radians.nc to meshfiles/scrip/scrip_radians/scrip_radians_grid.nc to match style of other meshfiles naming schemes.

Renames the new _scrip._convert_to_degrees() to _scrip._values_in_degrees(). It doesn't always convert; and it also does more than just converting, because it gives numpy array from DataArray. Clarified docstring.

ran the following, so it should now pass ruff checks: pre-commit run --all-files

---------

Co-authored-by: Sam Evans <s7evans11@gmail.com>
Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com>
Co-authored-by: Rajeev Jain <rajeeja@gmail.com>

* Support Python 3.14 and upgrade YAC to v3.18 (with DNN remapping) (#1563)

* Upgrade YAC to v3.18, expose DNN remapping, drop pathlib backport

- upgrade YAC CI to v3.18.0 on Python 3.14 (numba>=0.63, py3.14 classifier,
  cython>=3.1 via conda for the bindings build)
- fix add_average: reduction_type -> weight_type (renamed in YAC v3.18)
- expose distance-nearest-neighbour (yac_method='dnn', new in YAC v3.15) + test
- remove the obsolete 'pathlib' backport dependency: it shadows the stdlib and
  breaks tools that import pathlib on Python 3.10+ (broke YAC's Cython build)

* Reference #1561 in numba version pin comment

---------

Co-authored-by: Christopher Dupuis <45972964+cmdupuis3@users.noreply.github.com>

* Bump actions/download-artifact in the actions group across 1 directory (#1572)

Bumps the actions group with 1 update in the / directory: [actions/download-artifact](https://github.com/actions/download-artifact).


Updates `actions/download-artifact` from 7 to 8
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v7...v8)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [pre-commit.ci] pre-commit autoupdate (#1565)

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.20 → v0.15.21](astral-sh/ruff-pre-commit@v0.15.20...v0.15.21)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Accusphere: gca_gca benchmarks and better arc sampling

* Accusphere: fix sum_of_squares accuracy bug + tests

* Accusphere: numba fma refactor and fix validation tautology

* Accusphere: scalarized _counts_as_crossing

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: vakudo <127726617+vakudo@users.noreply.github.com>
Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com>
Co-authored-by: Rajeev Jain <rajeeja@gmail.com>
Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com>
Co-authored-by: zarzycki <colin.zarzycki@gmail.com>
Co-authored-by: Sam Evans <s7evans11@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
rajeeja added a commit that referenced this pull request Jul 28, 2026
…tion, point-in-face, face bounds) (#1513)

* Port AccuSphGeom EFT algorithms and add spherical geometry user guide

Closes #1509

* Fix pre-commit: remove unused imports and variable

* Fix pre-commit: split semicolons in notebook cells

* Address Hongyu's review: port AccuSphGeom compensated arithmetic, rewrite intersections, add 241 baseline testsgit status! - most came from accusphere

* Fix RTD: remove RST-invalid numbered list from gca_const_lat_intersection docstring

* Fix RTD notebook kernel metadata

* Address AccuSphGeom review feedback

* Separate intersection kernels into three layers: numerical core, status/mask, dispatcher

* Fix NaN/Inf propagation in intersection kernels for denom=0 and planar_sq<0

* Add geometry kernel benchmarks and to_raster auto-extent test

- benchmarks/geometry_kernels.py: ASV micro-benchmarks for all three
  layers of the EFT intersection stack (_accux_gca, _try_gca_gca_intersection,
  gca_gca_intersection, _accux_constlat, _try_gca_const_lat_intersection,
  gca_const_lat_intersection) plus EFT primitives and point-in-polygon;
  all functions warmed before timing so results reflect steady-state cost
- test/test_plot.py: add test_to_raster_auto_extent verifying that the
  axis limits change and the raster contains finite data

* Address AccuSphGeom review: precision wording, dead code removal, correctness fixes

Review comments addressed:
- Remove "near-double precision" / "sufficient" overclaims; say "roughly twice
  as accurate" and note the robustness tier boundary clearly
- Explain _lon_bounds_from_vertices is required for UXarray antimeridian
  encoding and cannot be removed
- Add block comment before _no_extreme functions clarifying they are
  pre-existing edge screeners unrelated to the EFT stack
- Document SoS as explicit future work in _point_in_polygon_sphere docstring
- L2 pos_fin/neg_fin: replace ternary with int(); exploit neg=-pos symmetry
- Label computation: drop dead local*0 term, use integer mask arithmetic
- Remove vertex-lat snap from bounds: _face_location_info already captures
  interior arc extrema accurately via the compensated kernel
- _ON_MINOR_ARC_TOL: document intentional 1e-10 vs C++ 1e-8 divergence

Bug fixes:
- on_minor_arc: add antipodal-endpoint guard; a x b = 0 for antipodal inputs
  so every point on the great circle passes the collinearity test (false pos)
- bounds.py: replace mask arithmetic use_ext*z_ext + (1-use_ext)*z_edge with
  plain if/else; 0*NaN = NaN propagates when norm=0, if/else does not
- _point_in_polygon_sphere: ray-nudge now restarts the loop from i=0 so all
  edges are counted with the same ray (mid-loop nudge corrupted crossing parity)

Cleanup:
- Remove _flip_sign, _SIGN_NEG, _SIGN_POS, _SIGN_ZERO dead code from
  point_in_face.py; inline literals in _counts_as_crossing
- Remove _SNAP_TOL_DEG constant and snap_tol_deg parameter throughout bounds.py
- Notebook: fix Grid.get_point_on_face -> get_faces_containing_point; remove
  incorrect geometry.py row from Section 4 table; add accucross_pair and
  acc_sqrt_re to Section 2 building-blocks table

* Pin tornado<6.5.7 to fix Windows CI; remove dead shim; fix notebook API name

- ci/environment.yml: pin tornado<6.5.7 to avoid ssl.SSLError in panel 1.9.3
  on Python 3.11 Windows (conda-forge regression, 2026-06-10)
- intersections.py: remove _gca_gca_intersection_cartesian shim (dead code);
  add comment explaining _snap_const_lat_endpoint snap_sq constant
- test_intersections.py: update 4 call sites to use gca_gca_intersection directly
- spherical-geometry-accuracy.ipynb: fix stale Grid.get_point_on_face ->
  Grid.get_faces_containing_point (2 occurrences)

* Trim computing.py docstring; drop inline=always from L1 kernels

* Revert tornado pin; root cause is openssl 3.6.3 on Windows

* Port AccuSphGeom EFT algorithms and add spherical geometry user guide

Closes #1509

* Fix pre-commit: remove unused imports and variable

* Fix pre-commit: split semicolons in notebook cells

* Address Hongyu's review: port AccuSphGeom compensated arithmetic, rewrite intersections, add 241 baseline testsgit status! - most came from accusphere

* Fix RTD: remove RST-invalid numbered list from gca_const_lat_intersection docstring

* Fix RTD notebook kernel metadata

* Address AccuSphGeom review feedback

* Separate intersection kernels into three layers: numerical core, status/mask, dispatcher

* Fix NaN/Inf propagation in intersection kernels for denom=0 and planar_sq<0

* computing: use FMA two_prod where available (portable Veltkamp fallback)

Add an LLVM fma intrinsic and route two_prod through a single fused
multiply-add for its error term on hardware that supports it, selected at
import time and validated to be bit-exact against the Veltkamp split. Falls
back to the portable Veltkamp form otherwise, so there is no hard FMA
dependency.

The FMA path is ~2x faster in the compensated geometry kernels (each
two_prod drops from ~17 flops to one FMADD) and is numerically identical:
all 241 AccuSphGeom baseline cases pass unchanged.

* intersections: add allocation-free scalar const-lat L1 kernel

Add _accux_constlat_scalar, which takes the arc endpoints as six scalars and
returns the candidate coordinates as scalars instead of two np.empty(3)
arrays. _accux_constlat now wraps it so the array API is unchanged.

Returning scalars lets Numba keep the candidates in registers, so a batch
loop over many edges does no per-point heap allocation. On a 16M-point
const-lat sweep this is ~2.7x faster than the array-returning path and drops
the AccuX/FP64 cost ratio from ~19x to ~7x. Bit-identical results; all 241
AccuSphGeom baseline cases pass.

* style: apply ruff formatting to computing.py

* Clean up spherical geometry review items

* Scalarize constant-latitude intersection dispatcher to remove per-call heap allocations

* Import benchmark kernels inside setup to avoid asv collection failures

Move the uxarray kernel imports from module level into each benchmark
class's setup, so a stale or mismatched environment build only errors the
affected benchmark instead of aborting collection of every benchmark in
the directory. Thanks @cmdupuis3 for catching the asv import failure.

* Cmd/accusphere (#1579)

* Honor quadrature kwargs in calculate_total_face_area

calculate_total_face_area accepted quadrature_rule, order and
latitude_adjusted_area but ignored them, always returning the cached
default-parameter face_areas. As a result the gaussian/corrected path
produced the same total as the triangular one.

Keep the cached fast path for the default parameters (which also preserves
the equal-area values used for HEALPix grids) and route any non-default
quadrature settings through _compute_face_areas so the requested rule,
order and latitude adjustment actually take effect.

* Restore matplotlib backend after HoloViews matplotlib plot (#1538)

* Restore matplotlib backend after HoloViews matplotlib plot

plot(backend='matplotlib') calls hv.extension('matplotlib'), which switches
the active matplotlib backend and clobbers the IPython inline display hook,
silently breaking subsequent native matplotlib/xarray .plot() calls. Restore
the original matplotlib backend right after the HoloViews extension switch;
HoloViews objects still display via Store.current_backend, so this is safe.

Closes #1537

* Address review: capture backend at switch, accurate docstring, effective test

* Reconfigure IPython inline display hook when restoring backend

mpl.use() restores the matplotlib backend name but does not re-register
IPython's inline display integration that hv.extension('matplotlib')
clobbers. In a Jupyter kernel without an explicit %matplotlib inline,
native matplotlib/xarray .plot() calls after a uxarray matplotlib plot
still failed to render. Re-run configure_inline_support when the restored
backend is inline so the display hook is reinstated. See #1537.

* Restore matplotlib backend via IPython shell reactivation to fix inline display

---------

Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com>

* Allow SCRIP reader to respect units w/r/t radians (#1433)

* Allow SCRIP reader to respect units wrt to radians

* Add test file and unit test for SCRIP radian coordinate handling

* minor formatting changes for scrip radians fix

Moves meshfiles/scrip/scrip_radians.nc to meshfiles/scrip/scrip_radians/scrip_radians_grid.nc to match style of other meshfiles naming schemes.

Renames the new _scrip._convert_to_degrees() to _scrip._values_in_degrees(). It doesn't always convert; and it also does more than just converting, because it gives numpy array from DataArray. Clarified docstring.

ran the following, so it should now pass ruff checks: pre-commit run --all-files

---------

Co-authored-by: Sam Evans <s7evans11@gmail.com>
Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com>
Co-authored-by: Rajeev Jain <rajeeja@gmail.com>

* Support Python 3.14 and upgrade YAC to v3.18 (with DNN remapping) (#1563)

* Upgrade YAC to v3.18, expose DNN remapping, drop pathlib backport

- upgrade YAC CI to v3.18.0 on Python 3.14 (numba>=0.63, py3.14 classifier,
  cython>=3.1 via conda for the bindings build)
- fix add_average: reduction_type -> weight_type (renamed in YAC v3.18)
- expose distance-nearest-neighbour (yac_method='dnn', new in YAC v3.15) + test
- remove the obsolete 'pathlib' backport dependency: it shadows the stdlib and
  breaks tools that import pathlib on Python 3.10+ (broke YAC's Cython build)

* Reference #1561 in numba version pin comment

---------

Co-authored-by: Christopher Dupuis <45972964+cmdupuis3@users.noreply.github.com>

* Bump actions/download-artifact in the actions group across 1 directory (#1572)

Bumps the actions group with 1 update in the / directory: [actions/download-artifact](https://github.com/actions/download-artifact).


Updates `actions/download-artifact` from 7 to 8
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](actions/download-artifact@v7...v8)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [pre-commit.ci] pre-commit autoupdate (#1565)

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.15.20 → v0.15.21](astral-sh/ruff-pre-commit@v0.15.20...v0.15.21)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Accusphere: gca_gca benchmarks and better arc sampling

* Accusphere: fix sum_of_squares accuracy bug + tests

* Accusphere: numba fma refactor and fix validation tautology

* Accusphere: scalarized _counts_as_crossing

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: vakudo <127726617+vakudo@users.noreply.github.com>
Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com>
Co-authored-by: Rajeev Jain <rajeeja@gmail.com>
Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com>
Co-authored-by: zarzycki <colin.zarzycki@gmail.com>
Co-authored-by: Sam Evans <s7evans11@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Add thread-scaling benchmark for FP64 vs AccuX constlat dispatcher

* Cap thread-scaling sweep at performance cores to avoid E-core artifact

* Fix thread-scaling plot ticks to show only 1,2,4,8

* o Fix pre-commit

* o doc fixes

* o Remove benchmark image from repo

* Split out point-in-face and lat-lon bounds to a separate PR

* o computing: match AccuSphGeom acc_sqrt_re exactly (residual order, branch-free)

* o intersections: compensated norm in AccuXGCA kernel, branch-free mask validity per AccuSphGeom

* o arcs: branch-free on_minor_arc int mask at tol 1e-8 matching AccuSphGeom

* o benchmarks: drop point-in-polygon kernel bench, moved to point-in-face PR

* o benchmarks: drop thread-scaling constlat, moved to separate scaling issue

* o docs: move spherical-geometry notebook out, depends on split-out point-in-face

* o docs: drop spherical-geometry notebook from user-guide toctree

* o test_plot: drop to_raster_auto_extent, unrelated to AccuXGCA/AccuXConstLat scope

* o fix on_minor_arc antipodal false positive

* o add on_minor_arc degeneracy regression tests

* fix diff_of_products addition grouping to match AccuSphGeom eft.hpp

* o intersections: collapse pos/neg-valid branches via XOR in gca_const_lat_intersection

* arcs: keep normal hi/lo split through _normal_dot_value via compensated dot

* arcs: fix comments overclaiming branch-free form and degeneracy-check robustness

* arcs: stop claiming on_minor_arc mirrors AccuSphGeom, it's an approximation

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Hongyu Chen <hyvchen@ucdavis.edu>
Co-authored-by: Christopher Dupuis <45972964+cmdupuis3@users.noreply.github.com>
Co-authored-by: vakudo <127726617+vakudo@users.noreply.github.com>
Co-authored-by: Sam Evans <47793072+Sevans711@users.noreply.github.com>
Co-authored-by: Orhan Eroglu <32553057+erogluorhan@users.noreply.github.com>
Co-authored-by: zarzycki <colin.zarzycki@gmail.com>
Co-authored-by: Sam Evans <s7evans11@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants