Problem
The custom-FMG transfer builders (src/underworld3/utilities/custom_mg.py, PR #290) receive bare coordinate arrays and rebuild coarse-grid connectivity from scratch:
barycentric_prolongation re-triangulates the coarse DOF cloud with scipy.spatial.Delaunay, then runs a per-fine-DOF Python loop for simplex location and weights.
rbf_prolongation assembles a dense coarse–coarse kernel matrix and does an O(n³) dense solve — fine for small clouds, infeasible at 3D velocity-block sizes.
This is redundant and, on curved domains, wrong-shaped: each coarse level is already a simplicial DMPlex — the triangulation exists, with UW3's optimised kd-tree cell location built on top, and FE basis evaluation available. On a 3D spherical shell the scipy Delaunay additionally fills the convex hull, so the shell cavity acquires phantom simplices and near-surface fine points fall out of the hull into nearest-neighbour fallbacks.
Measured impact
Rotated free-slip + custom FMG on the Zhong (#248) spherical shell, non-nested coarse pair (cellSize 0.19 → 0.125, ~11.5k fine cells): the solve converges in 5 outer iterations (post #306), but the prolongation build takes ~5.9 hours. The same problem with the GAMG velocity block completes in ~53 s end-to-end; a debug-size sphere (0.3/0.45) builds in ~2 min. The build, not the solve, is the bottleneck — and it is all scipy Delaunay + Python-loop location.
Proposed fix
Build P from the coarse mesh rather than the coarse point cloud:
- Locate fine-level DOF coordinates in coarse cells via the existing optimised point-location path (kd-tree cell hints + DMPlex location — the same machinery
uw.function.evaluate uses).
- Evaluate the coarse element's actual FE basis at each located point for the row weights — giving FE-exact transfers (including P2-exact for the velocity block, vs the current P1-over-an-artificial-triangulation).
- Keep a nearest-DOF fallback only for genuinely exterior points (curved-boundary chord gaps), which becomes rare once location uses the true mesh.
This removes the scipy Delaunay entirely, fixes the shell-cavity topology problem, vectorises the build, and should also simplify the parallel story (location is partition-aware; the current builders are serial-only by construction).
Low priority: non-nested 3D FMG is a rarely travelled path (nested refine() hierarchies and the GAMG fallback cover current use), but the barycentric builder as written will bite anyone who tries a 3D non-nested pair at production resolution.
Context: #248, #290, #306 (~/+Simulations/rotated_schur_pc_study/ holds the timing logs).
Underworld development team with AI support from Claude Code
Problem
The custom-FMG transfer builders (
src/underworld3/utilities/custom_mg.py, PR #290) receive bare coordinate arrays and rebuild coarse-grid connectivity from scratch:barycentric_prolongationre-triangulates the coarse DOF cloud withscipy.spatial.Delaunay, then runs a per-fine-DOF Python loop for simplex location and weights.rbf_prolongationassembles a dense coarse–coarse kernel matrix and does an O(n³) dense solve — fine for small clouds, infeasible at 3D velocity-block sizes.This is redundant and, on curved domains, wrong-shaped: each coarse level is already a simplicial DMPlex — the triangulation exists, with UW3's optimised kd-tree cell location built on top, and FE basis evaluation available. On a 3D spherical shell the scipy Delaunay additionally fills the convex hull, so the shell cavity acquires phantom simplices and near-surface fine points fall out of the hull into nearest-neighbour fallbacks.
Measured impact
Rotated free-slip + custom FMG on the Zhong (#248) spherical shell, non-nested coarse pair (cellSize 0.19 → 0.125, ~11.5k fine cells): the solve converges in 5 outer iterations (post #306), but the prolongation build takes ~5.9 hours. The same problem with the GAMG velocity block completes in ~53 s end-to-end; a debug-size sphere (0.3/0.45) builds in ~2 min. The build, not the solve, is the bottleneck — and it is all scipy Delaunay + Python-loop location.
Proposed fix
Build P from the coarse mesh rather than the coarse point cloud:
uw.function.evaluateuses).This removes the scipy Delaunay entirely, fixes the shell-cavity topology problem, vectorises the build, and should also simplify the parallel story (location is partition-aware; the current builders are serial-only by construction).
Low priority: non-nested 3D FMG is a rarely travelled path (nested
refine()hierarchies and the GAMG fallback cover current use), but the barycentric builder as written will bite anyone who tries a 3D non-nested pair at production resolution.Context: #248, #290, #306 (
~/+Simulations/rotated_schur_pc_study/holds the timing logs).Underworld development team with AI support from Claude Code