Skip to content

fix(constrained): guard empty stratum IS in interior-multiplier reduction (#291)#318

Merged
lmoresi merged 4 commits into
developmentfrom
bugfix/constrained-empty-stratum-guard
Jul 5, 2026
Merged

fix(constrained): guard empty stratum IS in interior-multiplier reduction (#291)#318
lmoresi merged 4 commits into
developmentfrom
bugfix/constrained-empty-stratum-guard

Conversation

@lmoresi

@lmoresi lmoresi commented Jul 5, 2026

Copy link
Copy Markdown
Member

Closes #291.

Summary

`Stokes_Constrained` segfaulted at np>1 in
`_constrain_interior_multipliers_in_section`
(`src/underworld3/cython/petsc_generic_snes_solvers.pyx:7001`).

The root cause was the boundary-IS guard. On a rank that owns zero points
with a given boundary label value,
`dm.getLabel("UW_Boundaries").getStratumIS(bvalue)` returns a valid PETSc
IS with `getSize() == 0`
— not None, non-zero handle, and
`bool(IS) == True`. So the existing guard `if bd_is is not None:` passed
through, and the subsequent `bd_is.getIndices()` crashed with SIGSEGV.

Change

`petsc_generic_snes_solvers.pyx:7001` — replace

```python
if bd_is is not None:
```

with

```python
if bd_is and bd_is.getSize() > 0:
```

`bool(bd_is)` short-circuits the null-handle case (the same idiom
`utilities/boundary_flux.py:70` uses); `.getSize() > 0` catches the
valid-but-empty case that neither the None check nor the truthiness
check protects against.

Verification

Lightweight unit-viscosity repro (16×16 quad, sinusoidal body force,
free-slip on all four walls; `_reduce_interior_multiplier = True` default):

| np | `|u|_inf` | wall | pre-fix |
|----|-------------|-------|---------------|
| 1 | 2.533e-02 | 6.6 s | (unchanged) |
| 2 | 2.533e-02 | 8.5 s | SIGSEGV |
| 4 | 2.533e-02 | 5.0 s | SIGSEGV |

Bit-identical parallel-vs-serial in this regime.

`tests/test_1062_constrained_solcx.py` (canonical SolCx `eta_B=1e6` test):
serial passes (198 s); the np=2 with-mpi rerun exceeded the 300 s timeout
under this session's compute (`Stokes_Constrained` is 34× slower than
Nitsche per #244), so a longer-timeout parallel variant of this test
(probably `tests/parallel/test_1062_constrained_solcx_parallel.py`) is
warranted as follow-up. Not blocking — the guard is a strict guard-widening
that cannot produce wrong physics; it only stops a previously-crashing rank
from calling `.getIndices()` on an empty IS.

Related

Underworld development team with AI support from Claude Code

lmoresi added 3 commits July 4, 2026 13:06
`ksp.solve` never raises on `KSP_DIVERGED_*`, so the two linear paths in
`rotated_bc` (the default fieldsplit-Schur iterative solve at
`_solve_rotated_iterative` and the opt-in direct MUMPS LU at
`solve_rotated_freeslip`) both returned partial answers silently.

That silence is what let the 3D rotation-nullspace bug fixed by #306 look
like a working solve for as long as it did: the outer Krylov hit its
`ksp_max_it` ceiling (300 in the pre-#306 code) with a residual well
above tolerance, and neither the KSP nor the caller flagged it. The
solution then propagated into downstream diagnostics that eventually
exposed it as wrong physics rather than as a solver failure.

Add a small `_warn_if_ksp_diverged` helper and call it immediately after
each of the two `ksp.solve` sites. Rank-0 warning only (via `uw.mpi.pprint`),
same style as the existing nonlinear-driver warning at the end of
`solve_rotated_freeslip_nonlinear`. No behaviour change on the happy path;
non-convergence is now loud instead of silent.

test_1018_rotated_freeslip.py: 14/14 pass (2D behaviour unchanged).

Underworld development team with AI support from Claude Code
…erged

Copilot review on #315 noted the convention split:

  * KSP.getConvergedReason() >  0  → converged
  * KSP.getConvergedReason() == 0  → KSP_CONVERGED_ITERATING (initial state;
                                     unexpected after ksp.solve() returns)
  * KSP.getConvergedReason() <  0  → diverged

The previous `reason >= 0` early-return silently accepted a hypothetical
`reason == 0` outcome, undermining the guard's purpose. Change to `reason > 0`
so a stray `KSP_CONVERGED_ITERATING` also fires the warning, matching the
project convention documented in petsc_generic_snes_solvers.pyx:2979.

test_1018_rotated_freeslip.py: 14/14 pass (unchanged).

Underworld development team with AI support from Claude Code
…tion (#291)

`Stokes_Constrained` segfaulted at np>1 in `_constrain_interior_multipliers_in_section`:
on a rank owning zero points with a given boundary label value,
`dm.getLabel("UW_Boundaries").getStratumIS(bvalue)` returns a VALID PETSc IS
with `getSize() == 0` — not None, non-zero handle, `bool(IS) == True`. The
existing `if bd_is is not None:` guard passed through, and the subsequent
`bd_is.getIndices()` crashed with SIGSEGV in the PETSc IS binding.

The failure was reliably reproduced by #291's repro (StructuredQuadBox +
free-slip on all four walls, np=2) and by a lighter unit-viscosity variant
which now solves bit-identically at np=1/2/4 with the fix.

Fix: replace `if bd_is is not None:` with `if bd_is and bd_is.getSize() > 0:`.
`bool(bd_is)` short-circuits the null-handle case (project convention, see
`utilities/boundary_flux.py:70`); `.getSize() > 0` catches the valid-but-empty
case that neither the None check nor the truthiness check protects against.

### Verification

Lightweight unit-viscosity repro (16x16 quad, sinusoidal body force,
free-slip on all four walls; `_reduce_interior_multiplier = True` default):

    np=1:  |u|_inf = 2.533e-02  (6.6 s)
    np=2:  |u|_inf = 2.533e-02  (8.5 s)   [previously: SIGSEGV]
    np=4:  |u|_inf = 2.533e-02  (5.0 s)   [previously: SIGSEGV]

Bit-identical parallel-vs-serial in this regime.

`tests/test_1062_constrained_solcx.py` (canonical SolCx eta_B=1e6 test):
serial passes (198 s); np=2 with-mpi rerun exceeds the default 300 s timeout
under this session's compute — a longer-timeout parallel check is warranted
as follow-up (probably belongs in `tests/parallel/`).

Underworld development team with AI support from Claude Code
Copilot AI review requested due to automatic review settings July 5, 2026 12:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a parallel crash in Stokes_Constrained interior-multiplier reduction by correctly guarding against valid-but-empty PETSc stratum IS objects (a case that occurs when a rank owns zero points for a given boundary value). Also adds explicit rank-0 warnings when the rotated free-slip helper KSPs diverge, preventing silent “last-iterate” returns.

Changes:

  • Widened the getStratumIS guard in _constrain_interior_multipliers_in_section to skip empty strata and avoid getIndices() segfaults in MPI.
  • Added _warn_if_ksp_diverged() and invoked it in both direct-LU and fieldsplit-Schur rotated free-slip solve paths.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/underworld3/cython/petsc_generic_snes_solvers.pyx Prevents MPI segfault by guarding empty boundary stratum IS before calling getIndices().
src/underworld3/utilities/rotated_bc.py Adds rank-0 warnings on KSP divergence to avoid silently returning a partial rotated free-slip solution.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 6999 to 7007
bd_is = dm.getLabel("UW_Boundaries").getStratumIS(bvalue)
# Guard for parallel: a rank owning no points with this label value
# gets back a valid non-None PETSc.IS with size 0 (bool(bd_is) = True,
# bd_is.handle != 0). Calling .getIndices() on that IS segfaults.
# `if bd_is` catches the null-handle case; `.getSize() > 0` catches
# the valid-but-empty case that None / handle checks miss (issue #291).
keep = set()
if bd_is is not None:
if bd_is and bd_is.getSize() > 0:
for bp in bd_is.getIndices().tolist():
…llel test

Copilot review on #318 rightly pointed out that a parallel-only SIGSEGV
regression should not merge without a re-enabled parallel test. Two
changes:

## 1. New lightweight parallel regression (tests/parallel/test_1062_constrained_stratum_guard_parallel.py)

Unit-viscosity 16×16 quad box, sinusoidal body force, free-slip on all
four walls at np≥2. On a 2-way axis-aligned partition (the default) this
puts one full boundary on each rank and zero points on the other — the
exact empty-stratum configuration that #291's guard now handles.

Runs in ~10 s at np=2 (vs the canonical test_1062 SolCx eta_B=1e6 which
timed out at 300 s in this session's parallel runs, per #244).

Asserts `|u|_inf` matches the serial GOLDEN of 2.53305e-02 to rtol 1e-3
(actual serial-vs-parallel spread on this workload is < 1e-9).

## 2. Un-skip test_parallel_custom_fmg_stokes_constrained (tests/parallel/test_1017_custom_mg_parallel_mpi.py)

Its skip reason explicitly cited "#291; auto-enables once #291 is fixed."
The guard fix is now in place, so the test can run. Marked `slow`
(SolCx eta_B=1e6 is not a fast benchmark) so CI selects it via
`pytest -m slow` or `-m "not slow"` per environment.

Underworld development team with AI support from Claude Code
@lmoresi

lmoresi commented Jul 5, 2026

Copy link
Copy Markdown
Member Author

Thanks @copilot-pull-request-reviewer — added in 958f6f4:

  1. New lightweight parallel regression at tests/parallel/test_1062_constrained_stratum_guard_parallel.py — unit-viscosity 16×16 quad box, free-slip on all four walls at np≥2. On a 2-way axis-aligned partition this puts one full boundary on each rank and zero on the other, triggering the empty-stratum path from the guard. Runs in ~10 s at np=2 and asserts |u|_inf matches the serial GOLDEN to rtol=1e-3. Pre-fix this test SIGSEGVs; post-fix it passes.
  2. Un-skipped test_parallel_custom_fmg_stokes_constrained (tests/parallel/test_1017_custom_mg_parallel_mpi.py) — its skip reason explicitly said "auto-enables once Stokes_Constrained segfaults at np>1 in the interior-multiplier section reduction #291 is fixed." Now marked slow (the SolCx eta_B=1e6 benchmark it uses is slow, per Stokes_Constrained is slow in parallel despite low KSP iterations #244) so CI can select via -m "not slow" if needed.

lmoresi added a commit that referenced this pull request Jul 5, 2026
… IS (#319) (#321)

Same class of latent parallel segfault as #291 (PR #318), at four more
sites in `boundary_flux.py` and `rotated_bc.py`:

  * `boundary_flux.py:70`  — in `_boundary_field_nodes`
  * `boundary_flux.py:106` — in `_boundary_reaction_by_facet`
  * `boundary_flux.py:154` — in `_boundary_facet_elements`
  * `rotated_bc.py:78`     — in `_boundary_velocity_nodes`

Each uses the guard

    if sis is None or sis.handle == 0: ...

which catches None and null-handle IS values, but MISSES the case that
crashed `_constrain_interior_multipliers_in_section` in #291: on a rank
owning zero points with a given boundary label value,
`dm.getLabel("UW_Boundaries").getStratumIS(bvalue)` returns a valid
non-None PETSc IS with `getSize() == 0`, non-zero `handle`, and
`bool(IS) == True`. The subsequent `sis.getIndices()` segfaults on that IS.

These sites fire in:

  * `add_nitsche_bc` (writes go through boundary_flux),
  * `add_rotated_freeslip_bc` (rotated_bc),
  * Consistent-Boundary-Flux traction / heat-flux / Nusselt recovery,
  * σ_nn / dynamic-topography hand-off.

Empirically none has bitten yet because most 2D partitions place at
least one point of each boundary on each rank. Fine decompositions or
axis-aligned splits on axis-aligned walls (exactly the #291 shape) can
still hit it.

Fix: replace each guard with

    if not (sis and sis.getSize() > 0): ...

`bool(sis)` short-circuits the null-handle case (project convention);
`.getSize() > 0` catches the valid-but-empty case that the None / handle
checks miss.

test_1018_rotated_freeslip.py: 14/14 pass. test_1019_boundary_flux.py:
1/1 pass. No behaviour change on the happy path.

Underworld development team with AI support from Claude Code
@lmoresi lmoresi merged commit b25453a into development Jul 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants