Repository navigation
Fix parallel MCMC sampling segfault (cores=1 workaround) #2
Description
Problem
PyTensor 2.38 + NumPy 2.4 causes segfaults during multiprocessing MCMC sampling. All tests currently use cores=1 as a workaround, which makes slow tests even slower.
Root Cause
Same underlying incompatibility as #1 — PyTensor's multiprocessing backend crashes with recent NumPy versions when forking processes for parallel chain sampling.
Current Workaround
All NUTSSampler calls in tests use cores=1:
sampler = NUTSSampler(draws=100, tune=100, chains=2, cores=1, random_seed=42)Desired Fix
- Monitor PyTensor releases for multiprocessing fix
- Once fixed, remove
cores=1constraint from test samplers - Consider defaulting
NUTSSampler.corestoos.cpu_count()or similar
Related
- Remove manual Cholesky workaround when PyMC fixes LKJ bug #1 (same PyMC/PyTensor compatibility issue)
Labels
tech-debt, testing
Activity
refactor: extract MA recursion + lift time-dim guard to base (closes #76
refactor: extract MA recursion + lift time-dim guard to base (closes #76
thomaspinder commentedon Jul 29, 2026
No actionable work remains here.
NUTSSampler.coresalready defaults to auto-detect (src/impulso/samplers.py:57), and the default backend is nutpie when installed (samplers.py:13-21), which uses threads rather than forked processes — the default path does not exercise PyMC's multiprocessing at all. The survivingcores=1sites are deliberate: the SV default sampler (src/impulso/sv/spec.py:58-61) and slow tests that explicitly pinnuts_sampler="pymc". The constraint is documented in CLAUDE.md. To be clear, the underlying PyMC/macOS crash is not verified fixed upstream — this is a close-for-hygiene: if a parallel-sampling crash resurfaces, a fresh issue with a reproducer will be more useful than this monitoring ticket.