Skip to content

Vector resolution interpreted as FWHM by refnx but as sigma by refl1d (2.355x inconsistency) #367

Description

@rozyczko

Summary

The ResolutionFunction.smearing() output is passed to both engines without per-engine width-convention conversion (outside the PercentageFwhm branch). refnx consumes it as x_err, which refnx defines as FWHM; refl1d consumes it as probe.dQ, which is sigma. The same numbers are therefore interpreted 2.355x apart depending on the engine.

Evidence

  • src/easyreflectometry/calculators/refnx/wrapper.py:192-198dq_vector = self._resolution_function.smearing(q_array) handed straight to model(x=q_array, x_err=dq_vector); conversion happens only for PercentageFwhm.
  • src/easyreflectometry/calculators/refl1d/wrapper.py:208-212 — sigma conversion (/ (2*sqrt(2*ln2))) applied only inside the isinstance(..., PercentageFwhm) branch; LinearSpline/Pointwise widths pass through unconverted.
  • model/resolution_functions.py:122Pointwise returns sqrt(sQz_variance) = sigma.
  • The library's own test confirms refnx expects FWHM: tests/calculators/refnx/test_refnx_wrapper.py:454-456 multiplies by sigma_to_fwhm = 2.355 before handing widths to refnx.

Impact

Resolution function semantics refnx (wants FWHM) refl1d (wants sigma)
PercentageFwhm FWHM % correct correct
LinearSpline (fwhm_values) FWHM correct over-smears x2.355
Pointwise (sqrt(sQz)) sigma under-smears x2.355 correct

Fitted thickness/roughness and their uncertainties are biased for any vector-resolution analysis, and results differ by engine.

Suggested fix

Standardize the smearing() contract (recommend sigma, matching Pointwise's natural output and ORSO's sQz), document it, and convert at each engine boundary (FWHM for refnx, none for refl1d), exactly as the PercentageFwhm branch already does. Add a cross-engine numerical regression test (same model + LinearSpline on refnx and refl1d).

There is currently no refl1d vector-resolution validation test against reference data — which is where this would have surfaced.

Found during deep code review (DEEP_ANALYSIS.md §4.1).

Metadata

Metadata

Assignees

No one assigned

    Labels

    [priority] highShould be prioritized soon[scope] bugBug report or fix (major.minor.PATCH)

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions