Describe the bug
In native/spark-expr/src/math_funcs/negative.rs, the ANSI-mode overflow check (check_overflow! macro) loops over Int8/16/32/64 and interval arrays comparing every slot's value to MIN, then calls neg_wrapping. The scan reads value(i) without checking validity, so a MIN value sitting in a null slot (arbitrary bytes left behind by filter/slice/FFI) raises a spurious ARITHMETIC_OVERFLOW error even though the row is null.
It is also a two-pass re-implementation of arrow's checked neg kernel, which negates with overflow detection in a single pass and correctly skips null slots.
Steps to reproduce
Construct an Int64 array where a null slot's underlying value is i64::MIN (for example by masking a valid i64::MIN entry to null via a validity buffer) and evaluate negative() on it in ANSI mode. The expression errors; Spark returns null for the null row and succeeds.
Expected behavior
Null slots must be ignored by the overflow check. Replacing the ANSI path with arrow::compute::kernels::numeric::neg fixes this, since the kernel skips invalid slots.
Additional context
Notes for the fix:
- The Spark error message per type must be reconstructed in a
map_err; if the exact offending value is required for Int8/Int16 wording, a rescan on the (cold) error path can recover it.
IntervalUnit::MonthDayNano is deliberately not overflow-checked today, and arrow's checked neg would check it, so that type must keep neg_wrapping unconditionally.
- Legacy (non-ANSI) mode keeps
neg_wrapping as today.
Found during an audit of native code that replicates existing arrow-rs kernels. The corresponding null-slot behavior is already pinned correctly for add in checked_arithmetic.rs tests.
Describe the bug
In
native/spark-expr/src/math_funcs/negative.rs, the ANSI-mode overflow check (check_overflow!macro) loops over Int8/16/32/64 and interval arrays comparing every slot's value toMIN, then callsneg_wrapping. The scan readsvalue(i)without checking validity, so aMINvalue sitting in a null slot (arbitrary bytes left behind by filter/slice/FFI) raises a spurious ARITHMETIC_OVERFLOW error even though the row is null.It is also a two-pass re-implementation of arrow's checked
negkernel, which negates with overflow detection in a single pass and correctly skips null slots.Steps to reproduce
Construct an Int64 array where a null slot's underlying value is
i64::MIN(for example by masking a validi64::MINentry to null via a validity buffer) and evaluatenegative()on it in ANSI mode. The expression errors; Spark returns null for the null row and succeeds.Expected behavior
Null slots must be ignored by the overflow check. Replacing the ANSI path with
arrow::compute::kernels::numeric::negfixes this, since the kernel skips invalid slots.Additional context
Notes for the fix:
map_err; if the exact offending value is required for Int8/Int16 wording, a rescan on the (cold) error path can recover it.IntervalUnit::MonthDayNanois deliberately not overflow-checked today, and arrow's checkednegwould check it, so that type must keepneg_wrappingunconditionally.neg_wrappingas today.Found during an audit of native code that replicates existing arrow-rs kernels. The corresponding null-slot behavior is already pinned correctly for add in
checked_arithmetic.rstests.