You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bug: flatten drops nulls and returns wrong results when a sub-array is null #4788
flatten on an array<array<T>> column returns wrong results and silently drops nulls when a row's outer array contains a null sub-array. Spark returns null for any such row; Comet returns a non-null, misaligned array. Silent data corruption, not a crash.
Match Spark: a null sub-array makes flatten return null for that row.
Additional context
Found while enabling CometLocalTableScanExec by default (#4393), but reproduces over a plain Parquet scan. Upstream test: DataFrameFunctionsSuite "flatten function".
Activity
michaelmitchell-bit commentedon Jul 4, 2026
take