Skip to content

Regression in CASE expression since DF 44 #14277

Description

@andygrove

Describe the bug

I am trying to upgrade Comet to the latest DataFusion and see some queries fail.

The regression seems to have been introduced in #13953.

Example Query:

SELECT array_remove(array(_2, _3,_4), _2) from t1 where _2 is null

Query plan:

ProjectionExec: expr=[CASE WHEN Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_0@0, eval_mode: Legacy] IS NULL THEN NULL ELSE array_remove(make_array(Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_0@0, eval_mode: Legacy], Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_1@1, eval_mode: Legacy], col_2@2), Cast [data_type: Int32, timezone: America/Los_Angeles, child: col_0@0, eval_mode: Legacy]) END as col_0]
  CometFilterExec: col_0@0 IS NULL
    ScanExec: source=[CometScan parquet  (unknown)], schema=[col_0: Int8, col_1: Int16, col_2: Int32]

Failure:

Invalid argument error: arguments need to have the same data type

Cause:

In the new expr_or_expr function, the following line is causing the failure:

Ok(ColumnarValue::Array(zip(&remainder, &else_, &then_value)?))

The type of else_ is ListArray and the type of then_value is NullArray

Solution:

Perhaps we just need to add a cast in here? I am going to add a repro to DataFusion and work on a fix.

To Reproduce

No response

Expected behavior

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions