[Feature] Support Spark expression: divide_dt_interval

[andygrove]

What is the problem the feature request solves?

Note: This issue was generated with AI assistance. The specification details have been extracted from Spark documentation and may need verification.

Comet does not currently support the Spark divide_dt_interval function, causing queries using this function to fall back to Spark's JVM execution instead of running natively on DataFusion.

The DivideDTInterval expression divides a day-time interval by a numeric value, returning a new day-time interval. This operation performs division with HALF_UP rounding mode and includes comprehensive overflow and divide-by-zero checks to ensure safe arithmetic operations.

Supporting this expression would allow more Spark workloads to benefit from Comet's native acceleration.

Describe the potential solution

Spark Specification

Syntax:

day_time_interval / numeric_value

Arguments:

Argument	Type	Description
interval	DayTimeIntervalType	The day-time interval to be divided
num	NumericType	The numeric divisor (integral, decimal, or fractional types)

Return Type: DayTimeIntervalType() - Returns a day-time interval representing the result of the division operation.

Supported Data Types:

• Left operand (interval): DayTimeIntervalType only
• Right operand (num): All numeric types including:
  • IntegralType (Byte, Short, Int, Long)
  • DecimalType
  • FractionalType (Float, Double)

Edge Cases:

• Null handling: Expression is null-intolerant (nullIntolerant = true) - returns null if either operand is null
• Divide by zero: Throws QueryExecutionErrors when divisor is zero
• Overflow behavior:
  • Checks for Long.MinValue / -1 overflow condition
  • Throws QueryExecutionErrors.overflowInIntegralDivideError on overflow
  • Uses longValueExact() for decimal operations to detect overflow
• Precision: All division operations use HALF_UP rounding mode for consistent behavior

Examples:

-- Divide a day-time interval by an integer
SELECT INTERVAL '2 12:30:45.123' DAY TO SECOND / 2;
-- Result: INTERVAL '1 06:15:22.561500' DAY TO SECOND

-- Divide by decimal value
SELECT INTERVAL '5 00:00:00' DAY TO SECOND / 2.5;
-- Result: INTERVAL '2 00:00:00' DAY TO SECOND

// Example DataFrame API usage
import org.apache.spark.sql.functions._

// Divide interval column by numeric literal
df.select(col("day_time_interval") / lit(3))

// Divide interval by another numeric column
df.select(col("day_time_interval") / col("divisor"))

Implementation Approach

See the Comet guide on adding new expressions for detailed instructions.

1. Scala Serde: Add expression handler in spark/src/main/scala/org/apache/comet/serde/
2. Register: Add to appropriate map in QueryPlanSerde.scala
3. Protobuf: Add message type in native/proto/src/proto/expr.proto if needed
4. Rust: Implement in native/spark-expr/src/ (check if DataFusion has built-in support first)

Additional context

Difficulty: Medium
Spark Expression Class: org.apache.spark.sql.catalyst.expressions.DivideDTInterval

Related:

• DivideYMInterval - Division operation for year-month intervals
• MultiplyDTInterval - Multiplication of day-time intervals
• IntervalDivide - Base trait for interval division operations
• BinaryExpression - Parent class for binary operations

---

This issue was auto-generated from Spark reference documentation.

[peterxcli]

take