Skip to content

Commit da095f9

Browse files
chenhao-dbcloud-fan
authored andcommitted
[SPARK-56663][SQL][FOLLOWUP] Fix silent overflow in date_trunc fast path.
### What changes were proposed in this pull request? Fixes the bug introduced by #55610. When `local` is a very large negative value, `Math.floorMod(local, unitMicros)` is positive because `unitMicros` is positive, and the subtraction can overflow to positive values. This produces a inconsistent result from the slow path. In most common cases, this optimization still takes effect. The performance loss is ignorable. ### Why are the changes needed? Fix incorrect result. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? New unit tests. It passes before #55610, or after this change. But it fails on the current master. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #56313 from chenhao-db/fix_trunc_overflow. Authored-by: chenhao-db <chenhao.li@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit 94d23c3) Signed-off-by: Wenchen Fan <wenchen@databricks.com>
1 parent 48e2c8d commit da095f9

2 files changed

Lines changed: 18 additions & 3 deletions

File tree

‎sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -510,7 +510,7 @@ object DateTimeUtils extends SparkDateTimeUtils {
510510
val offsetMicros = originalOffsetSec * MICROS_PER_SECOND
511511
try {
512512
val local = Math.addExact(micros, offsetMicros)
513-
val truncatedLocal = local - Math.floorMod(local, unitMicros)
513+
val truncatedLocal = Math.subtractExact(local, Math.floorMod(local, unitMicros))
514514
val candidate = Math.subtractExact(truncatedLocal, offsetMicros)
515515
if (!rules.isFixedOffset) {
516516
val candidateSec = Math.floorDiv(candidate, MICROS_PER_SECOND)
@@ -537,9 +537,9 @@ object DateTimeUtils extends SparkDateTimeUtils {
537537
level match {
538538
case TRUNC_TO_MICROSECOND => micros
539539
case TRUNC_TO_MILLISECOND =>
540-
micros - Math.floorMod(micros, MICROS_PER_MILLIS)
540+
Math.subtractExact(micros, Math.floorMod(micros, MICROS_PER_MILLIS))
541541
case TRUNC_TO_SECOND =>
542-
micros - Math.floorMod(micros, MICROS_PER_SECOND)
542+
Math.subtractExact(micros, Math.floorMod(micros, MICROS_PER_SECOND))
543543
case TRUNC_TO_MINUTE =>
544544
truncToUnitFast(micros, zoneId, MICROS_PER_MINUTE, ChronoUnit.MINUTES)
545545
case TRUNC_TO_HOUR =>

‎sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/DateExpressionsSuite.scala‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -853,6 +853,21 @@ class DateExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper {
853853
}
854854
}
855855

856+
test("TruncTimestamp of Long.MinValue overflows with ArithmeticException") {
857+
withDefaultTimeZone(UTC) {
858+
// Long.MinValue is the smallest representable timestamp value (in micros). Truncating it
859+
// rounds the value down to an earlier instant, which falls below the representable micros
860+
// range. The overflow must surface as an ArithmeticException instead of silently wrapping
861+
// around to a bogus (positive) timestamp.
862+
val minTimestamp = Literal.create(Long.MinValue, TimestampType)
863+
Seq("YEAR", "QUARTER", "MONTH", "WEEK", "DAY", "HOUR", "MINUTE",
864+
"SECOND", "MILLISECOND").foreach { fmt =>
865+
checkExceptionInExpression[ArithmeticException](
866+
TruncTimestamp(Literal.create(fmt, StringType), minTimestamp), "")
867+
}
868+
}
869+
}
870+
856871
test("unsupported fmt fields for trunc/date_trunc results null") {
857872
Seq("INVALID", "decade", "century", "millennium", "whatever", null).foreach { field =>
858873
testTruncDate(Date.valueOf("2000-03-08"), field, null)

0 commit comments

Comments
 (0)