[Fix](ttz) Fix TIMESTAMPTZ elapsed-time semantics to use UTC - #63161
Merged
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Collaborator
Author
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
I found a blocking test artifact issue. The TIMESTAMPTZ implementation itself is focused on elapsed-time UTC semantics, but the new regression output is incomplete, so the added coverage would fail or not validate several intended cases.
Critical checkpoint conclusions:
- Goal and proof: the code attempts to make TIMESTAMPTZ elapsed calculations use UTC instants, and it adds a regression suite, but the checked-in expected output does not match that suite.
- Scope: the GitHub PR file list is focused on TIMESTAMPTZ scalar/aggregate functions and one regression suite.
- Concurrency/lifecycle: no new concurrent or lifecycle-sensitive code paths identified in the reviewed changes.
- Config/compatibility: no new config items or storage/protocol format changes identified.
- Parallel paths: scalar diff functions, sequence functions, window_funnel, window_funnel_v2, and FE signatures were considered; no additional TIMESTAMPTZ path issue found beyond test output.
- Tests: blocking issue: the new
.outfile is missing outputs for multipleqt_/order_qt_cases and contains an incompletetopn_weightedresult shape. - Observability/transactions/persistence/data writes: not applicable to this PR.
User focus: no additional user-provided review focus was supplied.
linrrzqqq
force-pushed
the
fix-ttz-instance-calc
branch
from
May 12, 2026 08:28
9161b1e to
46c0460
Compare
Collaborator
Author
|
/review |
Contributor
There was a problem hiding this comment.
I reviewed the current GitHub PR diff (the timestamptz function support and regression test files) and did not find additional blocking issues beyond the already-known review context. The previously reported timestamptz .out mismatch appears addressed in the current diff: the output file now has entries for all 10 result-producing qt/order_qt cases, and the topn_weighted output has both grouped rows and both timestamp columns.\n\nCritical checkpoint conclusions:\n- Goal and tests: the PR adds TIMESTAMPTZ support for date/time diff, sequence, window_funnel, and topn_weighted paths, with regression coverage for DST fold/gap, mixed scale, aggregates, grouped results, columns, topn_weighted, and NULL handling.\n- Scope/focus: the current GitHub diff is focused on TIMESTAMPTZ function support and associated tests.\n- Concurrency/lifecycle: no concurrency or special lifecycle changes found in the current diff.\n- Compatibility/persistence/config: no persistence, storage format, config, or FE-BE thrift changes found.\n- Parallel code paths: FE signatures and BE registrations are updated for the relevant scalar and aggregate paths in the current diff.\n- Conditional checks: new type branches are consistent with adjacent DATETIMEV2 handling.\n- Test coverage: regression coverage is present; I did not run the regression suite in this review runner.\n- User focus: no additional user-provided review focus was supplied.
Collaborator
Author
|
run buildall |
Mryange
reviewed
May 12, 2026
Mryange
reviewed
May 12, 2026
Contributor
TPC-H: Total hot run time: 29747 ms |
Contributor
TPC-DS: Total hot run time: 171012 ms |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE Regression Coverage ReportIncrement line coverage |
Collaborator
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29783 ms |
Contributor
TPC-DS: Total hot run time: 172259 ms |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
FE Regression Coverage ReportIncrement line coverage |
Collaborator
Author
|
/review |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
There was a problem hiding this comment.
I found one blocking correctness issue in the new TIMESTAMPTZ timediff support.
Critical checkpoint conclusions:
- Goal/test coverage: The PR adds UTC semantics for TIMESTAMPTZ diff and aggregate functions with regression coverage, but the mixed-scale timediff test only covers the higher scale on the left side and misses the failing opposite order.
- Scope/focus: The changes are mostly focused on adding TIMESTAMPTZ overloads; the requested fix is small and localized.
- Concurrency/lifecycle/configuration: No new concurrency, lifecycle, or configuration concerns found in the reviewed paths.
- Compatibility/parallel paths: FE and BE compute function types independently; timediff now has a TIMESTAMPTZ FE signature but BE return type derivation is not updated to match FE scale derivation.
- Data correctness/performance/observability: No transaction/storage visibility or performance issues found. The blocking issue is result precision/type consistency for TIMESTAMPTZ timediff.
Existing review threads were considered and not duplicated. No additional user-provided focus points were present.
Mryange
approved these changes
May 14, 2026
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
github-actions Bot
pushed a commit
that referenced
this pull request
May 14, 2026
Problem Summary:
Fix TIMESTAMPTZ handling for elapsed-time semantics.
TIMESTAMPTZ represents an absolute instant. For calculations that depend
on the elapsed interval between two timestamps, such as time diff
functions and time-window matching logic, Doris should use the stored
UTC time values directly. This follows PostgreSQL-style semantics and
avoids treating TIMESTAMPTZ as local DATETIME before calculation.
### Before
The previous behavior could fall back to TIMESTAMPTZ-to-DATETIME
conversion before evaluating elapsed-time logic. That conversion depends
on the session time zone and produces local wall-clock time.
This is unstable around daylight saving time transitions. During DST
fall-back or spring-forward, local wall-clock time can repeat or skip,
so two TIMESTAMPTZ values with a fixed UTC interval may produce
different or unexpected elapsed-time results after conversion to
DATETIME.
```sql
Doris> SET time_zone = '+00:00';
Doris> SELECT milliseconds_diff(
-> CAST('2024-11-03 01:05:00 -05:00' AS TIMESTAMPTZ(6)),
-> CAST('2024-11-03 01:55:00 -04:00' AS TIMESTAMPTZ(6))) AS ms_utc;
+--------+
| ms_utc |
+--------+
| 600000 |
+--------+
Doris> SET time_zone = 'America/New_York';
-- 内部计算 cast 成 datetime(local_time), 在夏令时/冬令时转换节点结果不稳定)
Doris> SELECT milliseconds_diff(
-> CAST('2024-11-03 01:05:00 -05:00' AS TIMESTAMPTZ(6)),
-> CAST('2024-11-03 01:55:00 -04:00' AS TIMESTAMPTZ(6))) AS ms_ny;
+----------+
| ms_ny |
+----------+
| -3000000 |
+----------+
```
### Now
Handle TIMESTAMPTZ directly in affected elapsed-time paths, including
scalar time interval calculations and time-window matching logic, so
they operate on UTC values instead of local DATETIME values.
Add regression coverage for DST transition cases to verify that results
are based on absolute UTC elapsed time.
```sql
Doris> SET time_zone = 'America/New_York';
Doris> SELECT milliseconds_diff(
-> CAST('2024-11-03 01:05:00 -05:00' AS TIMESTAMPTZ(6)),
-> CAST('2024-11-03 01:55:00 -04:00' AS TIMESTAMPTZ(6))) AS ms_ny;
+--------+
| ms_ny |
+--------+
| 600000 |
+--------+
```
linrrzqqq
added a commit
to linrrzqqq/doris
that referenced
this pull request
May 14, 2026
yiguolei
added
dev/4.1.1-merged
and removed
dev/4.0.x
dev/4.0.x-conflict
dev/4.1.x
labels
linrrzqqq
added a commit
to linrrzqqq/doris
that referenced
this pull request
May 19, 2026
yiguolei
pushed a commit
that referenced
this pull request
May 27, 2026
zhaorongsheng
pushed a commit
to zhaorongsheng/doris
that referenced
this pull request
Jun 4, 2026
…63161) Problem Summary: Fix TIMESTAMPTZ handling for elapsed-time semantics. TIMESTAMPTZ represents an absolute instant. For calculations that depend on the elapsed interval between two timestamps, such as time diff functions and time-window matching logic, Doris should use the stored UTC time values directly. This follows PostgreSQL-style semantics and avoids treating TIMESTAMPTZ as local DATETIME before calculation. ### Before The previous behavior could fall back to TIMESTAMPTZ-to-DATETIME conversion before evaluating elapsed-time logic. That conversion depends on the session time zone and produces local wall-clock time. This is unstable around daylight saving time transitions. During DST fall-back or spring-forward, local wall-clock time can repeat or skip, so two TIMESTAMPTZ values with a fixed UTC interval may produce different or unexpected elapsed-time results after conversion to DATETIME. ```sql Doris> SET time_zone = '+00:00'; Doris> SELECT milliseconds_diff( -> CAST('2024-11-03 01:05:00 -05:00' AS TIMESTAMPTZ(6)), -> CAST('2024-11-03 01:55:00 -04:00' AS TIMESTAMPTZ(6))) AS ms_utc; +--------+ | ms_utc | +--------+ | 600000 | +--------+ Doris> SET time_zone = 'America/New_York'; -- 内部计算 cast 成 datetime(local_time), 在夏令时/冬令时转换节点结果不稳定) Doris> SELECT milliseconds_diff( -> CAST('2024-11-03 01:05:00 -05:00' AS TIMESTAMPTZ(6)), -> CAST('2024-11-03 01:55:00 -04:00' AS TIMESTAMPTZ(6))) AS ms_ny; +----------+ | ms_ny | +----------+ | -3000000 | +----------+ ``` ### Now Handle TIMESTAMPTZ directly in affected elapsed-time paths, including scalar time interval calculations and time-window matching logic, so they operate on UTC values instead of local DATETIME values. Add regression coverage for DST transition cases to verify that results are based on absolute UTC elapsed time. ```sql Doris> SET time_zone = 'America/New_York'; Doris> SELECT milliseconds_diff( -> CAST('2024-11-03 01:05:00 -05:00' AS TIMESTAMPTZ(6)), -> CAST('2024-11-03 01:55:00 -04:00' AS TIMESTAMPTZ(6))) AS ms_ny; +--------+ | ms_ny | +--------+ | 600000 | +--------+ ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem Summary:
Fix TIMESTAMPTZ handling for elapsed-time semantics.
TIMESTAMPTZ represents an absolute instant. For calculations that depend on the elapsed interval between two timestamps, such as time diff functions and time-window matching logic, Doris should use the stored UTC time values directly. This follows PostgreSQL-style semantics and avoids treating TIMESTAMPTZ as local DATETIME before calculation.
Before
The previous behavior could fall back to TIMESTAMPTZ-to-DATETIME conversion before evaluating elapsed-time logic. That conversion depends on the session time zone and produces local wall-clock time.
This is unstable around daylight saving time transitions. During DST fall-back or spring-forward, local wall-clock time can repeat or skip, so two TIMESTAMPTZ values with a fixed UTC interval may produce different or unexpected elapsed-time results after conversion to DATETIME.
Now
Handle TIMESTAMPTZ directly in affected elapsed-time paths, including scalar time interval calculations and time-window matching logic, so they operate on UTC values instead of local DATETIME values.
Add regression coverage for DST transition cases to verify that results are based on absolute UTC elapsed time.