Skip to content

fix(mysql): map %x and %r date format specifiers [CLAUDE] - #8142

Merged
fivetran-kwoodbeck merged 3 commits into
tobymao:mainfrom
codeAnqiang-ma:fix/mysql-iso-week-time-mapping
Aug 13, 2026
Merged

fix(mysql): map %x and %r date format specifiers [CLAUDE]#8142
fivetran-kwoodbeck merged 3 commits into
tobymao:mainfrom
codeAnqiang-ma:fix/mysql-iso-week-time-mapping

Conversation

@codeAnqiang-ma

@codeAnqiang-ma codeAnqiang-ma commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Partially addresses #8141 (%x and %r; %v is deliberately excluded — see below).

Adds the two MySQL format specifiers whose strftime equivalents do not collide with other native MySQL specifiers to TIME_MAPPING:

  • %x -> %G (ISO 8601 week-numbering year, used with %v)
  • %r -> %I:%M:%S %p (12-hour time, same shape as the existing "%T": "%H:%M:%S" entry)

Before this change they passed through untranslated, e.g. DATE_FORMAT(x, '%x') (mysql) became STRFTIME(x, '%x') (duckdb), which DuckDB reads as an ISO date — silently wrong data — and STRFTIME(x, '%G') (duckdb) became DATE_FORMAT(x, '%G') (mysql), where %G is not a MySQL specifier. Verified against DuckDB 1.5.5: STRFTIME(TIMESTAMP '2021-01-01 09:05:03', '%G') returns 2020, matching MySQL DATE_FORMAT(..., '%x').

Why %v is not mapped: the first revision also mapped %v -> %V, but as @fivetran-kwoodbeck pointed out, INVERSE_TIME_MAPPING is derived from TIME_MAPPING, so that entry rewrote MySQL's native %V (Sunday-based week, a different specifier) into %v on generation, and DATE_FORMAT(x, '%X-%V') no longer round-tripped (Presto/Trino were affected too, since they share MySQL.TIME_MAPPING). A lossless %v mapping is not possible with the current machinery: MySQL has four week specifiers (%U, %u, %V, %v) while strftime has three (%U, %W, %V), so any %v mapping collapses two MySQL specifiers onto one strftime token and one of them necessarily gets rewritten on generation. %v now passes through untranslated, exactly as on main, and new identity tests pin the %X-%V, %x-%v and %U round-trips.

Tests: tests/dialects/test_mysql.py::test_date_format covers the %x/%r translations in both directions plus the round-trip identities above. Full local run: SKIP_INTEGRATION=1 python -m unittest — 1202 tests, the only 3 errors being local-environment import failures (duckdb/pandas dev deps and a python alias unavailable), unrelated to this change. ruff==0.15.6 check / format --check on the touched files: clean.

Disclosure: this change was prepared with AI assistance; I reproduced every behavior above locally and reviewed each line.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread sqlglot/dialects/mysql.py Outdated
"%l": "%-I",
"%r": "%I:%M:%S %p",
"%T": "%H:%M:%S",
"%v": "%V",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One side effect here is this auto-generates inverse mapping, rewriting MySQL's uppercase %V into %v on generation. So DATE_FORMAT(x, '%X-%V') no longer round-trips.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks — confirmed. Since Presto/Trino share MySQL.TIME_MAPPING, the rewrite leaked there too:

# previous revision (d825d35)
mysql:  DATE_FORMAT(x, '%X-%V') --> DATE_FORMAT(x, '%X-%v')
presto: DATE_FORMAT(x, '%X-%V') --> DATE_FORMAT(x, '%X-%v')

I don't think there is a lossless way to keep the %v mapping: MySQL has four week specifiers (%U, %u, %V, %v) and strftime only three (%U, %W, %V), so any %v mapping collapses two MySQL specifiers onto one strftime token, and whichever one loses the inverse gets silently rewritten on generation. Pinning the inverse with a class-level INVERSE_TIME_MAPPING = {"%V": "%V"} just moves the rewrite (%x-%v then comes back as %x-%V), and mapping %V to %U instead corrupts native %U.

So in 361b6c5 I dropped the %v entry and kept the two collision-free mappings (%x -> %G, %r -> %I:%M:%S %p, whose strftime sides are not native MySQL specifiers):

mysql  -> mysql:  DATE_FORMAT(x, '%X-%V') --> DATE_FORMAT(x, '%X-%V')   (round-trips again)
mysql  -> duckdb: DATE_FORMAT(x, '%x')    --> STRFTIME(CAST(x AS TIMESTAMP), '%G')
duckdb -> mysql:  STRFTIME(x, '%G')       --> DATE_FORMAT(x, '%x')

%v now passes through untranslated in both directions again, exactly as on main, and I added identity tests for %X-%V, %x-%v and %U so this can't regress. Proper %v support would need a dedicated internal token that degrades gracefully in other dialects (along the lines of the %dstrict family), which felt out of scope here. Updated the PR title/description to match.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more side effect from the same mechanism, this one harmless: with %r -> %I:%M:%S %p, a MySQL round-trip now rewrites DATE_FORMAT(x, '%h:%i:%s %p') to DATE_FORMAT(x, '%r'). MySQL defines %r as exactly hh:mm:ss plus AM/PM, so the rendered output is identical — a normalization rather than a semantic change like the %V case. Flagging it rather than letting you find it. If you'd rather have no rewrite here at all, I can drop %r and keep only %x -> %G.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, thanks!

The inverse mapping derived from TIME_MAPPING rewrote MySQL's %V
(Sunday-based week) into %v on generation, so DATE_FORMAT(x, '%X-%V')
did not round-trip (Presto/Trino included, via the shared mapping).
MySQL has four week specifiers (%U, %u, %V, %v) but strftime only has
three (%U, %W, %V), so a lossless %v mapping is not possible; keep the
collision-free %x and %r entries and add round-trip regression tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
@codeAnqiang-ma codeAnqiang-ma changed the title fix(mysql): map %v, %x and %r date format specifiers [CLAUDE] fix(mysql): map %x and %r date format specifiers [CLAUDE] Aug 13, 2026
@fivetran-kwoodbeck
fivetran-kwoodbeck merged commit d89f392 into tobymao:main Aug 13, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants