fix(mysql): map %x and %r date format specifiers [CLAUDE] - #8142
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
| "%l": "%-I", | ||
| "%r": "%I:%M:%S %p", | ||
| "%T": "%H:%M:%S", | ||
| "%v": "%V", |
There was a problem hiding this comment.
One side effect here is this auto-generates inverse mapping, rewriting MySQL's uppercase %V into %v on generation. So DATE_FORMAT(x, '%X-%V') no longer round-trips.
There was a problem hiding this comment.
Good catch, thanks — confirmed. Since Presto/Trino share MySQL.TIME_MAPPING, the rewrite leaked there too:
# previous revision (d825d35)
mysql: DATE_FORMAT(x, '%X-%V') --> DATE_FORMAT(x, '%X-%v')
presto: DATE_FORMAT(x, '%X-%V') --> DATE_FORMAT(x, '%X-%v')
I don't think there is a lossless way to keep the %v mapping: MySQL has four week specifiers (%U, %u, %V, %v) and strftime only three (%U, %W, %V), so any %v mapping collapses two MySQL specifiers onto one strftime token, and whichever one loses the inverse gets silently rewritten on generation. Pinning the inverse with a class-level INVERSE_TIME_MAPPING = {"%V": "%V"} just moves the rewrite (%x-%v then comes back as %x-%V), and mapping %V to %U instead corrupts native %U.
So in 361b6c5 I dropped the %v entry and kept the two collision-free mappings (%x -> %G, %r -> %I:%M:%S %p, whose strftime sides are not native MySQL specifiers):
mysql -> mysql: DATE_FORMAT(x, '%X-%V') --> DATE_FORMAT(x, '%X-%V') (round-trips again)
mysql -> duckdb: DATE_FORMAT(x, '%x') --> STRFTIME(CAST(x AS TIMESTAMP), '%G')
duckdb -> mysql: STRFTIME(x, '%G') --> DATE_FORMAT(x, '%x')
%v now passes through untranslated in both directions again, exactly as on main, and I added identity tests for %X-%V, %x-%v and %U so this can't regress. Proper %v support would need a dedicated internal token that degrades gracefully in other dialects (along the lines of the %dstrict family), which felt out of scope here. Updated the PR title/description to match.
There was a problem hiding this comment.
One more side effect from the same mechanism, this one harmless: with %r -> %I:%M:%S %p, a MySQL round-trip now rewrites DATE_FORMAT(x, '%h:%i:%s %p') to DATE_FORMAT(x, '%r'). MySQL defines %r as exactly hh:mm:ss plus AM/PM, so the rendered output is identical — a normalization rather than a semantic change like the %V case. Flagging it rather than letting you find it. If you'd rather have no rewrite here at all, I can drop %r and keep only %x -> %G.
There was a problem hiding this comment.
looks good, thanks!
The inverse mapping derived from TIME_MAPPING rewrote MySQL's %V (Sunday-based week) into %v on generation, so DATE_FORMAT(x, '%X-%V') did not round-trip (Presto/Trino included, via the shared mapping). MySQL has four week specifiers (%U, %u, %V, %v) but strftime only has three (%U, %W, %V), so a lossless %v mapping is not possible; keep the collision-free %x and %r entries and add round-trip regression tests. Co-authored-by: Cursor <cursoragent@cursor.com>
Partially addresses #8141 (
%xand%r;%vis deliberately excluded — see below).Adds the two MySQL format specifiers whose strftime equivalents do not collide with other native MySQL specifiers to
TIME_MAPPING:%x->%G(ISO 8601 week-numbering year, used with%v)%r->%I:%M:%S %p(12-hour time, same shape as the existing"%T": "%H:%M:%S"entry)Before this change they passed through untranslated, e.g.
DATE_FORMAT(x, '%x')(mysql) becameSTRFTIME(x, '%x')(duckdb), which DuckDB reads as an ISO date — silently wrong data — andSTRFTIME(x, '%G')(duckdb) becameDATE_FORMAT(x, '%G')(mysql), where%Gis not a MySQL specifier. Verified against DuckDB 1.5.5:STRFTIME(TIMESTAMP '2021-01-01 09:05:03', '%G')returns2020, matching MySQLDATE_FORMAT(..., '%x').Why
%vis not mapped: the first revision also mapped%v->%V, but as @fivetran-kwoodbeck pointed out,INVERSE_TIME_MAPPINGis derived fromTIME_MAPPING, so that entry rewrote MySQL's native%V(Sunday-based week, a different specifier) into%von generation, andDATE_FORMAT(x, '%X-%V')no longer round-tripped (Presto/Trino were affected too, since they shareMySQL.TIME_MAPPING). A lossless%vmapping is not possible with the current machinery: MySQL has four week specifiers (%U,%u,%V,%v) while strftime has three (%U,%W,%V), so any%vmapping collapses two MySQL specifiers onto one strftime token and one of them necessarily gets rewritten on generation.%vnow passes through untranslated, exactly as on main, and new identity tests pin the%X-%V,%x-%vand%Uround-trips.Tests:
tests/dialects/test_mysql.py::test_date_formatcovers the%x/%rtranslations in both directions plus the round-trip identities above. Full local run:SKIP_INTEGRATION=1 python -m unittest— 1202 tests, the only 3 errors being local-environment import failures (duckdb/pandasdev deps and apythonalias unavailable), unrelated to this change.ruff==0.15.6check/format --checkon the touched files: clean.Disclosure: this change was prepared with AI assistance; I reproduced every behavior above locally and reviewed each line.