Skip to content

Port generate-encoding-data.py to Python 3 - #124

Merged
hsivonen merged 8 commits into
hsivonen:mainfrom
Nopey:py_2to3
Dec 19, 2025
Merged

hsivonen merged 8 commits into
hsivonen:mainfrom
Nopey:py_2to3

Conversation

@Nopey

@Nopey Nopey commented Aug 3, 2025

Copy link
Copy Markdown
Contributor

No changes to the generated files, as those still generate the same.

Nopey added 8 commits August 2, 2025 19:39
This was done with 2to3 version 3.11.2-1 installed from Debian bookworm.

2to3 is an automated source code translator for upgrading python2
scripts to python 3.
The previous commit hash, 1d519bf8e5555cef64cf3a712485f41cd1a6a990,
was part of PR whatwg/encoding#336, and eventually got merged as
commit 2c3853e461afd718be198772170d024e427aee21.

There is only a mild change between these two commits, where wording in
encoding.bs was changed.
Note that Python 3's open() function defaults to a platform dependent
encoding (locale.getencoding()), which is not what we want!
@Nopey

Nopey commented Aug 3, 2025

Copy link
Copy Markdown
Contributor Author

Drafting while I investigate what's going on in the generated codepage/{lib,test}.rs, which wants to add two encodings

+    assert_eq!(to_encoding(10007), Some(X_MAC_CYRILLIC));
+    assert_eq!(to_encoding(21010), Some(UTF_16LE));

EDIT: These seem to be correct, which would just make the codepage crate's source code out of date. Undrafting.
Commit 11b0b46 adds them to the python script:

  10007: "x-mac-cyrillic",
  21010: "UTF-16LE", # Undocumented; needed by calamine for Excel compat

EDIT 2: These two's absence is issue #70.

@Nopey
Nopey marked this pull request as draft August 3, 2025 05:38
@Nopey
Nopey marked this pull request as ready for review August 3, 2025 05:41
@hsivonen
hsivonen merged commit 01e369b into hsivonen:main Dec 19, 2025
@hsivonen

Copy link
Copy Markdown
Owner

Excellent! Thank you. (I previously gave up at cmp.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants