Repository navigation
Instant, ZonedDateTime, and OffsetDateTime values are converted to datetime through a float of epoch seconds: microseconds are lost after 2106, and 9999-12-31T23:59:59.999999 raises ValueError #127
Description
Level: bindings. convert_java_to_python turns a java.time.Instant, ZonedDateTime, or OffsetDateTime into a timezone-aware datetime with datetime.fromtimestamp(epoch_second + nano / 1e9). A double has about 15 significant digits, so from the year 2262 on a microsecond can be off by one (.854775 reads as .854774), at the year 3000 the microsecond is gone (.000001 reads as 0), and 9999-12-31T23:59:59.999999 rounds up to year 10000 and raises ValueError: year 10000 is out of range from Result.get, to_dict, and to_list. Values up to the year 2106 are exact. The conversion table lists these three classes as converted to datetime (api/type_conversion.md L137), and a database started with -Darcadedb.dateTimeImplementation=java.time.Instant returns Instant for a DATETIME_MICROS property (the second half of the repro), so the converter is reached through a database as well.
Repro
The script prints to stdout. The JVM's own log lines go to stderr and are discarded by the command below.
uv venv .venv
uv pip install --python .venv/bin/python arcadedb_embedded-26.10.1.dev0-cp312-cp312-manylinux_2_34_x86_64.whl
.venv/bin/python repro-27-instant-conversion-float.py 2>/dev/null"""Instant, ZonedDateTime, and OffsetDateTime reach Python through a float of epoch seconds: microseconds are lost after 2106, and year 9999 raises."""
import datetime as dt
import os
import tempfile
import jpype
import arcadedb_embedded as arcade
from arcadedb_embedded.jvm import start_jvm
from arcadedb_embedded.type_conversion import convert_java_to_python
start_jvm(heap_size="2g", jvm_args=["-Darcadedb.dateTimeImplementation=java.time.Instant"])
Instant = jpype.JClass("java.time.Instant")
ZonedDateTime = jpype.JClass("java.time.ZonedDateTime")
OffsetDateTime = jpype.JClass("java.time.OffsetDateTime")
texts = ["2024-05-06T07:08:09.123456Z", "2106-02-07T06:28:16.000001Z", "2262-04-11T23:47:16.854775Z", "3000-01-01T00:00:00.000001Z", "9999-12-31T23:59:59.999999Z"]
print(f"{'Java value':<30} {'expected':<32} Instant -> / ZonedDateTime -> / OffsetDateTime ->")
for text in texts:
expected = dt.datetime.fromisoformat(text.replace("Z", "+00:00"))
shown = []
for java_value in (Instant.parse(text), ZonedDateTime.parse(text), OffsetDateTime.parse(text)):
try:
got = convert_java_to_python(java_value)
shown.append("ok" if got == expected else str(got))
except Exception as e:
shown.append(f"{type(e).__name__}: {e}")
print(f"{text:<30} {str(expected):<32} {' / '.join(shown)}")
print()
db = arcade.create_database(os.path.join(tempfile.mkdtemp(prefix="hb27-"), "db"))
db.command("sql", "CREATE DOCUMENT TYPE T")
db.command("sql", "CREATE PROPERTY T.at DATETIME_MICROS")
with db.transaction():
for i, text in enumerate(texts):
db.command("sql", "INSERT INTO T SET i = :i, at = :v", {"i": i, "v": dt.datetime.fromisoformat(text.replace("Z", ""))})
print("with arcadedb.dateTimeImplementation=java.time.Instant, written then read back:")
for i, text in enumerate(texts):
try:
got = db.query("sql", "SELECT FROM T WHERE i = ?", i).first().get("at")
except Exception as e:
got = f"{type(e).__name__}: {e}"
print(f" {text} -> {got}")
db.close()Output
Wheel arcadedb_embedded-26.10.1.dev0-cp312-cp312-manylinux_2_34_x86_64.whl with its bundled JRE (OpenJDK 25.0.4.1), JPype 1.7.1, and Python 3.12.13. The engine build the wheel reports through com.arcadedb.Constants is 26.10.1-SNAPSHOT (build 04bb03ebec07fce19101595d65ca1e3bdf516755), an ancestor of upstream cecb369df9. The repository is at f92ac1381c8335249a6fde644cbf04594f6d6391, and the installed Python sources are identical to bindings/python/src/arcadedb_embedded/ at that commit.
Java value expected Instant -> / ZonedDateTime -> / OffsetDateTime ->
2024-05-06T07:08:09.123456Z 2024-05-06 07:08:09.123456+00:00 ok / ok / ok
2106-02-07T06:28:16.000001Z 2106-02-07 06:28:16.000001+00:00 ok / ok / ok
2262-04-11T23:47:16.854775Z 2262-04-11 23:47:16.854775+00:00 2262-04-11 23:47:16.854774+00:00 / 2262-04-11 23:47:16.854774+00:00 / 2262-04-11 23:47:16.854774+00:00
3000-01-01T00:00:00.000001Z 3000-01-01 00:00:00.000001+00:00 3000-01-01 00:00:00+00:00 / 3000-01-01 00:00:00+00:00 / 3000-01-01 00:00:00+00:00
9999-12-31T23:59:59.999999Z 9999-12-31 23:59:59.999999+00:00 ValueError: year 10000 is out of range / ValueError: year 10000 is out of range / ValueError: year 10000 is out of range
with arcadedb.dateTimeImplementation=java.time.Instant, written then read back:
2024-05-06T07:08:09.123456Z -> 2024-05-06 07:08:09.123456+00:00
2106-02-07T06:28:16.000001Z -> 2106-02-07 06:28:16.000001+00:00
2262-04-11T23:47:16.854775Z -> 2262-04-11 23:47:16.854774+00:00
3000-01-01T00:00:00.000001Z -> 3000-01-01 00:00:00+00:00
9999-12-31T23:59:59.999999Z -> ValueError: year 10000 is out of range
Cause
_conv_instant is datetime.fromtimestamp(value.getEpochSecond() + value.getNano() / 1_000_000_000.0, tz=timezone.utc) (type_conversion.py L304-L309), and _conv_zoned_datetime computes the same from toInstant(), which _conv_offset_datetime reuses (type_conversion.py L311-L321). The sum is a double. LocalDateTime is converted from its integer fields and is not affected.
Expected
The same datetime that the Java value holds, to the microsecond, for every year Python can represent.
Proposed fix
Build the value from the integer seconds and microseconds, with no float in between. I checked this with a run-time monkeypatch that leaves the repository untouched: every row of the repro is then exact in both halves, including year 9999.
--- a/bindings/python/src/arcadedb_embedded/type_conversion.py
+++ b/bindings/python/src/arcadedb_embedded/type_conversion.py
-from datetime import date, datetime, time, timezone
+from datetime import date, datetime, time, timedelta, timezone
+_EPOCH = datetime(1970, 1, 1, tzinfo=timezone.utc)
+
+
def _conv_instant(value):
- return datetime.fromtimestamp(
- value.getEpochSecond() + value.getNano() / 1_000_000_000.0,
- tz=timezone.utc,
- )
+ return _EPOCH + timedelta(seconds=int(value.getEpochSecond()), microseconds=int(value.getNano()) // 1000)
def _conv_zoned_datetime(value):
- instant = value.toInstant()
- return datetime.fromtimestamp(
- instant.getEpochSecond() + instant.getNano() / 1_000_000_000.0,
- tz=timezone.utc,
- )
+ return _conv_instant(value.toInstant())Related issues
Nothing found upstream (the converter is bindings code).
In this repository's tracker no issue is about this. The searches for datetime, microseconds, and DATETIME_MICROS found #58 (closed), which fixed the Python-to-Java direction of datetime and did not touch the Java-to-Python converters of Instant and ZonedDateTime.
Related reports from the same review: #123 (lookup_by_key datetime).
Activity
bindings: numpy bool, int32 arrays, lookup_by_key, append_samples, ge…