Skip to content
This repository was archived by the owner on Nov 21, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions bindings/python/docs/development/bridge.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ sources live in `bindings/python/src/java/com/arcadedb/python/`:
| `TimeSeriesBatcher` | Fills the engine's primitive `TimeSeriesBatch` one column per call, so numeric samples are never boxed |
| `RowAccess` | Hands a row's names and values to Python in one call (`namesAndValues`), and up to N such rows per call (`nextRows`); the values are the engine's own objects, so Python converts them with full type fidelity |

`RowBatcher.nextJsonBatch` and `RowAccess.nextRows` return fewer rows than asked for only when the result set is drained, and they close it before returning. Python therefore stops after a short batch and makes no further call into the bridge, not even `close()`: a one-row `to_list()` or `to_json_list()` costs one call into the bridge.

## Why it exists

Every JPype call from Python into the JVM pays a fixed boundary-crossing tax
Expand Down
1 change: 1 addition & 0 deletions bindings/python/docs/development/testing/test-core.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ Tests validate:
- **named parameters**: a dict on both binds `:v` (null) and `:w`; a list holding one dict is still the named map, in SQL and in openCypher
- **async**: `args=[None]`, `(None,)`, and `[None, 1]` store a null without an error
- **one Java overload per shape**: a JPype proxy of the `Database` interface, and of `DatabaseAsyncExecutor`, records the argument each shape arrives as
- **scalar parameters take the short path** (`TestScalarParametersTakeTheShortPath`): a dict of `str` keys with `int`, `float`, `str`, `bool`, or `None` values, and positional scalars, are bound without `convert_python_to_java` (a spy on it sees no call); the answers equal the general path's, for every scalar type including a 2**40 integer and a null. A `Decimal`, a list value, a subclass of `int`, and a numpy scalar still take the general path

### Other Features

Expand Down
10 changes: 10 additions & 0 deletions bindings/python/docs/development/testing/test-resultset.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,16 @@ Since 26.10.1's parallel scan (ArcadeData/arcadedb#8524) a query whose `LIMIT` i
- **exhaustion, first and one close the Java result set**: `list(rs)`, `to_list()`, `first()`, and `one()` each leave the result set closed.
- **a set closed before its end raises when read again**: after `first()`, each of `list()`, `to_list()`, `first()`, `count()`, `iter_json_batches()`, and `to_columns()` raises `ArcadeDBError` ("closed before"), and so does `to_list()` after a `with` block that took one row; a set read to its end reads as empty through `list()`, `to_list()`, `first()`, and `iter_json_batches()`.

### A small result costs one bridge call (`TestSmallResultsCostOneBridgeCall`)

A JPype call costs 3 to 4 microseconds, a fifth of a one-row read. `RowAccess.nextRows` and `RowBatcher.nextJsonBatch` return fewer rows than asked for only when the result set is drained, and close it themselves, so `to_list()` and `to_json_list()` make exactly one bridge call for a result that fits one batch, and no separate `close()` call.

- **`to_list()` makes one call for a short result**, including an empty one: a counting stand-in for the bridge class sees one `nextRows`, and the set is closed and exhausted
- **`to_json_list()` makes one call** for a short result
- **a full batch still ends on the next call**: exactly 512 rows (the `to_list()` batch) take two calls and return every row
- **a drained set reads as empty and is not an error**: `to_list()`, iteration, `first()`, and `iter_json_batches()` on a drained set return nothing, and `close()` stays idempotent
- **the bridge releases the engine cursor**: 60,000 documents walked in `@rid` pages of 5,000 with `to_list()` and `to_json_list()`, nobody calling `close()`, finish (a LIMIT that stops a parallel scan early parks its producers until the result set is closed, ArcadeData/arcadedb#8594)

### Results and records after their database is closed (`test_results_after_close.py`)

A result, a result set, or a record keeps the `Database` it came from alive and raises `ArcadeDBError` ("Database is closed") once that database is closed (humemai/arcadedb-embedded-python#117). Before, a record row read `{}`, a record property read `None`, and a plain scan raised a raw `TransactionException`.
Expand Down
23 changes: 23 additions & 0 deletions bindings/python/src/arcadedb_embedded/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@
# free and used to run per record in wrapper dispatch).
_JAVA_CLASSES = {}

# Parameter values that cross as they are (exact types, not subclasses): see
# Database._java_parameters.
_SCALAR_PARAM_TYPES = frozenset((int, float, str, bool, type(None)))


def _java_class(name):
cls = _JAVA_CLASSES.get(name)
Expand Down Expand Up @@ -130,6 +134,25 @@ def _java_parameters(args):
values = (
args[0] if len(args) == 1 and isinstance(args[0], (list, tuple)) else args
)
# THE COMMON CASE, WITHOUT THE GENERAL MACHINERY: parameters that are all
# plain scalars (a dict of str keys, or positional values) need no
# per-value conversion, and JPype boxes them exactly as the general path's
# unchanged values are boxed. The general path costs 6.5 to 9 us a call on
# the laptop, the fast one 4.3 to 5.7 us, on a statement that is 15 to
# 60 us in all. Exact types only: a bool is a bool, and a numpy scalar, a
# Decimal, or a date is anything else and takes the general path.
scalars = _SCALAR_PARAM_TYPES
if len(values) == 1:
first = values[0]
if type(first) is dict and all(
type(k) is str and type(v) in scalars for k, v in first.items()
):
params = _java_class("java.util.HashMap")()
for key, item in first.items():
params.put(key, item)
return jpype.JObject(params, _java_class("java.util.Map"))
if all(type(a) in scalars for a in values):
return jpype.JArray(jpype.JObject)(values)
if len(values) == 1 and isinstance(values[0], Mapping):
java_map = _java_class("java.util.Map")
params = values[0]
Expand Down
26 changes: 21 additions & 5 deletions bindings/python/src/arcadedb_embedded/results.py
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,18 @@ def _finish(self) -> None:
self._exhausted = True
self.close()

def _drained(self) -> None:
"""The bridge returned its last batch and closed the Java result set itself.

``RowBatcher.nextJsonBatch`` and ``RowAccess.nextRows`` return fewer rows
than asked for only when the result set is drained, and close it before
returning, so Python marks the set finished without another JVM
crossing (a JPype call is 3 to 4 microseconds, a fifth of a one-row
read through ``to_json_list()``).
"""
self._exhausted = True
self._closed = True

def __next__(self) -> "Result":
if self._readable():
try:
Expand Down Expand Up @@ -341,16 +353,19 @@ def to_list(self, convert_types: bool = True) -> List[Dict[str, Any]]:
batch = row_access.nextRows(self._java_result_set, 512)
except JException as exc:
raise _read_error(exc) from exc
if len(batch) == 0:
self._finish()
return out
count = len(batch)
for pair in batch:
out.append(
{
str(name): convert_java_to_python(value)
for name, value in zip(pair[0], pair[1])
}
)
if count < 512:
# a short batch is the last one, and the bridge closed
# the result set: no second call just to see an empty batch
self._drained()
return out
return list(self.iter_dicts(convert_types=convert_types))

def iter_dicts(self, convert_types: bool = True) -> Iterator[Dict[str, Any]]:
Expand Down Expand Up @@ -472,8 +487,9 @@ def iter_json_batches(
if len(batch) < size:
# nextJsonBatch stops short only when the result set is drained, so a
# short batch is the last one: no second call (a JVM crossing, a
# str(), and a json.loads) just to see "[]".
self._finish()
# str(), and a json.loads) just to see "[]". It closed the result
# set itself, so there is no close() crossing either.
self._drained()
if batch:
yield batch
return
Expand Down
6 changes: 5 additions & 1 deletion bindings/python/src/java/com/arcadedb/python/RowAccess.java
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,16 @@ public static Object[] namesAndValues(final Result row) {

/**
* Up to {@code max} rows, each as {@link #namesAndValues(Result)}; an empty array once the result set is drained.
* For callers that consume the whole result: rows are taken from the result set as they are batched.
* For callers that consume the whole result: rows are taken from the result set as they are batched. Fewer than
* {@code max} rows means the result set is drained, and it is closed here (see {@link RowBatcher#closeDrained}), so
* a caller stops after a short batch and does not use the result set again.
*/
public static Object[] nextRows(final ResultSet rs, final int max) {
final List<Object[]> rows = new ArrayList<>(Math.min(max, 1024));
while (rows.size() < max && rs.hasNext())
rows.add(namesAndValues(rs.next()));
if (rows.size() < max)
RowBatcher.closeDrained(rs);
return rows.toArray();
}
}
17 changes: 16 additions & 1 deletion bindings/python/src/java/com/arcadedb/python/RowBatcher.java
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ private RowBatcher() {
* Serialize up to {@code max} rows of the result set into a JSON array
* string. Returns fewer than {@code max} rows only when the result set is
* drained, so a caller can stop after a short batch instead of calling again
* to see {@code "[]"}.
* to see {@code "[]"}, and a drained result set is closed here, so the caller must not use it again.
*
* <p>The builder starts at the default size and grows: a 64 KB start was
* allocated on every call and cost 2.5x the engine lookup on a one-row
Expand All @@ -58,9 +58,24 @@ public static String nextJsonBatch(final ResultSet rs, final int max) {
appendRow(sb, row);
n++;
}
if (n < max)
closeDrained(rs);
return sb.append(']').toString();
}

/**
* The result set is drained, so the caller will not come back: close it here instead of making Python cross the
* bridge once more just to call close() (a JPype call costs 3 to 4 microseconds, a fifth of a one-row read).
* Best effort, like the Python-side close it replaces.
*/
static void closeDrained(final ResultSet rs) {
try {
rs.close();
} catch (final Exception ignored) {
// closing is hygiene; the rows were read
}
}

private static void appendRow(final StringBuilder sb, final Result row) {
final JSONObject obj = new JSONObject();
for (final String property : row.getPropertyNames())
Expand Down
101 changes: 101 additions & 0 deletions bindings/python/tests/test_parameter_binding.py
Original file line number Diff line number Diff line change
Expand Up @@ -209,3 +209,104 @@ def received(method, args=None, **params):
assert received(method, [{"p": 1}]) == ("Map", {"p": 1})
assert received(method, p=None) == ("Map", {"p": None})
assert seen == []


class TestScalarParametersTakeTheShortPath:
"""Plain scalars cross as they are, without the per-value conversion walk.

A dict of str keys with scalar values, or positional scalars, is bound with a HashMap or an
Object[] built directly: 4 to 6 microseconds a call against 6.5 to 9 for the general path.
Exact types only; anything else (numpy scalars, Decimal, date, list, a subclass of int) keeps
the general path and its conversions.
"""

@pytest.fixture
def counted(self, monkeypatch):
from arcadedb_embedded import core

calls = []
real = core.convert_python_to_java

def spy(value):
calls.append(type(value).__name__)
return real(value)

monkeypatch.setattr(core, "convert_python_to_java", spy)
return calls

def test_scalar_dict_is_not_converted(self, db, counted):
rows = db.query(
"sql", "SELECT k FROM T WHERE v = :v AND k = :k", {"v": 5, "k": "set"}
).to_list()
assert rows == [{"k": "set"}]
assert counted == []

def test_positional_scalars_are_not_converted(self, db, counted):
rows = db.query(
"sql", "SELECT k FROM T WHERE v = ? AND k = ?", 5, "set"
).to_list()
assert rows == [{"k": "set"}]
assert counted == []
rows = db.query("sql", "SELECT k FROM T WHERE v = ?", [5]).to_list()
assert rows == [{"k": "set"}]
assert counted == []

def test_every_scalar_type_binds_as_before(self, db):
with db.transaction():
db.command(
"sql",
"INSERT INTO T SET k = :k, i = :i, f = :f, b = :b, n = :n, big = :big",
{"k": "all", "i": 7, "f": 2.5, "b": True, "n": None, "big": 2**40},
)
db.command(
"sql",
"INSERT INTO T SET k = ?, i = ?, f = ?, b = ?, n = ?, big = ?",
"pos",
7,
2.5,
False,
None,
2**40,
)
rec = _records(db)
assert (
rec["all"]["i"] == 7 and rec["all"]["f"] == 2.5 and rec["all"]["b"] is True
)
assert rec["all"]["n"] is None and rec["all"]["big"] == 2**40
assert rec["pos"]["b"] is False and rec["pos"]["i"] == 7

def test_the_overloads_still_see_the_same_kind_of_argument(self, db):
# a dict goes to the Map overload and positional values to Object...: identical answers
named = db.query("sql", "SELECT k FROM T WHERE v = :v", {"v": 5}).to_list()
positional = db.query("sql", "SELECT k FROM T WHERE v = ?", 5).to_list()
assert named == positional == [{"k": "set"}]

def test_non_scalars_keep_the_general_path(self, db, counted):
import decimal

db.query(
"sql", "SELECT k FROM T WHERE v = :v", {"v": decimal.Decimal("5")}
).to_list()
assert counted # converted
del counted[:]
db.query("sql", "SELECT k FROM T WHERE v IN :vs", {"vs": [5, 6]}).to_list()
assert counted

def test_a_subclass_of_a_scalar_is_not_assumed_to_be_one(self, db, counted):
class MyInt(int):
pass

rows = db.query(
"sql", "SELECT k FROM T WHERE v = :v", {"v": MyInt(5)}
).to_list()
assert rows == [{"k": "set"}]
assert counted # took the general path

def test_numpy_scalars_keep_the_general_path(self, db, counted):
np = pytest.importorskip("numpy")
rows = db.query("sql", "SELECT k FROM T WHERE v = ?", np.int64(5)).to_list()
assert rows == [{"k": "set"}]
rows = db.query(
"sql", "SELECT k FROM T WHERE v = :v", {"v": np.float64(5.0)}
).to_list()
assert rows == [{"k": "set"}]
Loading
Loading