Skip to content

Native async create_record: one linger thread for the process instead of one per executor (refs #406) - #417

Merged
tae898 merged 1 commit into
mainfrom
perf/406-shared-linger
Nov 21, 2026
Merged

tae898 merged 1 commit into
mainfrom
perf/406-shared-linger

Conversation

@tae898

@tae898 tae898 commented Nov 21, 2026 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

On the native wheel, NativeAsyncExecutor.create_record collects the records after the first 128 of an executor and a linger thread hands a batch over after 50 ms. Until now every executor started its own thread for that, with its first collected record (_start_linger), and kept it, blocked in Condition.wait(), until close(). This draft replaces it with one daemon thread for the whole process, as proposed in #406.

What I changed, all in src/arcadedb/_native/extras.py:

  • A module-level _Linger keeps the due times of the collected batches in a heap on one daemon thread, started at first use. The first record of a batch calls add(executor, due) (_start_linger); an executor starts no thread.
  • An entry whose batch was handed over earlier by a flush point (wait_completion, close, query, a change of type, 1,000 records) is dropped when its time comes, without a thread (_linger_pending, a read without the lock). An entry whose batch is still the executor's current one starts a short-lived thread that runs _linger_fire, under the executor's _buf_lock, exactly the hand-over the old thread made. It is a separate thread because a hand-over can block in the engine (a full queue, back pressure): with a thread for each executor a blocked hand-over held back only its own executor, and I did not want the shared thread to change that.
  • os.register_at_fork(after_in_child=...) gives a forked child a fresh _Linger, since the child has none of the parent's threads and may inherit the lock held. This does not make the native library usable in a forked child; it only keeps the new lock from being a new way to hang.

What stays the same for a caller:

  • Ordering. The engine queue still sees the calls in the order they were made: every hand-over (flush point or linger) swaps the buffer out and hands it over while holding the executor's _buf_lock, and a create_record that arrives meanwhile waits for that lock and goes after it. Lock order is one: an executor's _buf_lock, then the _Linger lock. The timer thread never holds its lock while it touches an executor.
  • The 50 ms bound, the error that a linger hand-over meets (kept in _linger_error, raised by the next call that hands records over), per-record callbacks and error callbacks, close() flushing first and stopping the linger of that executor, _ASYNC_DIRECT = 128 and _ASYNC_FRAME_MIN = 64.
  • Durability and read-your-writes are those of the async API before: a collected record is not in the engine queue until a hand-over, and only wait_completion() on that executor covers it. This is Native: async create_record hands records over as columnar frames with per-record callbacks (refs #388, #315) #393's behaviour for records beyond the 128th and I did not change it. It does not touch commits or indexes, so I do not expect it to change the intermittent miss of Windows Python 3.14 parity: test_vector_read_your_writes missed 1 of 200 committed vectors once on main (intermittent) #409 (a sync-path commit and search); I have no evidence either way.
  • At interpreter exit the thread is a daemon, as before, so a batch that no flush point has seen is lost as before.

One thing is different on purpose: an executor that is dropped without close() is no longer kept alive by a thread blocked for good. Its heap entry keeps it for at most the linger time, so a dropped executor still hands its batch over after 50 ms.

Related issues

Refs #406

Performance changes

Measured on the laptop (cores 14 and 15, the owner away, the 1-minute load about 1 before the run), 12 interleaved rounds of the micro of this issue, paired ratios with a bootstrap interval. The published 26.11.1.post4 native wheel in every arm, with only _native/extras.py swapped: A is the merge base, B is the branch at 8d4a0bf343, C is the merge base again as the control. The control stays between 0.957 and 1.024, so the noise is about one percent.

records A median us B median us B/A (95% CI) C/A control (95% CI) verdict by the rule of #317
1 104.0 103.0 1.005 [0.977, 1.057] 0.991 [0.957, 1.024] GRAY
64 900.1 895.0 1.000 [0.977, 1.025] 0.997 [0.984, 1.014] NO CHANGE
96 1233.0 1234.0 1.001 [0.995, 1.007] 0.996 [0.992, 1.001] NO CHANGE
127 1626.6 1616.1 0.994 [0.986, 1.001] 1.002 [0.991, 1.003] NO CHANGE
128 1638.4 1627.7 0.995 [0.991, 0.998] 0.993 [0.990, 1.001] NO CHANGE
129 1827.3 1650.8 0.904 [0.898, 0.917] 0.987 [0.985, 0.999] GRAY
150 2148.3 1921.2 0.901 [0.890, 0.910] 0.993 [0.978, 1.004] GRAY
191 2665.2 2439.9 0.915 [0.904, 0.922] 0.998 [0.982, 1.012] GRAY
192 2181.6 1995.0 0.913 [0.891, 0.926] 1.004 [0.981, 1.014] GRAY
200 2202.9 2041.1 0.925 [0.917, 0.935] 0.997 [0.987, 1.004] GRAY
256 2375.4 2262.8 0.957 [0.947, 0.964] 1.007 [0.989, 1.020] GRAY
500 3080.8 2894.1 0.940 [0.932, 0.948] 1.003 [0.995, 1.006] GRAY
1000 4428.4 4234.9 0.953 [0.950, 0.962] 1.006 [0.998, 1.022] GRAY

Up to 128 records nothing changes (B/A between 0.994 and 1.005). From 129 records, where the linger thread starts, B takes 4 to 10 percent less time than A. By the rule of #317 that is GRAY, not a WIN: no row has its upper bound under 0.90. The count that moves is threads: at the end of the last round A holds 621 threads, B holds 2 and the control holds 621, because every executor of A leaves its own blocked linger thread behind.

The A/B is a script outside the repository, the one described on #406; the numbers are also posted there. I do not claim a win by the rule of #317. I claim that the executor no longer leaves a thread behind and that nothing gets slower. I did not change _ASYNC_DIRECT.

Docs

docs/modules/ROOT/pages/how-to/async-executor.adoc said that a thread of the executor hands a batch over and that a shared thread "would remove this cost". It now says what post4 does and what this change does, and that the figures on the page are post4's and the shared thread has not been timed.

Tests

tests/test_native_async_create_batch_unit.py (no JVM, no native library: the library call is a recorder, so these count crossings and threads, not seconds). The two tests that named the removed _linger_thread now check the process-wide thread. New: an executor starts no thread of its own (20 executors, the thread count does not move), 10 executors are handed over in time by the one thread, a stale entry starts no hand-over thread, the entry of an earlier batch does not cut a later batch short, the order of the calls holds across linger hand-overs and flush points, a hand-over that blocks holds back no other executor, _linger_after_fork starts afresh, and close() leaves nothing for the linger thread to hand over.

Checked so far: compile() and pyflakes on both files (one older global _upcall_ref warning that is not from this change), uvx pre-commit run --files on the three files (black reformatted my lines, the run after it is clean), bandit clean on both. I have not run the unit file under pytest yet (a draft skips the platform jobs, so CI will not either), so the box below stays open. Instead I ran the same cases as a plain script against the fake library, one real os.fork with a child that creates and hands over a batch, and a 203-record run of the #406 micro against the real native library of the post4 wheel with this branch's extras.py laid over it (203 of 203 records stored). I will run the unit file before marking this ready.

Not tested: a real fork on Windows (no os.fork; register_at_fork is guarded), the interaction with a running engine under back pressure (the blocked hand-over is simulated), and anything about speed.

Checklist

  • A regression test fails on the old code and passes on the new
  • The test suite that the change can affect passes
  • uvx pre-commit run --files <changed files> passes

@tae898 tae898 added concurrency Threads, transactions and conflicts native The arcadedb package (native library build) performance Speed or memory of the bindings, with a measurement python Pull requests that update python code labels Nov 21, 2026
@tae898 tae898 self-assigned this Nov 21, 2026
@tae898
tae898 marked this pull request as ready for review November 21, 2026 14:41
@tae898

tae898 commented Nov 21, 2026

Copy link
Copy Markdown
Collaborator Author

The red check on this pull request ("Test arcadedb (linux/amd64, Python 3.12)", 2 failed of 3,598) is not from this change. It is the two vector-metadata tests that name the engine's old default for addHierarchy, which ArcadeData/arcadedb#12616 changed on 2026-11-20; every pull request fails on them. The fix is in #419 (issue #418). I will update this branch on main after #419 is merged and let the platform jobs run again before merging.

… of one per executor (refs #406)

The collected path of NativeAsyncExecutor.create_record started a Python thread with the first collected record of every executor (_start_linger)
and kept it, blocked in Condition.wait(), until close(); an executor that is never closed also kept its thread and itself alive.

Now a module-level _Linger keeps the due times of the batches (a heap of (due, serial, executor)) on one daemon thread that is started at first use.
The first record of a batch registers the batch with add(); a flush point that hands the batch over earlier leaves the entry to expire, and an
expired entry whose batch is gone or replaced is dropped without a thread (_linger_pending, read without the lock). A batch that is still due is
handed over by _linger_fire on a short-lived thread, not on the timer thread: the hand-over can block in the engine (a full queue, back pressure),
and one executor's blocked hand-over must not hold back the batches of the others, as it could not when each executor had its own thread.

What stays: the hand-over runs under the executor's _buf_lock exactly as before (so the engine queue sees the calls in the order they were made, and a
caller's next create_record waits for it), the 50 ms bound, the error kept in _linger_error and raised by the next call that hands records over,
close() flushing first and stopping the linger of that executor, _ASYNC_DIRECT = 128 and _ASYNC_FRAME_MIN = 64 (they need a measurement, not a guess).
Lock order is one: an executor's _buf_lock, then the _Linger lock; the timer thread never holds its lock while it touches an executor.
Fork: os.register_at_fork(after_in_child) starts a fresh _Linger, since the child has no thread of the parent and may inherit its lock held.
This does not make the native library usable in a forked child; it only keeps the new lock and thread from being a new way to hang.

Not measured: whether this removes the 198 to 388 us that #406 saw for executors of 129 to 191 records. The code shows that the thread start,
the Thread and Event objects and the leaked thread per executor are gone from the caller's path, and that the copy of the properties and the lock per
collected record stay; the split of the measured cost is not established. The A/B script is outside the repository.

The two tests that named the removed _linger_thread now check the process-wide thread; new tests cover: no thread per executor, many executors
handed over in time, a stale entry starting no hand-over thread, the entry of an earlier batch not cutting a later one short, the order of the calls
across linger hand-overs and flush points, a blocked hand-over holding back no other executor, the fork reset, and close.
@tae898
tae898 force-pushed the perf/406-shared-linger branch from 8d4a0bf to bb8782c Compare November 21, 2026 15:56
@tae898

tae898 commented Nov 21, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main, which now has #419 (the fix for the two red vector-metadata tests). src/arcadedb/_native/extras.py is byte for byte what the A/B measured (sha256 starting 201165114a8f), so the numbers above stand for the new head. The platform jobs are running again.

@tae898
tae898 merged commit 5592084 into main Nov 21, 2026
43 checks passed
@tae898
tae898 deleted the perf/406-shared-linger branch November 21, 2026 17:01
@github-actions github-actions Bot added this to the 26.12.1 milestone Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

concurrency Threads, transactions and conflicts native The arcadedb package (native library build) performance Speed or memory of the bindings, with a measurement python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant