Repository navigation
Native async create_record: one linger thread for the process instead of one per executor (refs #406) - #417
Merged
Merged
Conversation
tae898
marked this pull request as ready for review
November 21, 2026 14:41
Collaborator
Author
|
The red check on this pull request ("Test arcadedb (linux/amd64, Python 3.12)", 2 failed of 3,598) is not from this change. It is the two vector-metadata tests that name the engine's old default for |
… of one per executor (refs #406) The collected path of NativeAsyncExecutor.create_record started a Python thread with the first collected record of every executor (_start_linger) and kept it, blocked in Condition.wait(), until close(); an executor that is never closed also kept its thread and itself alive. Now a module-level _Linger keeps the due times of the batches (a heap of (due, serial, executor)) on one daemon thread that is started at first use. The first record of a batch registers the batch with add(); a flush point that hands the batch over earlier leaves the entry to expire, and an expired entry whose batch is gone or replaced is dropped without a thread (_linger_pending, read without the lock). A batch that is still due is handed over by _linger_fire on a short-lived thread, not on the timer thread: the hand-over can block in the engine (a full queue, back pressure), and one executor's blocked hand-over must not hold back the batches of the others, as it could not when each executor had its own thread. What stays: the hand-over runs under the executor's _buf_lock exactly as before (so the engine queue sees the calls in the order they were made, and a caller's next create_record waits for it), the 50 ms bound, the error kept in _linger_error and raised by the next call that hands records over, close() flushing first and stopping the linger of that executor, _ASYNC_DIRECT = 128 and _ASYNC_FRAME_MIN = 64 (they need a measurement, not a guess). Lock order is one: an executor's _buf_lock, then the _Linger lock; the timer thread never holds its lock while it touches an executor. Fork: os.register_at_fork(after_in_child) starts a fresh _Linger, since the child has no thread of the parent and may inherit its lock held. This does not make the native library usable in a forked child; it only keeps the new lock and thread from being a new way to hang. Not measured: whether this removes the 198 to 388 us that #406 saw for executors of 129 to 191 records. The code shows that the thread start, the Thread and Event objects and the leaked thread per executor are gone from the caller's path, and that the copy of the properties and the lock per collected record stay; the split of the measured cost is not established. The A/B script is outside the repository. The two tests that named the removed _linger_thread now check the process-wide thread; new tests cover: no thread per executor, many executors handed over in time, a stale entry starting no hand-over thread, the entry of an earlier batch not cutting a later one short, the order of the calls across linger hand-overs and flush points, a blocked hand-over holding back no other executor, the fork reset, and close.
tae898
force-pushed
the
perf/406-shared-linger
branch
from
November 21, 2026 15:56
8d4a0bf to
bb8782c
Compare
Collaborator
Author
|
Rebased onto main, which now has #419 (the fix for the two red vector-metadata tests). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
On the native wheel,
NativeAsyncExecutor.create_recordcollects the records after the first 128 of an executor and a linger thread hands a batch over after 50 ms. Until now every executor started its own thread for that, with its first collected record (_start_linger), and kept it, blocked inCondition.wait(), untilclose(). This draft replaces it with one daemon thread for the whole process, as proposed in #406.What I changed, all in
src/arcadedb/_native/extras.py:_Lingerkeeps the due times of the collected batches in a heap on one daemon thread, started at first use. The first record of a batch callsadd(executor, due)(_start_linger); an executor starts no thread.wait_completion,close,query, a change of type, 1,000 records) is dropped when its time comes, without a thread (_linger_pending, a read without the lock). An entry whose batch is still the executor's current one starts a short-lived thread that runs_linger_fire, under the executor's_buf_lock, exactly the hand-over the old thread made. It is a separate thread because a hand-over can block in the engine (a full queue, back pressure): with a thread for each executor a blocked hand-over held back only its own executor, and I did not want the shared thread to change that.os.register_at_fork(after_in_child=...)gives a forked child a fresh_Linger, since the child has none of the parent's threads and may inherit the lock held. This does not make the native library usable in a forked child; it only keeps the new lock from being a new way to hang.What stays the same for a caller:
_buf_lock, and acreate_recordthat arrives meanwhile waits for that lock and goes after it. Lock order is one: an executor's_buf_lock, then the_Lingerlock. The timer thread never holds its lock while it touches an executor._linger_error, raised by the next call that hands records over), per-record callbacks and error callbacks,close()flushing first and stopping the linger of that executor,_ASYNC_DIRECT = 128and_ASYNC_FRAME_MIN = 64.wait_completion()on that executor covers it. This is Native: async create_record hands records over as columnar frames with per-record callbacks (refs #388, #315) #393's behaviour for records beyond the 128th and I did not change it. It does not touch commits or indexes, so I do not expect it to change the intermittent miss of Windows Python 3.14 parity: test_vector_read_your_writes missed 1 of 200 committed vectors once on main (intermittent) #409 (a sync-path commit and search); I have no evidence either way.One thing is different on purpose: an executor that is dropped without
close()is no longer kept alive by a thread blocked for good. Its heap entry keeps it for at most the linger time, so a dropped executor still hands its batch over after 50 ms.Related issues
Refs #406
Performance changes
Measured on the laptop (cores 14 and 15, the owner away, the 1-minute load about 1 before the run), 12 interleaved rounds of the micro of this issue, paired ratios with a bootstrap interval. The published
26.11.1.post4native wheel in every arm, with only_native/extras.pyswapped: A is the merge base, B is the branch at8d4a0bf343, C is the merge base again as the control. The control stays between 0.957 and 1.024, so the noise is about one percent.Up to 128 records nothing changes (B/A between 0.994 and 1.005). From 129 records, where the linger thread starts, B takes 4 to 10 percent less time than A. By the rule of #317 that is GRAY, not a WIN: no row has its upper bound under 0.90. The count that moves is threads: at the end of the last round A holds 621 threads, B holds 2 and the control holds 621, because every executor of A leaves its own blocked linger thread behind.
The A/B is a script outside the repository, the one described on #406; the numbers are also posted there. I do not claim a win by the rule of #317. I claim that the executor no longer leaves a thread behind and that nothing gets slower. I did not change
_ASYNC_DIRECT.Docs
docs/modules/ROOT/pages/how-to/async-executor.adocsaid that a thread of the executor hands a batch over and that a shared thread "would remove this cost". It now says what post4 does and what this change does, and that the figures on the page are post4's and the shared thread has not been timed.Tests
tests/test_native_async_create_batch_unit.py(no JVM, no native library: the library call is a recorder, so these count crossings and threads, not seconds). The two tests that named the removed_linger_threadnow check the process-wide thread. New: an executor starts no thread of its own (20 executors, the thread count does not move), 10 executors are handed over in time by the one thread, a stale entry starts no hand-over thread, the entry of an earlier batch does not cut a later batch short, the order of the calls holds across linger hand-overs and flush points, a hand-over that blocks holds back no other executor,_linger_after_forkstarts afresh, andclose()leaves nothing for the linger thread to hand over.Checked so far:
compile()andpyflakeson both files (one olderglobal _upcall_refwarning that is not from this change),uvx pre-commit run --fileson the three files (black reformatted my lines, the run after it is clean),banditclean on both. I have not run the unit file under pytest yet (a draft skips the platform jobs, so CI will not either), so the box below stays open. Instead I ran the same cases as a plain script against the fake library, one realos.forkwith a child that creates and hands over a batch, and a 203-record run of the #406 micro against the real native library of the post4 wheel with this branch'sextras.pylaid over it (203 of 203 records stored). I will run the unit file before marking this ready.Not tested: a real fork on Windows (no
os.fork;register_at_forkis guarded), the interaction with a running engine under back pressure (the blocked hand-over is simulated), and anything about speed.Checklist
uvx pre-commit run --files <changed files>passes