Skip to content

Native async create_record: a linger thread shared by the process would remove the cost of executors of 97 to 159 records #406

Closed

Description

@tae898

I found this while fixing the one-record slowdown of #393 (refs #388). The collected path of NativeAsyncExecutor.create_record has a fixed cost per executor: _start_linger starts a thread for every executor that collects a record, and each collected record goes through the lock and a copy of its properties. For a small executor that cost is larger than what the columnar frame saves, so #393 sends the first 128 records of an executor one op each, as in 26.11.1.post3 (_ASYNC_DIRECT), and collects only beyond that, with a frame only when 64 or more records are collected (_ASYNC_FRAME_MIN), so a frame appears from 192 records.

What that leaves is a band. An executor of 129 to 191 records collects a few records, hands them over one op each and pays the collection cost without getting a frame. I measured it with an untimed micro (a new executor, N records, wait_completion; the native wheel built from b77f887 against the published 26.11.1.post3 native wheel; medians in microseconds; cores 14 and 15 of the laptop):

records in the executor post3 (us) b77f887 (us) ratio
1 97 91 0.94
64 821 874 1.06
96 1178 1223 1.04
127 1510 1592 1.05
128 1528 1611 1.05
129 1544 1805 1.17
150 1794 2136 1.19
191 2254 2642 1.17
192 2265 2147 0.95
200 2390 2158 0.90
256 3115 2281 0.73
500 5951 2939 0.49
1000 11969 4272 0.36

Up to 128 records the executor is the post3 path and the ratios sit at 0.94 to 1.06. In the band the head costs 198 to 388 us more than post3 per executor (ratios 1.11 to 1.19; I ran the micro twice, the table is the first session and the ratios of the second at 96, 128, 129, 150, 191, 192 and 256 records were 96: 1.07, 128: 1.04, 129: 1.14, 150: 1.11, 191: 1.11, 192: 0.87, 256: 0.75). From 192 records the frame carries 64 or more records and the executor is faster than post3 (0.36 at 1000 records).

I expected the cost of the band to be a near-constant amount per executor, so that its relative size would fall as the direct phase grows; this micro is too noisy to show that, and it does not show the amount as constant either. With a direct phase of 96 (an earlier head, 7854838) the band was 97 to 159 records and 100 records read 1.19 (ratios at 96, 97, 100, 128 and 159 records: 96: 1.03, 97: 1.15, 100: 1.19, 128: 1.15, 159: 1.12); 128 moved it out of the 100-record case of the laptop A/B. The band is not removed, it moved, and the release notes will say so.

The bare start of a Python thread, measured next to the earlier micro of this change, was about 30 us, so I do not think the thread start explains all of the 198 us or more. Before changing anything I would split the cost: thread start, the condition and lock traffic, the copy, and the first hand-over.

My proposal is a linger thread shared by the whole process: one daemon thread, started at first use, that keeps the due times of the executors with a pending buffer. An executor then starts no thread, and the direct phase of 128 records could shrink to a few records or go away, which would also remove the band. I would measure that against this table before deciding. The data are untimed micro numbers, not a study figure; the laptop A/B of the post4 glue (the sizes 1, 100, 1k and 100k) has the study figures.

The micro script (`micro_async_fix3.py`)
# micro: one fresh async executor, N records, wait_completion (us per executor-call): post3 against a PR 393 wheel; argv[1] = label, argv[2] = optional _ASYNC_DIRECT override, argv[3] = optional _ASYNC_FRAME_MIN override
import sys, time, tempfile, threading, statistics
import arcadedb as a
if len(sys.argv) > 2 and sys.argv[2]:
    from arcadedb._native import extras
    extras._ASYNC_DIRECT = int(sys.argv[2])
    if len(sys.argv) > 3 and sys.argv[3]:
        extras._ASYNC_FRAME_MIN = int(sys.argv[3])
d = tempfile.mkdtemp(prefix="micro-async2-")
db = a.create_database(d + "/db")
db.command("sql", "CREATE DOCUMENT TYPE T")
PC = time.perf_counter
def one(n_rec, reps):
    out = []
    for _ in range(reps):
        docs = []
        for i in range(n_rec):
            x = db.new_document("T"); x.set("a", i); x.set("s", f"s{i}"); docs.append(x)
        ex = db.async_executor()
        t0 = PC()
        for x in docs:
            ex.create_record(x)
        ex.wait_completion()
        out.append((PC() - t0) * 1e6)
    return statistics.median(out)
for _ in range(30):
    one(1, 1)
SIZES = (1, 64, 96, 127, 128, 129, 150, 191, 192, 200, 256, 500, 1000)
res = {n: one(n, 300 if n < 10 else (100 if n <= 200 else 40)) for n in SIZES}
print(sys.argv[1], "median us per executor-call (create + wait):", {k: round(v, 1) for k, v in res.items()})
db.close()

Edited 2026-11-15 19:52 KST: the table and the numbers are now for the head b77f887 (direct phase of 128, band 129 to 191, two micro sessions); the first text had the head 7854838 (direct phase of 96, band 97 to 159).

Activity

  1. self-assigned this
    on Nov 15, 2026
  2. added
    performanceSpeed or memory of the bindings, with a measurement
    nativeThe arcadedb package (native library build)
    on Nov 15, 2026
  3. added
    concurrencyThreads, transactions and conflicts
    pythonPull requests that update python code
    on Nov 15, 2026
  4. added this to the 26.12.1 milestone on Nov 15, 2026
  5. tae898 commented on Nov 21, 2026

    @tae898
    CollaboratorAuthor

    The A/B for the shared linger thread (#417) is done.

    Measured on the laptop (cores 14 and 15, the owner away, the 1-minute load about 1 before the run), 12 interleaved rounds of the micro of this issue, paired ratios with a bootstrap interval. The published 26.11.1.post4 native wheel in every arm, with only _native/extras.py swapped: A is the merge base, B is the branch at 8d4a0bf343, C is the merge base again as the control. The control stays between 0.957 and 1.024, so the noise is about one percent.

    records A median us B median us B/A (95% CI) C/A control (95% CI) verdict by the rule of #317
    1 104.0 103.0 1.005 [0.977, 1.057] 0.991 [0.957, 1.024] GRAY
    64 900.1 895.0 1.000 [0.977, 1.025] 0.997 [0.984, 1.014] NO CHANGE
    96 1233.0 1234.0 1.001 [0.995, 1.007] 0.996 [0.992, 1.001] NO CHANGE
    127 1626.6 1616.1 0.994 [0.986, 1.001] 1.002 [0.991, 1.003] NO CHANGE
    128 1638.4 1627.7 0.995 [0.991, 0.998] 0.993 [0.990, 1.001] NO CHANGE
    129 1827.3 1650.8 0.904 [0.898, 0.917] 0.987 [0.985, 0.999] GRAY
    150 2148.3 1921.2 0.901 [0.890, 0.910] 0.993 [0.978, 1.004] GRAY
    191 2665.2 2439.9 0.915 [0.904, 0.922] 0.998 [0.982, 1.012] GRAY
    192 2181.6 1995.0 0.913 [0.891, 0.926] 1.004 [0.981, 1.014] GRAY
    200 2202.9 2041.1 0.925 [0.917, 0.935] 0.997 [0.987, 1.004] GRAY
    256 2375.4 2262.8 0.957 [0.947, 0.964] 1.007 [0.989, 1.020] GRAY
    500 3080.8 2894.1 0.940 [0.932, 0.948] 1.003 [0.995, 1.006] GRAY
    1000 4428.4 4234.9 0.953 [0.950, 0.962] 1.006 [0.998, 1.022] GRAY

    Up to 128 records nothing changes (B/A between 0.994 and 1.005). From 129 records, where the linger thread starts, B takes 4 to 10 percent less time than A. By the rule of #317 that is GRAY, not a WIN: no row has its upper bound under 0.90. The count that moves is threads: at the end of the last round A holds 621 threads, B holds 2 and the control holds 621, because every executor of A leaves its own blocked linger thread behind.

    This is one host and one mode (open). I put the same numbers in the pull request and take it out of draft, so its platform jobs run.

  6. tae898 commented on Nov 21, 2026

    @tae898
    CollaboratorAuthor

    Merged as #417 (bb8782c), after a rebase onto main whose platform jobs all passed (43 of 43). The measurement is the one posted above: no change up to 128 records, 4 to 10 percent less time from 129 records (GRAY by the rule of #317), and the thread count at the end of the run 2 instead of 621. Closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

concurrencyThreads, transactions and conflictsnativeThe arcadedb package (native library build)performanceSpeed or memory of the bindings, with a measurementpythonPull requests that update python code

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions