[zahariash]

This work was produced with the help of language models.

Addresses #1581.

Once a recursive join's frontier passes 1,000 nodes, each frontier and vertex-compute task is scheduled with launchNewWorkerThread, which starts a std::thread to take the waiting thread's scheduler slot. On a stand-in for the #1581 graph, 15,000 nodes and 104,745 hub-biased edges, that is about six threads per SHORTEST query.

In the multithreaded WASM build each of those threads needs a Web Worker. The scheduler starts one thread per hardware thread and the build pre-creates 8 Workers, so with 8 or more hardware threads none is spare. The recursive join runs on the thread blocked in the synchronous execute(), and Emscripten starts a new Worker only after its creator returns to the event loop, so the query waits for a thread that cannot start. This follows from the Emscripten runtime and its pthreads documentation; it has not been reproduced in a browser.

Fix. Since #1482 a waiting thread that holds a slot works on the task it waits for, which is what the new thread was for. GDS always runs inside a task, so the waiting thread now runs the task and pool workers join it. Recursive joins start no threads.

Native Release builds; each value is the mean of two rounds' medians of 9 runs, 5 on the generated graphs:

query	threads	main, ms	this PR, ms
#1581 stand-in, median of 50 pairs	1	9.0	7.0
#1581 stand-in, median of 50 pairs	14	7.8	7.8
IC13	1	28.3	25.1
IC13	16	49.0	47.7
IC14	1	227.2	213.4
IC14	16	140.1	135.7
1000×1000 grid, 200 hops	1	1,936	1,649
1000×1000 grid, 200 hops	16	868	842
2M-node random graph, 7 hops	1	1,200	1,234
2M-node random graph, 7 hops	16	767	777

No SF1 plan changes, and all 58 SF1 queries return the same rows up to order among ties and inside collect().

Cost. At one thread, searches on the random graph run 1 to 3% slower. The task's scan states are now allocated in the calling thread's malloc arena instead of a fresh thread's; with one arena (GLIBC_TUNABLES=glibc.malloc.arena_max=1) both builds run within 0.5%.

Tests. ApiTest.DenseRecursiveJoinWithOneThread runs a variable-length join with a dense second hop on a one-worker database: on the caller, on the worker under a query timeout, and with two threads requested. It passes on main, and hangs if the waiting thread does not run the task. Full ctest passes, 3,444 tests.

[adsharma]

Minors from reviews/PR-1582-review.md (Approve — non-blocking). Two consistency notes in body: (1) extension/algo/.../in_mem_gds_utils.cpp:36, extension/llm/.../ai_extract.cpp:203, ai_extract_record.cpp:471 still pass launchNewWorkerThread=true — worth a follow-up issue to apply the #1482 reasoning there; (2) WASM causality is doc-derived, keep #1581 open until a multithreaded-WASM measurement confirms, and the 1–3% random-graph delta is noise-adjacent.

[adsharma]

Minor — invariant lives only in prose. "GDS always runs inside a task" holds for all in-tree callers traced (processor.cpp:88 → ProcessorTask, recursive_extend.cpp:291-299), but a future direct caller (background index build, DDL, vector-index) would silently fall back to queue-and-wait with no slot. Consider REQUIRES: caller holds a scheduler slot naming or a DASSERT-style guard so the next caller can't miss it.

[adsharma]

Nit — bare default re-invites true. Both dense sites now rely on the default launchNewWorkerThread=false. Consider a trailing /* launchNewWorkerThread=false: waiter runs it (§1482) */ so nobody "optimizes" this back to true.

[adsharma]

Minor — guard, not repro + unexplained phase 3. As the body notes this passes on main and only hangs against a hypothetical pure-blocking waiter — worth stating in the test name/comment. Phase 3 (CALL threads=2 on a maxNumThreads=1 DB at :666, then re-run) needs a one-line note on capping semantics; readers will expect 2 threads to mean something. (3000-node hop vs 1000-node dense threshold is the load-bearing detail — good that it's commented.)

[zahariash]

Follow-ups:

• The invariant and the bare default in gds_utils.cpp: comments in #1610. It isn't an invariant, since a caller without a slot doesn't hang: it runs a one-thread task in a free slot or leaves the task to the workers. So the comment says that rather than asserting it, and the second call names launchNewWorkerThread.
• DenseRecursiveJoinWithOneThread: the comment in #1610 now says it guards against a waiter that leaves the task to the pool, and that threads=2 adds no worker but makes the task skip the one-thread shortcut.
• algo in_mem_gds_utils.cpp, llm ai_extract.cpp and ai_extract_record.cpp: LadybugDB/extensions#147. It relies on #1482, so it should ship with the 0.23.0 extensions: on 0.22.0–0.22.3 a task of two or more threads, or any task under a query timeout, can hang there when no other slot is free.
• #1581 stays open; the reporter is asked there to rerun on the next release.