[malloc32]

AI-written. This PR was prepared by an AI (Claude Opus 5.5, via Claude Code) at the submitter's request; the code, the port and the measurements were done by the AI on the submitter's machine, and the submitter reviewed it before opening.

Update 2026-10-07: rebased onto 0.1.40.2 (e8ca9af). The one conflict was the Stager's wait in src/prefill/prefill.cpp: the release's sleeping wait (stager_sleep(), Linux by default) now applies to each job of a claimed run, beside the batched reads.

What

A port of @adonizm's #833 onto the current main (it was closed by the history rewrite, not rejected). Two changes:

1. The prompt path's stager claims runs of consecutive jobs and reads them with one read_direct batch (ExpertSource::copy_blobs), merging neighbouring blobs into requests of up to 32 MiB. STRATA_STAGER_BATCH (default 8 for file-backed blobs, 1 otherwise).
2. Once the file tier reads unbuffered, the mapped view of experts.bin is closed (FileExpertSource::drop_mapping, Windows): while it is mapped, NTFS serves the unbuffered reads one at a time. STRATA_KEEP_MAPPING=1 keeps it.

Two fixes on the way: unmapped_ is declared on every platform (mapped_blob reads it outside the Windows block, so Linux did not compile), and copy_blobs asks resident_blob, which follows the exchange rotation (STRATA_EXCHANGE_ROTATE).

Measured

i7-12700K, 64 GB, 2x RTX 5060 Ti (layer split), IQ3_S, experts.bin on a Samsung 990 PRO, --resident-budget-gib 8, unbuffered reads: prompt 513 -> 623 tok/s at 8K, 962 -> 1,198 at 32K. Batching gives most of it; closing the view adds 4-5%. The original author's numbers: 1,261 -> 1,669 tok/s at 47.6K.

Not tested

Linux and HIP builds (the Linux fix was checked with a syntax-only compile).

🤖 Generated with Claude Code

[Niko1221]

Thank you malloc32 for bringing @adonizm's #833 back onto the current tree. It is in 0.1.41 with adonizm's authorship kept, merged with the sleeping-wait stager code. On the RTX 5070 Windows PC (file tier) the prompt time fell 45-52% with identical tokens in 18 of 18 runs. STRATA_STAGER_BATCH and STRATA_KEEP_MAPPING=1 are as written. We measured on Windows only. Closing as ported.