[RubenTorney]

Title

Issue: #1903

Summary

On Windows cudaHostRegister of the whole arena fails (WDDM shared budget, about half the RAM) and the sliced fallback
registers the first ~30 GiB, layers 0-31 for Qwen3.8-Flash-Next IQ3_S: layers 32-47 never get a PCIe share, while ~10 GiB of the
registered prefix hold GPU-resident experts that DMA never reads. With STRATA_PIN_BY_NEED=1 the arena gives every blob a
page-aligned stride (+0.15 % RAM) and registers the expert profile's non-GPU-resident experts by rank, up to
STRATA_ARENA_PIN_GIB (29 GiB by default on Windows, to leave the shared segment some room). STRATA_PCIE_MIN1=1 sends one
miss over PCIe as soon as a layer has two (the share (nmiss * pcie_num) >> 8 rounds to 0 at 1-2 misses). Both opt-in.

RTX 3090 + Ryzen 5 5600X: 28.96 GiB registered = 92 % of the non-resident experts, all 48 layers get a PCIe share.
Prompt read +5-7 % at 56K-117K tokens (2,172 → 2,313 and 2,091 → 2,208 tok/s). Decode: level on the blocking tier
(x0.999 cold, 6/12), x1.02-1.03 (10 of 12 prompts faster) on top of --adapt-async 1 (the other PR) with --pcie-frac 0.6.

What changed

• src/core/pinned.cu, include/strata/core/pinned.hpp: a PinnedArena constructor that registers a list of byte ranges
  in order before any page is touched, stops at the cap or the first refusal, and unregisters them in the destructor;
  PadMap, the per-blob placement used by the loaders (load_experts_ranges, load_experts_gguf) when the layout is padded.
  STRATA_ARENA_PIN_GIB=auto applies to this path too.
• src/platform/memory.cpp, .hpp: lock_resident_ranges keeps the holes between the ranges resident (one working-set raise).
• src/core/expert_source.cpp, .hpp (ArenaExpertSource): arena_off() (one offset function for blob, blob_at,
  pinned, device_alias, the ranges); set_pin_priority; pinned() / device_alias() / pcie_layer() by ranges (a blob is
  pinned only if it lies wholly inside one registered range); STRATA_PCIE_MIN1 in the PCIe share.
• src/program/generate.cpp: before the arena is loaded, the profile's pairs past the estimated GPU-resident count (from
  --expert-cache or the free VRAM, as the cache's auto computes it) are ranked into ranges; one log line
  pin by need: N ranges, X GiB pinned, Y % of the non-resident experts.
• docs/DETAILS.md, bench/results/2026-10-11-community-rtx3090-5600x-arena-pin-by-need/.

How the default path was checked

• Without STRATA_PIN_BY_NEED the layout, the loaders and the sliced pin are untouched. Bit-exact mode
  (STRATA_IQ_MT_MIN=1 --pcie-frac 0 --adapt-every 0), greedy, 6 prompts: on the 5 prompts that v0.1.42 reproduces between
  two of its own fresh engines, this branch without the variables gives the same answers byte for byte (sha256). The 6th
  prompt differs between two runs of v0.1.42 itself in that mode on this machine.
• Built: CUDA only (Windows, VS 2022 17.14, CUDA 12.8, sm_86). Not built: HIP, SYCL; pinned.cu and expert_source.cpp
  are shared with those builds (the new code is host code behind the variable).

Extra Notes

• The padded layout reads experts.bin through the buffered loader (a contiguous unbuffered read cannot land strided); load
  speed was unchanged here (6.7 GiB/s).
• STRATA_VERIFY_ARENA's checksum is over the padded layout when the variable is on (not comparable with the packed one).
• The resident count is estimated before the cache exists; an error costs DMA-registered bytes on resident experts or skips a
  few hot ones, never correctness (STRATA_PIN_BY_NEED_SKIP=N sets it).
• Independent of the other PR (no shared code); the decode gain shows only with it.

🤖 Generated with Claude Code

https://claude.ai/code/session_015oFHMEgBzk7kQSqUVWoc67

[Niko1221]

Thanks, this is a good diagnosis of the WDDM budget problem (prefix pinning that spends the budget on layers whose experts the GPU already holds).

We ported it to the 0.1.44 branch with your authorship kept, as an opt-in, with a few changes of ours (separate commits):

• On a 12 GB RTX 5070 / Windows / Q2_0 (arena 31.6 GiB, the default sliced pin already covers 30 GiB of it) the version as submitted made the prompt read 9.5% slower (0 of 5 pairs): by the time of the prompt the adaptive tier has changed the cache, so a tenth of what the prompt streams was not pinned. We now put the pairs taken as GPU-resident into a second tier of ranges that only gets what the cap leaves over. Result: prompt read +4.4% (5 of 5 pairs), decode +0.9%; with the pin budget capped at 16 GiB the prompt read is +8.5% (5 of 5).
• blob_at (from the other PR) follows the padded layout; no padded layout with --shared-expert-arena.
• load_experts_ranges with the PadMap is an overload instead of a default argument: the SYCL build has its own pinned.dp.cpp and failed to link with the extra parameter.
  STRATA_PCIE_MIN1 did not help on our card (async + pin + MIN1 was slower than async alone), so it stays a separate switch. Gates: answers identical with the switches on and off (pcie-frac 0, 10 prompts + 4K/20K on CUDA sm_86, HIP gfx1201, and the 5070), builds on CUDA, HIP, SYCL, Metal. It is CUDA/HIP only and aimed at Windows, so it stays opt-in; we will name it in the 0.1.44 notes with your name.