[imanu86]

Title

Issue: none (a start-up default that misfired on a slow SSD, and read diagnostics).

Summary

Since 0.1.42 the engine probes the PLE table at start-up and, under 15,000 rows/s, switches --ple-io direct to
mmap on its own. The probe reads 64 rows a batch from a cold table, a far lower queue depth than a prompt chunk's
batched read, so a healthy SATA SSD can measure "slow" and the switch costs a long prompt more than three times its
time. This change keeps the probe and its warning but makes the switch opt-in (STRATA_PLE_AUTO_MMAP=1); an
explicit --ple-io is untouched. It also adds a few PLE read diagnostics.

Why

Measured on one PC (PLE table on a SATA SSD, 131K-token prompt, two-card layer split): the probe read 1,259 rows/s,
switched to mmap, and the prompt took 260 s; the same engine with direct I/O took 75 s and decode was 3.9%
faster (the mmap mode also drops the row cache and the in-flight reads). mmap faults the rows in one 4 KiB page at a
time; direct I/O keeps --ple-inflight reads in flight and the chunk read-ahead hides them. The probe's figure is a
cold, low-depth read and does not predict the prompt path's throughput.

What changed

• src/program/generate.cpp: the probe still runs and prints PLE direct read probe: N rows/s; under the threshold it
  warns and keeps direct unless STRATA_PLE_AUTO_MMAP=1 (then it switches as before). With
  STRATA_DECODE_TIMING the decode summary adds the primary verifier's PLE gather time per window and the reader's
  cumulative counters before and after the request (strata decode PLE primary/before/after).
• src/ngram/ple_reader.cpp, include/strata/ngram/ple_reader.hpp: ple io: adds the read mean, the maximum and the
  count of reads over 100 ms (all reads, where p50/p99 cover the last 65,536).
• src/prefill/prefill.cpp: a chunk that waited more than 1 s for its PLE rows logs one line
  (strata prefill: the chunk at token N waited X ms for its PLE rows).
• src/kernels/ngram.cpp, include/strata/core/verify.hpp: the gather time counter.

No kernel changes; the output tokens are the same in both I/O modes.

Measurements

RTX 2080 Ti 22 GB + RTX 3060 12 GB, Windows 11, CUDA 12.6, --layer-split 16 --pipeline-windows 2, Qwen3.8-Flash-Next
IQ3_XXS, 131K-token prompt, PLE table on a SATA SSD (cold):

	prompt 131K	decode (131K, 1024 tokens)
0.1.42 default (probe -> mmap)	260 s	reference
direct kept (this change)	75 s	+3.9%

With the table on an NVMe the probe says fine (91,220 rows/s here) and nothing changes.

Built and started on v0.1.43 (7695859a, RTX 2080 Ti + RTX 3060 layer split, Windows): the probe reports 43,290 rows/s (fine) on the NVMe, the engine serves as before; the same code has run in this fork's daily build since 0.1.42.

Risks / limits

A disk that really is too slow for direct I/O keeps direct unless STRATA_PLE_AUTO_MMAP=1 is set; the warning says
so and names the variable. --ple-io ram and an explicit --ple-io mmap behave as before.

Part of a series from a public patch registry (stack/ in the fork imanu86/Strata-2080Ti): one lever per PR, each
opt-in or bit-exact, measured on the same two-card box.

[imanu86]

@Niko1221 this is the first of a short series of small, independent PRs from our two-card (RTX 2080 Ti + RTX 3060, layer split, Windows) fork, each one lever, opt-in or bit-exact, measured on the same box: next come the prompt-prefix file, the Turing decode attention (once its parity harness has run here), and three Turing prefill pieces. They all come from the fork's public patch registry, https://github.com/imanu86/Strata-2080Ti/blob/main/stack/INDEX.md, which lists every patch we carry on top of v0.1.43 with its switch, origin, measured value and insertion anchors; #2058 and #2059 are the two that went out before this one. Happy to split, rename or drop anything.

[Niko1221]

Thanks. 0.1.43.1 already contains this change (probe warns, direct kept, STRATA_PLE_AUTO_MMAP=1 for the old switch). The extra diagnostics are welcome as a separate small PR.

[Niko1221]

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.