[rafatxf]

With --pcie-frac 0 the pool never plans PCIe experts, yet every layer of a verify window still runs the PCIe-share stage over zero groups: the flag B wait, fetch_blobs + rebase_ptrs (staging mode 2) and the grouped gate/up, SwiGLU and down launches with kPcieGroupRows rows that find nothing to do. Six dependent kernels, ~18 µs a layer on an RTX 2080 Ti (kernel-node nsys), ~0.86 ms of a ~25 ms window.

STRATA_NO_PCIE_GROUP=1 records the window graph without that stage. It is opt-in and only valid with --pcie-frac 0 (the graphs are captured once, and the share can otherwise be refined at run time): if the pool ever publishes a plan with PCIe experts, the host loop stops the engine with a message (STRATA_NO_PCIE_GROUP=1 needs --pcie-frac 0 (layer L planned N PCIe experts)) after releasing the GPU's waits, instead of letting the GPU skip experts. I checked that path by starting the engine without --pcie-frac 0: it stops at the first window that plans a PCIe expert.

On this card, with the expert cache warm (~96-97 % VRAM hits in decode), the CPU takes the few misses the PCIe share would have taken without a measurable loss: --pcie-frac 0 alone was neutral (Spanish 86.9 vs 87.8 t/s, code 106.3 vs 106.1, 32K 78.2 vs 78.4 - within the run-to-run spread).

Setup

RTX 2080 Ti 22 GB (sm_75), i7-6700 (4 cores), 56 GB DDR4-2133, PCIe 3.0 x16, CUDA 13.0, SM 1650 MHz, 240 W. Flash-Next, canonical Q2_0 pack, --resident-experts, --max-context 524288, --serve, with the other open Turing decode PRs applied.

Engine (512 new tokens, two runs each)

	--pcie-frac auto	--pcie-frac 0 + STRATA_NO_PCIE_GROUP=1
Spanish chat (57-token prompt)	87.8 t/s	89.8 t/s	+2.3 %
code	106.1	108.4	+2.2 %
English, 32K	78.4	80.1	+2.2 %
English, 128K	75.0	76.8	+2.4 %
English, 389,560 tokens	74.7	75.1	+0.6 %

The prompt reads are unchanged (32K 17.6 s, 390K 223.6 s).

A follow-up could make this automatic when --pcie-frac 0 is given (no env), if you prefer that.

Part of the Turing set tracked in #2197.

🤖 Generated with Claude Code

[Niko1221]

Thanks rafatxf. We ported this and measured it: interleaved 6 pairs with --pcie-frac 0 against --pcie-frac 0 plus STRATA_NO_PCIE_GROUP=1, on an RTX 3060 (53.36 vs 53.16 tok/s, 3/6 pairs faster) and an RTX A4000 (61.72 vs 62.17, 4/6). That is noise on those cards, and your +2.2 % on the 2080 Ti is under the 3 % we ask for a new switch, so we are not taking it for now. If a Turing owner can reproduce 3 % or more, or you see it grow with larger verify windows, please tell us and we will take it as an opt-in.

[rafatxf]

Measured on the 0.1.44 build with the series, 2x RTX 2080 Ti in a layer split with --pcie-frac 0, Q2_0 pack, the harness from #2069. With and without STRATA_NO_PCIE_GROUP=1, the mean decode of es/code/4K/16K is 114.8 vs 113.1 tok/s (+1.5 %). That is under your 3 % bar, so I'm closing this one. Thanks for measuring it on the 3060 and the A4000.