Repository navigation
Conversation
W1nge
force-pushed
the
codex/windows-prefill-staging-release
branch
from
October 8, 2026 23:44
9ee824e to
83d4f1a
Compare
Maddoxkkm
added a commit
to Maddoxkkm/Strata
that referenced
this pull request
Oct 10, 2026
…1221#1520 (release staged pages), Niko1221#1107 (grouped gather on short prompts) Three of the candidates we read apply and are not superseded by upstream: - Niko1221#1660 lets the MMQ resident sort take native-layout packs (slot/src/off are built the same way for both layouts; the PR measures 1.2K prompt 7.27 -> 5.32 s and 35K 63.3 -> 54.1 s on 4xGP100, same bits). - Niko1221#1520 extends STRATA_FILE_RELEASE to prefill staging copies (opt-in; the PR's own numbers: +11.7 GiB available RAM for 3.7-4.4% prefill - a trade worth testing on this 32 GB box). - Niko1221#1107 stages the short-prompt walk through a grouped ring (STAGE_GRP=32) instead of one slot at a time. Not applied, and why: Niko1221#1101 and Niko1221#1237 are already upstream in newer form (upstream has the stager sleep plus a stager_sleep helper and a batched copy_blobs loop, and pinned stage buffers behind STAGE_PIN_DEFAULT=0 - the PR's older always-pin revision would re-arm the IQ3_S decode corruption the release gate found).
Maddoxkkm
added a commit
to Maddoxkkm/Strata
that referenced
this pull request
Oct 10, 2026
…batch Niko1221#1416 is superseded by upstream 0.1.41 (which carries STRATA_PREFILL_CPU_SHARE_MAX and a newer two-argument arm_cpu_share the PR lacks); Niko1221#1101 and Niko1221#1237 are upstream already, and Niko1221#1237's older always-pin revision would re-arm the IQ3_S decode corruption the 0.1.41 gate found. Applied instead: Niko1221#1660 (native-layout MMQ resident sort), Niko1221#1520 (release staged pages, opt-in) and Niko1221#1107 (grouped gather on short prompts) - +4.7% prefill at 4K, +4.3% at 16K, with STRATA_FILE_RELEASE measured at -6.3% and left off. Adds pr-triage.py, pr-check.py, the report and the cached candidate diffs.
W1nge
marked this pull request as draft
October 11, 2026 02:05
Owner
|
Thanks. Windows-only memory behaviour; we have no Windows box to verify it, so it is not ported for now. |
Owner
|
Thanks for the pull request. We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected. To reopen it takes about two minutes:
The rules are in CONTRIBUTING.md. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On Windows, prefill copies expert weights from file mappings into the Stager's buffers, but the source pages can remain in the process working set after the copy. With a resident CPU expert pool, these duplicate pages can consume the remaining RAM during a long prompt.
Keep the source and expert identity on stable-pointer jobs and call the existing
ExpertSource::releaseafter the host copies complete, before publishing each job as ready. Stable pointers still usememcpy; transient blobs keep 0.1.41'scopy_blobsbatching. The mapping, resident RAM copies and CUDA-registered buffers remain intact. This uses the existing opt-inSTRATA_FILE_RELEASE=1; it adds no setting or numerical change.Validation:
fb58e0d); three files, +22/-13. The Stager definition matches the model-tested integrated build byte for byte.file_expert_source_testandgr_parity --selftestpass.Full model validation was on this Windows helper-GPU configuration and native IQ3_XXS pack. Other backends, GGUF-in-place staging and concurrent layer-split batches were not exercised by those model runs.