Repository navigation
Conversation
This was referenced Oct 7, 2026
W1nge
marked this pull request as ready for review
October 7, 2026 23:22
This was referenced Oct 7, 2026
Maddoxkkm
added a commit
to Maddoxkkm/Strata
that referenced
this pull request
Oct 10, 2026
…tch-audit.py + fetched PR diffs) Line-level audit of the eight patches applied on tuned-1403 against the tuned-141 tree and against upstream 0.1.41: Niko1221#1323 and Niko1221#1372 are upstream's now, Niko1221#1381/Niko1221#1439/Niko1221#1451 are re-applied on 141, while Niko1221#1416 (CPU share), Niko1221#1448 (q8_1 finite-ds) and Niko1221#1457 (HC scratch reuse) have no counterpart there. patch-audit.py compares a diff's added code lines against a tree, which survives the rebases and reworkings that defeat 'git apply -R --check'.
Maddoxkkm
added a commit
to Maddoxkkm/Strata
that referenced
this pull request
Oct 10, 2026
…ocation Our 0.1.41 application landed Niko1221#1451's helper and its test but never called them: prefill.cpp still allocated both PLE host buffers eagerly in init(), sized T*N*4 regardless of what a run needs, so the helper was dead code and the pinned footprint was not conditional. This takes the PR's current shape: the allocation moves into run_impl, buffered lazily (buffer two only when a following chunk exists) with the retained capacity kept, and init() stops allocating.
…mization-pr # Conflicts: # src/prefill/prefill.cpp
W1nge
marked this pull request as draft
October 11, 2026 01:34
Owner
|
Thanks. It saves host memory and claims no speed, so it is outside this round and not ported. |
Owner
|
Thanks for the pull request. We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected. To reopen it takes about two minutes:
The rules are in CONTRIBUTING.md. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Prefill initialization reserves two maximum-chunk PLE host buffers even when batched PLE is unused. Allocate them when the batched path runs: one for a single chunk, two for multiple chunks, retaining capacity for reuse.
What changed
Wait for the previous upload before growing storage and preserve the pinned/pageable fallback. At 32,768 rows x 2,560 floats, each avoided buffer is 320 MiB. No model-specific constants, tuning flags or numerical changes.
Validation
Updated to main
61b3fb5d. The production helper passes 64 delayed-DMA growth/reuse and fallback cases on each of RTX 2080 Ti and P100. Full MSVC/CUDA integration build passes forsm60;sm75.Draft pending affected HIP/SYCL builds. Current-main full-model inference was not repeated; allocation cost moves to the first batched PLE request.