Repository navigation
Conversation
… (with or without an expert source) 0.1.39 took this PR's first part with Niko1221#583 (the xn / grs count, STRATA_RING_BYTES). What is left: bytes_needed counted the fused layout's smaller MoE buffers for every caller (fused_layout(T, true)), while init takes them only when it gets an ExpertSource (fused_layout(T, m.src != nullptr)). A prompt path without one was lent fewer bytes than its unfused buffers take. bytes_needed, bytes_needed_no_ring and (0.1.40) bytes_needed_owned - STRATA_OWNED_PRICE=exact's sizing - now take the same `src` the callers pass to init. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio
Author
|
Rebased onto e8ca9af (v0.1.40.2). The only conflict was the new Default identity against v0.1.40.2 on IQ2_XS (
|
sergqwer
force-pushed
the
pr/prefill-bytes-needed
branch
from
October 7, 2026 14:43
ac45e87 to
e7635b6
Compare
fragtion
pushed a commit
to fragtion/StrataPool
that referenced
this pull request
Oct 7, 2026
…uffers for the layout init picks (with or without an expert source) prefill: bytes_needed sizes the MoE buffers for the layout init picks (with or without an expert source) 0.1.39 took this PR's first part with Niko1221#583 (the xn / grs count, STRATA_RING_BYTES). What is left: bytes_needed counted the fused layout's smaller MoE buffers for every caller (fused_layout(T, true)), while init takes them only when it gets an ExpertSource (fused_layout(T, m.src != nullptr)). A prompt path without one was lent fewer bytes than its unfused buffers take. bytes_needed, bytes_needed_no_ring and (0.1.40) bytes_needed_owned - STRATA_OWNED_PRICE=exact's sizing - now take the same `src` the callers pass to init. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0157fjixQD2nRxvCvtXq66DD
Author
|
Thank you for taking this in 0.1.42 (a26fcbc, the same change). Closing. |
fragtion
added a commit
to fragtion/StrataPool
that referenced
this pull request
Oct 12, 2026
…-support Upstream now carries these of our picks, in its own versions: Niko1221#1237 (pinned stage buffers, now with fences and the STRATA_STAGE_PIN switch), Niko1221#1332, Niko1221#1359, Niko1221#1372, Niko1221#1370, Niko1221#1251, Niko1221#1190 and Niko1221#1283. CPU help for short prompts: Strata's own CPU prefill share (0.1.41: on by default below 1,024 tokens on one NVIDIA GPU; -20..-34% prompt time at 512-1,000 tokens on an RTX 3060 in upstream's measurement) replaces architectds' CPU assist, which measured +11% at ~560 tokens on our desktop. prefill.cpp is upstream's, with the pool's hooks (remote_next to the workers, a headless worker stage) and Niko1221#1368's quantize-once put back. A pool PC runs its own layers alone (no local stages), so the share's default applies on every PC. Pipelined decode: upstream's N-stage loop (front stages + the last) with the pool's remote stage as the last (pl_remote via stage_ver_p, pl_ns = 2 on a pool coordinator), and architectds' verified-path-first yield, the lookup carried across windows and the pool head's penalty rows ported onto it. Batch: upstream's batch_step (the adaptive tier in batch windows, per-stage timing, Niko1221#879 non-finite logits) with Niko1221#1290's per-slot row width and the pool head's slot histories; the pool's batch groups keep their PoolLink stage. Also: the conversation cache's eviction keeps both the pool-borrowed entry and pinned prefixes (evict_oldest); the verifier's adaptive-tier gate (STRATA_ADAPT_OVERLAP) also guards the overlapped split's prelaunch (Niko1221#1120); the server's read-once body takes upstream's size limit; replicas and the pool roles both start engines; serve/pool.py keeps the engine's newer spawn tuple; the worker's parking reads available_host_bytes(); setup tests that need a ready-made engine are skipped (StrataPool compiles its own). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0157fjixQD2nRxvCvtXq66DD
DarthCY-K
pushed a commit
to DarthCY-K/Strata
that referenced
this pull request
Oct 20, 2026
…ured PCIe share, Niko1221#1690 Niko1221#1693 Niko1221#1696 Niko1221#1283 as upstream's code) - README, docs/NVFP4.md and the bundle version Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #1042, which GitHub closed when
main's history was rewritten. The closing note says this was not a rejection. It is the same commit, cherry-picked onto the newmain(v0.1.40.1).Prefill::bytes_needed()sizes the region the prompt path borrows from the expert cache. It must count exactly whatcarve()allocates. 0.1.39 took the first part of #547 with #583 (thexn/grscount). This is what is left.The MoE layout.
bytes_neededsizes the MoE scratch withfused_layout(T, true)for every caller.carvetakes the fused layout only wheninitgets anExpertSource(fused_layout(T, m.src != nullptr)). So a prompt path without one is lent the fused layout's smaller MoE buffers, and its unfused buffers do not fit in them.The change.
bytes_needed_impltakes asrcflag. So dobytes_needed,bytes_needed_no_ringand 0.1.40'sbytes_needed_owned(#796), all defaulting totrue. Every caller ingenerate.cpppassessrcp != nullptr, the same source it handsinit. That includesSTRATA_OWNED_PRICE=exact's sizing.Identity. With an
ExpertSource(every normal run) nothing changes. The logits and the 32 greedy tokens after a 32K prompt equal v0.1.40 on IQ2_XS: identical bytes, same expert cache. The engine's sources here are the same bytes as on that branch, because 0.1.40.1 changed only the server and setup.🤖 Generated with Claude Code
https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio