Repository navigation
bench: P4 prefill 78.8 -> 114.8 tok/s with existing auto sizing; expert reuse roofline - #1695
Closed
CC-David-CC wants to merge 1 commit into
Closed
CC-David-CC wants to merge 1 commit into
CC-David-CC wants to merge 1 commit into
Conversation
Niko1221
pushed a commit
that referenced
this pull request
Oct 10, 2026
…rt reuse roofline (PR #1695)
Owner
|
Thank you, in v0.1.42 with your commit (results only, as submitted), with a row in bench/results/COMMUNITY.md. |
giridhart
pushed a commit
to giridhart/Strata
that referenced
this pull request
Oct 15, 2026
…rt reuse roofline (PR Niko1221#1695)
giridhart
pushed a commit
to giridhart/Strata
that referenced
this pull request
Oct 15, 2026
…21#1638 Niko1221#1647 Niko1221#1655 Niko1221#1681 Niko1221#1692 Niko1221#1695 Niko1221#1716 Niko1221#1717 Niko1221#1657 Niko1221#1640; the 6900 XT 0.1.40.1 row now has the numbers Niko1221#1657 added
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Independent P4 prefill measurements inspired by @vizdatom's #1670. Results-only: this does not implement or run the proposed layer-major path.
One Tesla P4, Flash-Next-GSQ-RCO IQ3_XXS, int8 KV, CUDA 12.0, dual Xeon E5-2697 v3. Base fb58e0d. Warm page cache, zero prompt reuse, one generated token/request. Auto selected up to 1,792 tokens. Instrumented controls disable CPU expert sharing; matching fixed-512 controls were 77.928650/78.933397 tok/s.
8K fixed-512 copied 262.788941 GB of expert weights, 30.871322 GB distinct (88.25% repeated payload). Auto copied 193.275469 GB. Measured pinned H2D bandwidth: 11.886827 GB/s; VRAM read: 147.402334 GB/s. Copy waits were 6.2% of the fixed-512 timeline; fewer bytes are not an equivalent wall-time gain.
Includes exact prompts/output IDs, all 16 request records, configs, logs, replay driver, bandwidth sources, analysis and chart. The opt-in 21-line counter patch is a reproduction artifact only; engine source is unchanged. CUDA tested; no HIP/SYCL build or multi-token quality gate. Model revision/full asset hashes were not recorded.
Full report: bench/results/2026-10-09-community-p4-prefill-roofline/README.md