Repository navigation
Community benchmark: mixed dual V100 + RTX PRO 4000 Blackwell on Windows - #2142
Open
intproduct wants to merge 1 commit into
Open
intproduct wants to merge 1 commit into
intproduct wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This results-only report measures IQ2_XS with 262144 context, INT8 KV, MTP and Vision on Windows 10, EPYC 7A23 and 64 GB RAM, using the existing CUDA 12.9 engine 0.1.40.
It compares the dual-V100 baseline, both three-stage GPU orders, and a dual-V100 language engine with Vision on the RTX PRO 4000 Blackwell. Each configuration has three fresh-prompt runs at 4096/32768/128000/258000 input tokens, 256 output tokens and zero prefix reuse (48 trials). With PRO first, 258000-token median TTFT changed from 158.86 to 87.24 seconds and decode from 82.2 to 95.9 tok/s. Nine long-context recall checks and API/image/tool checks passed; all four coding scores, including failed edge cases, are disclosed.
The report includes sanitized configs, measured allocations, all per-run data/outputs, timing excerpts and small reproduction scripts. Limitations include different reserve settings, auto-split changes, fixed test order, background desktop activity, thermal/power limits, an older installed binary, and ambiguous coding-task boundary wording. This does not claim universal CUDA 12/Blackwell compatibility or broad Agent quality. No engine changes, binaries, model files or private transcripts are included.