Skip to content

Community benchmark: mixed dual V100 + RTX PRO 4000 Blackwell on Windows - #2142

Open
intproduct wants to merge 1 commit into
Niko1221:mainfrom
intproduct:community-v100-pro4000-windows
Open

intproduct wants to merge 1 commit into
Niko1221:mainfrom
intproduct:community-v100-pro4000-windows

Conversation

@intproduct

Copy link
Copy Markdown

This results-only report measures IQ2_XS with 262144 context, INT8 KV, MTP and Vision on Windows 10, EPYC 7A23 and 64 GB RAM, using the existing CUDA 12.9 engine 0.1.40.

It compares the dual-V100 baseline, both three-stage GPU orders, and a dual-V100 language engine with Vision on the RTX PRO 4000 Blackwell. Each configuration has three fresh-prompt runs at 4096/32768/128000/258000 input tokens, 256 output tokens and zero prefix reuse (48 trials). With PRO first, 258000-token median TTFT changed from 158.86 to 87.24 seconds and decode from 82.2 to 95.9 tok/s. Nine long-context recall checks and API/image/tool checks passed; all four coding scores, including failed edge cases, are disclosed.

The report includes sanitized configs, measured allocations, all per-run data/outputs, timing excerpts and small reproduction scripts. Limitations include different reserve settings, auto-split changes, fixed test order, background desktop activity, thermal/power limits, an older installed binary, and ambiguous coding-task boundary wording. This does not claim universal CUDA 12/Blackwell compatibility or broad Agent quality. No engine changes, binaries, model files or private transcripts are included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant