Repository navigation
Conversation
…t; STRATA_EMULATE_CC for tests - generate.cpp: before the expert arena loads, the engine names the card and its compute capability and checks that the build has code for it (device_code_error: cudaFuncGetAttributes on a kernel). A binary built for other GPUs used to fail at the first kernel, after the whole arena had loaded; now it stops in ~0.3 s and says why. - include/strata/core/emulate.hpp: STRATA_EMULATE_CC=75|80|86|89 makes the host's device queries (compute capability, shared memory per block) answer as that generation, for a PTX build of it JIT-compiled on a newer card - ggml's MMQ config, the QSA kernels' choice, fused_gr's shared memory. Tests only. - STRATA_QSA_WARP=1|select|attn: the pre-sm_80 QSA kernels on any card (A/B). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio
|
Thanks! This is in the 0.1.32 batch. We take it with a small fix (a placeholder name when the device query fails); every change goes through our identity gate (byte-identical output on four models) and a speed A/B before it ships. |
|
Two things from rebuilding our fork on 0.1.31 with this PR applied.
The card check stays before the expert arena starts loading. On 0.1.31 that is right before the expert sources are opened. On 0.1.31: sergqwer/strata-nvfp4@6b5954a 🤖 Generated with Claude Code |
…s; HIP builds compile (no device_code_error probe) - generate.cpp: the startup line names "(an unnamed GPU)" when cudaGetDevice/cudaGetDeviceProperties fail (the error is cleared), and the "no code for" message uses the same name; a HIP build prints the gfx architecture instead of a CUDA compute capability. - device.cu: device_code_error() returns "" on HIP - cudaFuncAttributes/cudaFuncGetAttributes have no mapping in hip_compat, and gpu_arch_problem() already stops a HIP run on an architecture the build lacks. - fused_gr.cu: STRATA_EMULATE_CC also answers the two compute-capability queries there (small tile, the sm_70 capacity rule), as the PR's description promises; with the variable unset nothing changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
|
Released in 0.1.32 with a placeholder name when the device query fails. Merged under your name (GitHub doesn't show it as merged because it went in by hand). Thank you! |
…s; HIP builds compile (no device_code_error probe) - generate.cpp: the startup line names "(an unnamed GPU)" when cudaGetDevice/cudaGetDeviceProperties fail (the error is cleared), and the "no code for" message uses the same name; a HIP build prints the gfx architecture instead of a CUDA compute capability. - device.cu: device_code_error() returns "" on HIP - cudaFuncAttributes/cudaFuncGetAttributes have no mapping in hip_compat, and gpu_arch_problem() already stops a HIP run on an architecture the build lacks. - fused_gr.cu: STRATA_EMULATE_CC also answers the two compute-capability queries there (small tile, the sm_70 capacity rule), as the PR's description promises; with the variable unset nothing changes.
…ery fails; HIP builds compile (no device_code_error probe) - generate.cpp: the startup line names "(an unnamed GPU)" when cudaGetDevice/cudaGetDeviceProperties fail (the error is cleared), and the "no code for" message uses the same name; a HIP build prints the gfx architecture instead of a CUDA compute capability. - device.cu: device_code_error() returns "" on HIP - cudaFuncAttributes/cudaFuncGetAttributes have no mapping in hip_compat, and gpu_arch_problem() already stops a HIP run on an architecture the build lacks. - fused_gr.cu: STRATA_EMULATE_CC also answers the two compute-capability queries there (small tile, the sm_70 capacity rule), as the PR's description promises; with the variable unset nothing changes.
…ery fails; HIP builds compile (no device_code_error probe) - generate.cpp: the startup line names "(an unnamed GPU)" when cudaGetDevice/cudaGetDeviceProperties fail (the error is cleared), and the "no code for" message uses the same name; a HIP build prints the gfx architecture instead of a CUDA compute capability. - device.cu: device_code_error() returns "" on HIP - cudaFuncAttributes/cudaFuncGetAttributes have no mapping in hip_compat, and gpu_arch_problem() already stops a HIP run on an architecture the build lacks. - fused_gr.cu: STRATA_EMULATE_CC also answers the two compute-capability queries there (small tile, the sm_70 capacity rule), as the PR's description promises; with the variable unset nothing changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
Squashed from 920 commit(s) fb58e0d..1f029aa on https://github.com/eelgaev/Strata-AC922, restricted to the paths this fork still mirrors: 21 file(s) under src/kernels -- 14 clean, 7 with conflict markers. Conflict-marked files -- a pull request with conflicts: resolve them, do not discard the work: src/kernels/cpu/native_expert.cpp src/kernels/cpu/pool.cpp src/kernels/cuda/dequant_bf16.cu src/kernels/cuda/fused_gr.cu src/kernels/cuda/iq_kernels.cu src/kernels/cuda/verify_kernels.cu src/kernels/ngram.cpp Original authors: 4EverBuilder,Andy,BlueKingMuch,Chanyeong Lim Original commits touching these paths (151 subject(s)): - Niko1221#152 as an opt-in (the owner's choice): STRATA_IQ_MT_MIN=1 makes CPU expert rounding independent of the drafting; the default stays 0.1.29's rule (IQ3_S decoded 1-3% slower with it on); docs; the width test runs the opt-in - Niko1221#289 follow-up: a placeholder GPU name when the properties query fails; HIP builds compile (no device_code_error probe) - Niko1221#315 follow-up: its variants and switch renamed (plain / split / staged, STRATA_HC_SPLIT) - no clash with STRATA_GR_V3 - Niko1221#646 on 0.1.39b: the layer split and the CPU path as before, the resident plan for 8-token windows - Niko1221#646: the single-matrix mmvq does not stage the IQ grids (10-20% slower on the RTX 5070 at 3-8 columns) - AMD HIP backend for gfx1100 (PR Niko1221#121, squashed onto its 0.1.21 base) - AMD HIP: RDNA2 gfx1030 (tested), gfx(1031, 1032) untested. - C-1, C-2: the prompt path's QSA select grid reaches only the active blocks; the indexer appends a chunk at once Review artifact produced by fork-replay.sh (a spike adapted from sync-upstream.sh): a filtered, squashed replay for reading -- not built, not tested, and not evidence that any of it works. Fork-Replay-Source: https://github.com/eelgaev/Strata-AC922 Fork-Replay-Branch: ac922 Fork-Replay-Base: fb58e0d Fork-Replay-Tip: 1f029aa Fork-Replay-Group: src/kernels Fork-Replay-Automated: true
Squashed from 920 commit(s) fb58e0d..1f029aa on https://github.com/eelgaev/Strata-AC922, restricted to the paths this fork still mirrors: 1 file(s) under src/program -- 0 clean, 1 with conflict markers. Conflict-marked files -- a pull request with conflicts: resolve them, do not discard the work: src/program/generate.cpp Original authors: Andy,Anon,BlueKingMuch,Constantin Original commits touching these paths (241 subject(s)): - Niko1221#253: the 8 GiB arena pin cap only under WDDM; STRATA_ARENA_PIN_GIB; the split's prompt chunk steps down - Niko1221#272 follow-up: --pool-affinity all (the pool's layout so far) is the default; auto and p-cores are opt-in - Niko1221#282 follow-up: chunks above 8192 are opt-in (--prefill auto:16384 / auto:32768, or STRATA_PREFILL_AUTO_MAX) - Niko1221#284 follow-up: wait for the async commit on an event, at every point that touches the session afterwards - Niko1221#289 follow-up: a placeholder GPU name when the properties query fails; HIP builds compile (no device_code_error probe) - Niko1221#293 follow-up: the drafter stores rotated INT8 K/V as INT8; the rotation is part of the snapshot format - Niko1221#340 prototype: STRATA_SPLIT_SMALL_MAX=M - a split request of at most M prompt tokens reads in the stages' own S-token chunks (nothing lent) - Niko1221#340 prototype: a layer split refills every stage's prompt loan concurrently (queue all, then wait); STRATA_REFILL_SERIAL=1: stage after stage; STRATA_TRACE times the loan and the refill Review artifact produced by fork-replay.sh (a spike adapted from sync-upstream.sh): a filtered, squashed replay for reading -- not built, not tested, and not evidence that any of it works. Fork-Replay-Source: https://github.com/eelgaev/Strata-AC922 Fork-Replay-Branch: ac922 Fork-Replay-Base: fb58e0d Fork-Replay-Tip: 1f029aa Fork-Replay-Group: src/program Fork-Replay-Automated: true
Squashed from 920 commit(s) fb58e0d..1f029aa on https://github.com/eelgaev/Strata-AC922, restricted to the paths this fork still mirrors: 21 file(s) under src/kernels -- 14 clean, 7 with conflict markers. Conflict-marked files -- a pull request with conflicts: resolve them, do not discard the work: src/kernels/cpu/native_expert.cpp src/kernels/cpu/pool.cpp src/kernels/cuda/dequant_bf16.cu src/kernels/cuda/fused_gr.cu src/kernels/cuda/iq_kernels.cu src/kernels/cuda/verify_kernels.cu src/kernels/ngram.cpp Original authors: 4EverBuilder,Andy,BlueKingMuch,Chanyeong Lim Original commits touching these paths (151 subject(s)): - Niko1221#152 as an opt-in (the owner's choice): STRATA_IQ_MT_MIN=1 makes CPU expert rounding independent of the drafting; the default stays 0.1.29's rule (IQ3_S decoded 1-3% slower with it on); docs; the width test runs the opt-in - Niko1221#289 follow-up: a placeholder GPU name when the properties query fails; HIP builds compile (no device_code_error probe) - Niko1221#315 follow-up: its variants and switch renamed (plain / split / staged, STRATA_HC_SPLIT) - no clash with STRATA_GR_V3 - Niko1221#646 on 0.1.39b: the layer split and the CPU path as before, the resident plan for 8-token windows - Niko1221#646: the single-matrix mmvq does not stage the IQ grids (10-20% slower on the RTX 5070 at 3-8 columns) - AMD HIP backend for gfx1100 (PR Niko1221#121, squashed onto its 0.1.21 base) - AMD HIP: RDNA2 gfx1030 (tested), gfx(1031, 1032) untested. - C-1, C-2: the prompt path's QSA select grid reaches only the active blocks; the indexer appends a chunk at once Review artifact produced by fork-replay.sh (a spike adapted from sync-upstream.sh): a filtered, squashed replay for reading -- not built, not tested, and not evidence that any of it works. Fork-Replay-Source: https://github.com/eelgaev/Strata-AC922 Fork-Replay-Branch: ac922 Fork-Replay-Base: fb58e0d Fork-Replay-Tip: 1f029aa Fork-Replay-Group: src/kernels Fork-Replay-Automated: true
Squashed from 920 commit(s) fb58e0d..1f029aa on https://github.com/eelgaev/Strata-AC922, restricted to the paths this fork still mirrors: 1 file(s) under src/program -- 0 clean, 1 with conflict markers. Conflict-marked files -- a pull request with conflicts: resolve them, do not discard the work: src/program/generate.cpp Original authors: Andy,Anon,BlueKingMuch,Constantin Original commits touching these paths (241 subject(s)): - Niko1221#253: the 8 GiB arena pin cap only under WDDM; STRATA_ARENA_PIN_GIB; the split's prompt chunk steps down - Niko1221#272 follow-up: --pool-affinity all (the pool's layout so far) is the default; auto and p-cores are opt-in - Niko1221#282 follow-up: chunks above 8192 are opt-in (--prefill auto:16384 / auto:32768, or STRATA_PREFILL_AUTO_MAX) - Niko1221#284 follow-up: wait for the async commit on an event, at every point that touches the session afterwards - Niko1221#289 follow-up: a placeholder GPU name when the properties query fails; HIP builds compile (no device_code_error probe) - Niko1221#293 follow-up: the drafter stores rotated INT8 K/V as INT8; the rotation is part of the snapshot format - Niko1221#340 prototype: STRATA_SPLIT_SMALL_MAX=M - a split request of at most M prompt tokens reads in the stages' own S-token chunks (nothing lent) - Niko1221#340 prototype: a layer split refills every stage's prompt loan concurrently (queue all, then wait); STRATA_REFILL_SERIAL=1: stage after stage; STRATA_TRACE times the loan and the refill Review artifact produced by fork-replay.sh (a spike adapted from sync-upstream.sh): a filtered, squashed replay for reading -- not built, not tested, and not evidence that any of it works. Fork-Replay-Source: https://github.com/eelgaev/Strata-AC922 Fork-Replay-Branch: ac922 Fork-Replay-Base: fb58e0d Fork-Replay-Tip: 1f029aa Fork-Replay-Group: src/program Fork-Replay-Automated: true
Summary
capability, and checks that the build has code for it (
cudaFuncGetAttributeson a kernel). A binary built forother GPUs used to fail at its first kernel, after the whole arena had loaded. Now it stops in ~0.3 s, names the
card and its
sm_, and says to rebuild (setup does).STRATA_EMULATE_CC=75|80|86|89, for tests. With a PTX build for an older generation (e.g.CMAKE_CUDA_ARCHITECTURES=86-virtual, JIT-compiled on a newer card), the host's device queries answer as thatcard: compute capability for ggml's MMQ configs and the QSA kernel choice, shared memory per block (64 KB on
Turing). The kernels are then the real card's, so their results are too, though their speed is not. This is how
the fork tested RTX 20/30/40 code paths on an RTX 5090.
STRATA_QSA_WARP=1|select|attn: the pre-sm_80 QSA kernels on any card, for A/B.Measured
On the fork, an sm_86-only build on the RTX 5090 stops in 0.3 s with the card's name. Emulated sm_75/86/89 builds
give first-token KL to the native sm_120a build of 0.0005 (sm_89), 0.0003 (sm_86) and 0.0034 (sm_75), all with the
same top-1. On main (IQ2_XS) the check prints the card and changes nothing else.
🤖 Generated with Claude Code
https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio