Skip to content

Start: name the GPU and stop at once when the build has no code for it; STRATA_EMULATE_CC - #289

Closed
sergqwer wants to merge 1 commit into
Niko1221:mainfrom
sergqwer:pr/gpu-check-emulate
Closed

sergqwer wants to merge 1 commit into
Niko1221:mainfrom
sergqwer:pr/gpu-check-emulate

Conversation

@sergqwer

Copy link
Copy Markdown

Summary

  • A clear stop for a wrong build. Before the expert arena loads, the engine names the card and its compute
    capability, and checks that the build has code for it (cudaFuncGetAttributes on a kernel). A binary built for
    other GPUs used to fail at its first kernel, after the whole arena had loaded. Now it stops in ~0.3 s, names the
    card and its sm_, and says to rebuild (setup does).
  • STRATA_EMULATE_CC=75|80|86|89, for tests. With a PTX build for an older generation (e.g.
    CMAKE_CUDA_ARCHITECTURES=86-virtual, JIT-compiled on a newer card), the host's device queries answer as that
    card: compute capability for ggml's MMQ configs and the QSA kernel choice, shared memory per block (64 KB on
    Turing). The kernels are then the real card's, so their results are too, though their speed is not. This is how
    the fork tested RTX 20/30/40 code paths on an RTX 5090.
  • STRATA_QSA_WARP=1|select|attn: the pre-sm_80 QSA kernels on any card, for A/B.

Measured

On the fork, an sm_86-only build on the RTX 5090 stops in 0.3 s with the card's name. Emulated sm_75/86/89 builds
give first-token KL to the native sm_120a build of 0.0005 (sm_89), 0.0003 (sm_86) and 0.0034 (sm_75), all with the
same top-1. On main (IQ2_XS) the check prints the card and changes nothing else.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio

…t; STRATA_EMULATE_CC for tests

- generate.cpp: before the expert arena loads, the engine names the card and its compute capability and checks that
  the build has code for it (device_code_error: cudaFuncGetAttributes on a kernel). A binary built for other GPUs
  used to fail at the first kernel, after the whole arena had loaded; now it stops in ~0.3 s and says why.
- include/strata/core/emulate.hpp: STRATA_EMULATE_CC=75|80|86|89 makes the host's device queries (compute
  capability, shared memory per block) answer as that generation, for a PTX build of it JIT-compiled on a newer
  card - ggml's MMQ config, the QSA kernels' choice, fused_gr's shared memory. Tests only.
- STRATA_QSA_WARP=1|select|attn: the pre-sm_80 QSA kernels on any card (A/B).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio
@Niko1221

Niko1221 commented Oct 1, 2026

Copy link
Copy Markdown
Owner

Thanks! This is in the 0.1.32 batch. We take it with a small fix (a placeholder name when the device query fails); every change goes through our identity gate (byte-identical output on four models) and a speed A/B before it ships.

@sergqwer

sergqwer commented Oct 1, 2026

Copy link
Copy Markdown
Author

Two things from rebuilding our fork on 0.1.31 with this PR applied.

  1. 0.1.31's new sm_75 tile in fused_gr_read_multi reads the real compute capability. cudaDeviceGetAttribute there decides small_tile, so under STRATA_EMULATE_CC=75 the host picks the 2560-float tile while the device code is Turing's. That breaks the rule the emulation rests on: the host must answer as the emulated card. In our release:

    const bool small_tile = strata::cc_major_of(cc_maj) * 10 + strata::cc_minor_of(cc_min) == 75;
  2. device_info() can answer as the emulated card too. Our fork does it:

    d.cc_major = cc_major_of(p.major);
    d.cc_minor = cc_minor_of(p.minor);

    It also prints one EMULATING compute capability line, so anything that sizes from DeviceInfo agrees with the kernels. It is not in this PR; add it if you take the emulation.

The card check stays before the expert arena starts loading. On 0.1.31 that is right before the expert sources are opened.

On 0.1.31: sergqwer/strata-nvfp4@6b5954a

🤖 Generated with Claude Code

Niko1221 added a commit that referenced this pull request Oct 1, 2026
…s; HIP builds compile (no device_code_error probe)

- generate.cpp: the startup line names "(an unnamed GPU)" when cudaGetDevice/cudaGetDeviceProperties fail (the
  error is cleared), and the "no code for" message uses the same name; a HIP build prints the gfx architecture
  instead of a CUDA compute capability.
- device.cu: device_code_error() returns "" on HIP - cudaFuncAttributes/cudaFuncGetAttributes have no mapping in
  hip_compat, and gpu_arch_problem() already stops a HIP run on an architecture the build lacks.
- fused_gr.cu: STRATA_EMULATE_CC also answers the two compute-capability queries there (small tile, the sm_70
  capacity rule), as the PR's description promises; with the variable unset nothing changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
@Niko1221

Niko1221 commented Oct 1, 2026

Copy link
Copy Markdown
Owner

Released in 0.1.32 with a placeholder name when the device query fails. Merged under your name (GitHub doesn't show it as merged because it went in by hand). Thank you!

@Niko1221 Niko1221 closed this Oct 1, 2026
Niko1221 added a commit that referenced this pull request Oct 6, 2026
…s; HIP builds compile (no device_code_error probe)

- generate.cpp: the startup line names "(an unnamed GPU)" when cudaGetDevice/cudaGetDeviceProperties fail (the
  error is cleared), and the "no code for" message uses the same name; a HIP build prints the gfx architecture
  instead of a CUDA compute capability.
- device.cu: device_code_error() returns "" on HIP - cudaFuncAttributes/cudaFuncGetAttributes have no mapping in
  hip_compat, and gpu_arch_problem() already stops a HIP run on an architecture the build lacks.
- fused_gr.cu: STRATA_EMULATE_CC also answers the two compute-capability queries there (small tile, the sm_70
  capacity rule), as the PR's description promises; with the variable unset nothing changes.
anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026
…ery fails; HIP builds compile (no device_code_error probe)

- generate.cpp: the startup line names "(an unnamed GPU)" when cudaGetDevice/cudaGetDeviceProperties fail (the
  error is cleared), and the "no code for" message uses the same name; a HIP build prints the gfx architecture
  instead of a CUDA compute capability.
- device.cu: device_code_error() returns "" on HIP - cudaFuncAttributes/cudaFuncGetAttributes have no mapping in
  hip_compat, and gpu_arch_problem() already stops a HIP run on an architecture the build lacks.
- fused_gr.cu: STRATA_EMULATE_CC also answers the two compute-capability queries there (small tile, the sm_70
  capacity rule), as the PR's description promises; with the variable unset nothing changes.
anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026
…ery fails; HIP builds compile (no device_code_error probe)

- generate.cpp: the startup line names "(an unnamed GPU)" when cudaGetDevice/cudaGetDeviceProperties fail (the
  error is cleared), and the "no code for" message uses the same name; a HIP build prints the gfx architecture
  instead of a CUDA compute capability.
- device.cu: device_code_error() returns "" on HIP - cudaFuncAttributes/cudaFuncGetAttributes have no mapping in
  hip_compat, and gpu_arch_problem() already stops a HIP run on an architecture the build lacks.
- fused_gr.cu: STRATA_EMULATE_CC also answers the two compute-capability queries there (small tile, the sm_70
  capacity rule), as the PR's description promises; with the variable unset nothing changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
k0valik added a commit to k0valik/Strata that referenced this pull request Oct 15, 2026
Squashed from 920 commit(s) fb58e0d..1f029aa on https://github.com/eelgaev/Strata-AC922, restricted to the
paths this fork still mirrors: 21 file(s) under src/kernels -- 14 clean,
7 with conflict markers.

Conflict-marked files -- a pull request with conflicts: resolve them,
do not discard the work:
  src/kernels/cpu/native_expert.cpp
  src/kernels/cpu/pool.cpp
  src/kernels/cuda/dequant_bf16.cu
  src/kernels/cuda/fused_gr.cu
  src/kernels/cuda/iq_kernels.cu
  src/kernels/cuda/verify_kernels.cu
  src/kernels/ngram.cpp

Original authors: 4EverBuilder,Andy,BlueKingMuch,Chanyeong Lim

Original commits touching these paths (151 subject(s)):
  - Niko1221#152 as an opt-in (the owner's choice): STRATA_IQ_MT_MIN=1 makes CPU expert rounding independent of the drafting; the default stays 0.1.29's rule (IQ3_S decoded 1-3% slower with it on); docs; the width test runs the opt-in
  - Niko1221#289 follow-up: a placeholder GPU name when the properties query fails; HIP builds compile (no device_code_error probe)
  - Niko1221#315 follow-up: its variants and switch renamed (plain / split / staged, STRATA_HC_SPLIT) - no clash with STRATA_GR_V3
  - Niko1221#646 on 0.1.39b: the layer split and the CPU path as before, the resident plan for 8-token windows
  - Niko1221#646: the single-matrix mmvq does not stage the IQ grids (10-20% slower on the RTX 5070 at 3-8 columns)
  - AMD HIP backend for gfx1100 (PR Niko1221#121, squashed onto its 0.1.21 base)
  - AMD HIP: RDNA2 gfx1030 (tested), gfx(1031, 1032) untested.
  - C-1, C-2: the prompt path's QSA select grid reaches only the active blocks; the indexer appends a chunk at once

Review artifact produced by fork-replay.sh (a spike adapted from
sync-upstream.sh): a filtered, squashed replay for reading -- not built,
not tested, and not evidence that any of it works.

Fork-Replay-Source: https://github.com/eelgaev/Strata-AC922
Fork-Replay-Branch: ac922
Fork-Replay-Base: fb58e0d
Fork-Replay-Tip: 1f029aa
Fork-Replay-Group: src/kernels
Fork-Replay-Automated: true
k0valik added a commit to k0valik/Strata that referenced this pull request Oct 15, 2026
Squashed from 920 commit(s) fb58e0d..1f029aa on https://github.com/eelgaev/Strata-AC922, restricted to the
paths this fork still mirrors: 1 file(s) under src/program -- 0 clean,
1 with conflict markers.

Conflict-marked files -- a pull request with conflicts: resolve them,
do not discard the work:
  src/program/generate.cpp

Original authors: Andy,Anon,BlueKingMuch,Constantin

Original commits touching these paths (241 subject(s)):
  - Niko1221#253: the 8 GiB arena pin cap only under WDDM; STRATA_ARENA_PIN_GIB; the split's prompt chunk steps down
  - Niko1221#272 follow-up: --pool-affinity all (the pool's layout so far) is the default; auto and p-cores are opt-in
  - Niko1221#282 follow-up: chunks above 8192 are opt-in (--prefill auto:16384 / auto:32768, or STRATA_PREFILL_AUTO_MAX)
  - Niko1221#284 follow-up: wait for the async commit on an event, at every point that touches the session afterwards
  - Niko1221#289 follow-up: a placeholder GPU name when the properties query fails; HIP builds compile (no device_code_error probe)
  - Niko1221#293 follow-up: the drafter stores rotated INT8 K/V as INT8; the rotation is part of the snapshot format
  - Niko1221#340 prototype: STRATA_SPLIT_SMALL_MAX=M - a split request of at most M prompt tokens reads in the stages' own S-token chunks (nothing lent)
  - Niko1221#340 prototype: a layer split refills every stage's prompt loan concurrently (queue all, then wait); STRATA_REFILL_SERIAL=1: stage after stage; STRATA_TRACE times the loan and the refill

Review artifact produced by fork-replay.sh (a spike adapted from
sync-upstream.sh): a filtered, squashed replay for reading -- not built,
not tested, and not evidence that any of it works.

Fork-Replay-Source: https://github.com/eelgaev/Strata-AC922
Fork-Replay-Branch: ac922
Fork-Replay-Base: fb58e0d
Fork-Replay-Tip: 1f029aa
Fork-Replay-Group: src/program
Fork-Replay-Automated: true
k0valik added a commit to k0valik/Strata that referenced this pull request Oct 15, 2026
Squashed from 920 commit(s) fb58e0d..1f029aa on https://github.com/eelgaev/Strata-AC922, restricted to the
paths this fork still mirrors: 21 file(s) under src/kernels -- 14 clean,
7 with conflict markers.

Conflict-marked files -- a pull request with conflicts: resolve them,
do not discard the work:
  src/kernels/cpu/native_expert.cpp
  src/kernels/cpu/pool.cpp
  src/kernels/cuda/dequant_bf16.cu
  src/kernels/cuda/fused_gr.cu
  src/kernels/cuda/iq_kernels.cu
  src/kernels/cuda/verify_kernels.cu
  src/kernels/ngram.cpp

Original authors: 4EverBuilder,Andy,BlueKingMuch,Chanyeong Lim

Original commits touching these paths (151 subject(s)):
  - Niko1221#152 as an opt-in (the owner's choice): STRATA_IQ_MT_MIN=1 makes CPU expert rounding independent of the drafting; the default stays 0.1.29's rule (IQ3_S decoded 1-3% slower with it on); docs; the width test runs the opt-in
  - Niko1221#289 follow-up: a placeholder GPU name when the properties query fails; HIP builds compile (no device_code_error probe)
  - Niko1221#315 follow-up: its variants and switch renamed (plain / split / staged, STRATA_HC_SPLIT) - no clash with STRATA_GR_V3
  - Niko1221#646 on 0.1.39b: the layer split and the CPU path as before, the resident plan for 8-token windows
  - Niko1221#646: the single-matrix mmvq does not stage the IQ grids (10-20% slower on the RTX 5070 at 3-8 columns)
  - AMD HIP backend for gfx1100 (PR Niko1221#121, squashed onto its 0.1.21 base)
  - AMD HIP: RDNA2 gfx1030 (tested), gfx(1031, 1032) untested.
  - C-1, C-2: the prompt path's QSA select grid reaches only the active blocks; the indexer appends a chunk at once

Review artifact produced by fork-replay.sh (a spike adapted from
sync-upstream.sh): a filtered, squashed replay for reading -- not built,
not tested, and not evidence that any of it works.

Fork-Replay-Source: https://github.com/eelgaev/Strata-AC922
Fork-Replay-Branch: ac922
Fork-Replay-Base: fb58e0d
Fork-Replay-Tip: 1f029aa
Fork-Replay-Group: src/kernels
Fork-Replay-Automated: true
k0valik added a commit to k0valik/Strata that referenced this pull request Oct 15, 2026
Squashed from 920 commit(s) fb58e0d..1f029aa on https://github.com/eelgaev/Strata-AC922, restricted to the
paths this fork still mirrors: 1 file(s) under src/program -- 0 clean,
1 with conflict markers.

Conflict-marked files -- a pull request with conflicts: resolve them,
do not discard the work:
  src/program/generate.cpp

Original authors: Andy,Anon,BlueKingMuch,Constantin

Original commits touching these paths (241 subject(s)):
  - Niko1221#253: the 8 GiB arena pin cap only under WDDM; STRATA_ARENA_PIN_GIB; the split's prompt chunk steps down
  - Niko1221#272 follow-up: --pool-affinity all (the pool's layout so far) is the default; auto and p-cores are opt-in
  - Niko1221#282 follow-up: chunks above 8192 are opt-in (--prefill auto:16384 / auto:32768, or STRATA_PREFILL_AUTO_MAX)
  - Niko1221#284 follow-up: wait for the async commit on an event, at every point that touches the session afterwards
  - Niko1221#289 follow-up: a placeholder GPU name when the properties query fails; HIP builds compile (no device_code_error probe)
  - Niko1221#293 follow-up: the drafter stores rotated INT8 K/V as INT8; the rotation is part of the snapshot format
  - Niko1221#340 prototype: STRATA_SPLIT_SMALL_MAX=M - a split request of at most M prompt tokens reads in the stages' own S-token chunks (nothing lent)
  - Niko1221#340 prototype: a layer split refills every stage's prompt loan concurrently (queue all, then wait); STRATA_REFILL_SERIAL=1: stage after stage; STRATA_TRACE times the loan and the refill

Review artifact produced by fork-replay.sh (a spike adapted from
sync-upstream.sh): a filtered, squashed replay for reading -- not built,
not tested, and not evidence that any of it works.

Fork-Replay-Source: https://github.com/eelgaev/Strata-AC922
Fork-Replay-Branch: ac922
Fork-Replay-Base: fb58e0d
Fork-Replay-Tip: 1f029aa
Fork-Replay-Group: src/program
Fork-Replay-Automated: true
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants