Skip to content

parity tool: a new-kernel mismatch must fail the run; dispatch comment fallback was stale - #64

Merged
Niko1221 merged 1 commit into
Niko1221:mainfrom
pipeob0:gh/avx2-followups
Sep 28, 2026
Merged

Niko1221 merged 1 commit into
Niko1221:mainfrom
pipeob0:gh/avx2-followups

Conversation

@pipeob0

@pipeob0 pipeob0 commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

The two follow-ups you listed in your #43 merge note. Both are yours to shape; nothing here changes a kernel.

1. The comment above the gate/up dispatch described a fallback that no longer exists. It said STRATA_NO_IQ512 / STRATA_NO_IQ256 fall back to ggml-cpu's single-token vec_dot. Since #43, STRATA_NO_IQ512 drops an AVX-512 CPU to the AVX-2 kernel, STRATA_NO_IQ256 drops the AVX-2 kernel, and ggml's vec_dot is reached only with both set (and on a CPU without AVX-512, STRATA_NO_IQ512 changes nothing at all). Comment-only commit.

2. The parity tool printed the new kernels' distance from ggml's vec_dot without affecting the exit code. A kernel that silently diverged still reported 0 failures, which is the one property of that tool people rely on. Both comparisons (the gate/up rows in check, and the IQ4_NL down) are now gated at 1e-5 and increment failures. Measured headroom: gate/up rows 3.1-3.4e-08, IQ4_NL down ~1e-7, so the gate sits ~100x above the observed spread.

Verified on this box (IQ2_XS, IQ2_XXS, IQ3_S, IQ4_NL layers):

  • threshold 1e-5 → native_expert_parity: 0 failures (unchanged, as it should be);
  • threshold temporarily dropped to 1e-9, i.e. below the observed spread, to prove the gate is live:
iq2_xs  avx2 gate rows MISMATCH (rel 3.12e-08 > 1e-9)
iq4_nl  down           MISMATCH (rel 9.91e-08 > 1e-9)
iq2_xxs avx2 gate rows MISMATCH (rel 3.35e-08 > 1e-9)
iq3_s   avx2 gate rows MISMATCH (rel 3.34e-08 > 1e-9)

Both sites fire, and the run exits non-zero. The threshold is back at 1e-5 in the commit.

Left alone deliberately: the q2_0 down comparison prints ~2.1e-04, which is the pre-existing Q2_0 kernel against ggml and is expected to be that coarse — gating it at 1e-5 would be a false alarm, so it stays print-only.

The gate/up row and IQ4_NL down comparisons printed their distance from ggml's vec_dot without affecting the exit code, so a kernel that silently diverged still reported 0 failures. Gate both at 1e-5 (measured 3e-8 and 1e-7, so 100x headroom). Verified: 0 failures at 1e-5, and with the threshold dropped to 1e-9 the gate fires on every layer.

Also corrects the comment above the gate/up dispatch: STRATA_NO_IQ512 drops an AVX-512 CPU to the AVX-2 kernel, not to ggml; ggml's vec_dot needs both switches set.
@Niko1221
Niko1221 merged commit bd237ff into Niko1221:main Sep 28, 2026
@Niko1221

Copy link
Copy Markdown
Owner

Thanks @pipeob0 - this closes both follow-ups from #43, and showing the gate fires at 1e-9 is exactly the proof I wanted. Merged, in v0.1.20.

@pipeob0
pipeob0 deleted the gh/avx2-followups branch October 1, 2026 20:43
MistVVK pushed a commit to MistVVK/XeStrata that referenced this pull request Oct 3, 2026
Hugua700 pushed a commit to Hugua700/Strata that referenced this pull request Oct 6, 2026
anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants