Integrate DiffusionGemma via llama.cpp when block-diffusion PRs land #1451

Overview

DiffusionGemma (released June 10, 2026) is Google DeepMind's first open-weight discrete diffusion language model — a 26B MoE (~3.8B active params/step) that denoises 256-token blocks in parallel rather than generating tokens autoregressively. Google claims >1,000 tok/s on H100. Quality is currently below standard Gemma 4 but the architecture is fundamentally different and worth tracking.

llama.cpp support is not yet in mainline. Two open PRs add block-diffusion inference:

• DiffusionGemma ggml-org/llama.cpp#24423 — adds `llama-diffusion-cli` binary for diffusion inference
• Add diffusion-gemma block-diffusion support ggml-org/llama.cpp#24427 — adds block-diffusion architecture support (had merge conflicts as of 2026-06-18)

Neither Ollama nor LM Studio can load DiffusionGemma GGUF today as both build on mainline. The community workaround is building from the PR branch directly.

Why It Requires Tracking

Diffusion inference uses a fundamentally different sampling loop (mask → denoise, not autoregressive). The PRs introduce a separate binary because the inference path doesn't fit the existing `llama-cli` machinery. For our Rust path via `llama-cpp-2`, this is 3 steps away:

1. PRs merge into llama.cpp mainline
2. `llama-cpp-2` crate ships a release bundling the new llama.cpp version
3. We add Rust bindings and bump the crate in `packages/nlp-engine`

No code changes are needed on our side until step 1 completes.

Acceptance Criteria

• llama.cpp PRs #24423 and #24427 (or successors) merged into mainline
• `llama-cpp-2` releases a version bundling block-diffusion support
• Bump `llama-cpp-2` version constraint in `packages/nlp-engine/Cargo.toml`
• Add any required Rust bindings for `llama_diffusion_*` C API calls
• Evaluate DiffusionGemma GGUF against NodeSpace use cases (tool calls, summarization) — quality vs speed tradeoff
• Document findings and decide whether to enable as an optional model target

Technical Specifications

Reference Files

• Cargo constraint: `packages/nlp-engine/Cargo.toml` — `llama-cpp-2` version pin
• Eval harness: `scripts/aichat-matrix.ts` — re-run against DiffusionGemma GGUF when available
• Upstream PRs: DiffusionGemma ggml-org/llama.cpp#24423, Add diffusion-gemma block-diffusion support ggml-org/llama.cpp#24427

Watch For

• New `llama-cpp-2` release on crates.io that bundles post-PR-merge llama.cpp
• PRs may be superseded or rebased — search for "diffusion" in llama.cpp PRs if links go stale
• Model quality is currently below Gemma 4 autoregressive — re-evaluate at time of integration

Related Issues

• Upgrade llama-cpp-sys-2 when Gemma 4 12B peg-gemma4 parser fixes land upstream #1348 (Gemma 4 12B peg-gemma4 parser fix — different issue, same crate)
• A/B test Gemma 4 12B (Q4_K_M) vs E4B for the local ai-chat agent #1329 (12B A/B trial — hardware constraints context)