[Andgihat]

Adds support for Flint (Recursive, 39.8B-A2.5B MoE, FlintForCausalLM), branched from v0.4.9.

Why a port is needed

BeeLlama v0.4.9 is based on llama.cpp fd55cb98f (2026-10-15). Flint landed upstream on 2026-10-21, so no branch of this repo can load a Flint GGUF today.

The branch is exactly three upstream commits cherry-picked on top of v0.4.9, with authorship preserved:

commit	upstream	what it is
5036a22	ggml-org#30861	llm_graph_amax_src1 + POOL_1D split state — required by the Flint graph
1f00f16	ggml-org#30928	Flint architecture itself
de8a9da	ggml-org#30929	hex string escapes in the jinja lexer — required by Flint's chat template (\x00)

Without ggml-org#30929 the server refuses to start: chat template parsing error: lexer: unknown escape character \x.

Conflict resolution

Only two real conflicts:

• src/llama-graph.cpp — BeeLlama's MoE path passes its own slots argument to build_lora_mm_id. Kept it inside the upstream amax block, so the cache path is unchanged.
• tests/test-llama-archs.cpp — the upstream hunk also references MUSE_SPARK and KOLIBRI1, which do not exist in the v0.4.9 base. Added only LLM_ARCH_FLINT to the existing list.

Everything else applied cleanly, including llama-kv-cache.cpp (K-shift only for layers with RoPE). llama_hparams::has_rope() still defaults to true for models without a rope_pattern, so behaviour for existing architectures is unchanged.

Verified (Windows 11, CUDA 12.8, sm_120a, RTX 5060 Ti 16 GB)

Built with GGML_CUDA_KVARN=ON, GGML_CUDA_FA_ALL_QUANTS=ON, -j 6 (higher parallelism OOMs cicc on 32 GB RAM).

unsloth/flint-GGUF UD-Q3_K_XL (17.2 GiB, does not fit fully in 16 GB — experts partially on CPU):

ctx	KV	--n-cpu-moe	VRAM	prefill	decode
32K	f16	12	15.74 GB	1525 t/s	77 t/s
32K	q8_0	11	15.58 GB	1576 t/s	75-79 t/s
32K	kvarn3/kvarn2	12	15.00 GB	904 t/s	49-53 t/s
128K	f16	20	15.60 GB	1093 t/s	61-64 t/s
128K	q8_0	15	15.75 GB	1310 t/s	67-69 t/s
128K	kvarn3/kvarn2	13	15.07 GB	853 t/s	48-51 t/s
32K	f16	11	15.95 GB	920 t/s	9 t/s (VRAM spill)
128K	q8_0	14	15.96 GB	1143 t/s	30 t/s (VRAM spill)

KVarN observation: Flint loads and runs with kvarn3/kvarn2 (the cache does shrink — 0.6-0.7 GiB less than q8_0), but decode is consistently ~30% slower than q8_0 even after spending the freed VRAM on two more expert layers. Likely the attention sinks force the materialization route instead of direct-record decode, but that is not verified. q8_0 is the better lever on this model.

Also verified:

• OpenAI-compatible tool calling returns finish_reason=tool_calls with correct arguments.
• An existing model (LFM2.5-8B-A1B, lfm2moe) still loads and decodes at 260 t/s on the same build.
• --moe-cache-mib made decode worse on this model (44-47 t/s at 128K/ncmoe 24 vs 54-58 without), so it is not used.

Notes

• PR is based on v0.4.9 rather than main: main is 240 commits behind that branch, so a PR against main would carry the whole v0.4.9 upstream merge.
• Flint has no MTP head and no draft model, so speculative decoding does not apply.

[Anbeeld]

Merged newer upstream master into v0.4.9, so this is superseded.