Repository navigation
feat(gallery): add flint and bump llama.cpp to a37cc461 - #12854
Merged
Merged
Conversation
Move the llama.cpp pin to a37cc461, the merge of the Flint architecture (ggml-org/llama.cpp#30928). The TTS task patch is gone: upstream now carries SERVER_TASK_TYPE_TTS, server_task::tts_inp, server_task_result_tts, server_context_meta::has_cap_tts and the mtmd gen-audio helpers. The patch is detected as already applied on the new pin. The gRPC server uses the same symbols. The bonsai and turboquant forks still lack them and keep using disable-tts-task.sh, so they are unchanged. The score task patch is still needed: upstream has no SERVER_TASK_TYPE_SCORE. It is regenerated against the new pin and applies with no offset or fuzz. Upstream added SERVER_TASK_TYPE_TRANSCRIBE_RT to server_task::need_logits(), which broke one hunk, so SCORE is added next to it by hand. The C++ backend was not built for this change. Assisted-by: Claude:claude-sonnet-5-5
Add Recursive Flint, a 39.8B mixture-of-experts coding model with about 2.5B active parameters, from the Unsloth GGUF repository. It runs on llama.cpp with the Flint architecture support added upstream in ggml-org/llama.cpp#30928. Three quantizations are listed, with the default declaring the others as variants: UD-Q4_K_XL (25 GB), UD-Q2_K_XL (13.6 GB) for small hosts and UD-Q6_K_XL (36 GB) for larger ones. Sampling defaults follow the model card (temperature 1, top_p 0.95, top_k 20). The library file gallery/models/index.yaml is edited and gallery/index.yaml is regenerated from it. Assisted-by: Claude:claude-sonnet-5-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
a37cc46104eb50900d5ab4a9ba02a0abdac039db(2026-10-21), the merge of models: Add Flint model (FlintForCausalLM) ggml-org/llama.cpp#30928 that adds the Flint architecture. Flint cannot load on the previous pin.flintgallery entry (Unsloth GGUF of RecursiveSI/flint) with two variants.Patch decisions
0002-add-server-task-type-tts.patch: removed. Upstream now providesSERVER_TASK_TYPE_TTS,server_task::tts_inp,server_task_result_tts,server_context_meta::has_cap_ttsand the mtmd gen-audio helpers. On the new pin the patch is detected as already applied, andgrpc-server.cppuses the same symbols. The bonsai and turboquant forks still lack them and keepdisable-tts-task.sh.0001-add-server-task-type-score.patch: still needed, upstream has noSERVER_TASK_TYPE_SCORE. Regenerated against the new pin; it applies with no offset or fuzz using theprepare.shmethod (patch -p1). One hunk needed a manual fix because upstream addedSERVER_TASK_TYPE_TRANSCRIBE_RTtoserver_task::need_logits();SCOREnow sits next to it.Gallery entry
flintis UD-Q4_K_XL (25 GB), the default, and declaresflint-q2(UD-Q2_K_XL, 13.6 GB, small hosts) andflint-q6(UD-Q6_K_XL, 36 GB, larger hosts). This mirrors the default-plus-smaller-variant shape of neighbouring entries. IQ1/IQ2/IQ3 and Q5/Q8 are left out to keep the list short; they can be added later.x-linked-etagheader; no weights were downloaded.gallery/models/index.yamledited;gallery/index.yamlregenerated withmodelcatalog(additions only).apache-2.0, which this entry uses. The GGUF metadata reportsother(nameflint-preview). This needs a maintainer check.Verification
go test ./scripts/models/... ./core/gallery/...: pass.modelcatalogregeneration: clean, 2215/2215 entries round-trip.Not verified
grpc-server.cppwas not compiled against the new pin. Other API changes between the old and new pin were only reviewed by reading headers.go build ./...fails in this environment on an unrelated cgo dependency (go-piper needs spdlog headers).Assisted-by: Claude:claude-sonnet-5-5
🤖 Generated with Claude Code