Repository navigation
Conversation
…-omni, release v0.8.57 No llama_nif.cpp change needed: include/llama.h changes comments only, and the chat API refactor (#30210) leaves common_chat_templates_inputs / common_chat_templates_apply, the only chat calls the NIF makes, intact. Default params unchanged; RPC protocol stays 8.0.0. The three release-guide defects still stand by source diff. - decision.cpp / Decision: port the lfm2-d1-omni type (#30114, LiquidAI/d1-omni-600M): mask-marker scores from a memory-less non-causal LFM2, per-piece token budgets (fill_task_d1omni), special-token escaping, null state, yes/no noul descriptions, one-batch evaluation. parse_state reads "files" with "images" as alias and rejects input_audio parts. "Ported at" is 23b0202a1. - decision_compare.exs: three more requests (yes/no noul, over-budget options, special text). ALL MATCH vs upstream llama-server (Metal) on tinylaya, tinyopenjev, Clef Flash, d1-3B and d1-omni-600M. - :mtp_cancel aborts the VM again (4/6 runs), identically at 88dcc460d; recorded in the test and test_helper comments. Verified on Metal (M1 Max): 430 default, 574 smoke/embeddings/slow/mtp, 441 embeddings (embeddinggemma-2), 442 decision, 3/3 decision_clef, 436 passed / 6 skipped mtp_sidecar; CPU 442 decision. Format and dialyzer clean; Hex source build clones the pinned SHA.
nyo16
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bumps llama.cpp from
88dcc460d(b11479+4) to23b0202a1(b11552), 69 commits, and adds support for upstream's newlfm2-d1-omnidecision model (LiquidAI/d1-omni-600M, upstream #30114).llama_nif.cppneeded no changes.include/llama.hchanges comments only. The chat API refactor (#30210) leaves the two chatcalls the NIF uses,
common_chat_templates_inputsandcommon_chat_templates_apply, unchanged. The default params are thesame, and the RPC protocol stays at 8.0.0.
Changes
lfm2-d1-omni, ported intodecision.cpp:<|mask|>markerfill_task_d1omni)nilstatenouldescriptions under"yes"/"no"LlamaCppEx.Decisionmaps the type to:lfm2_d1_omniand gives it an embeddings context.parse_state): accepts"files"as another name for"images", as upstream does, and rejectsinput_audioparts. This build doesn't link libmtmd, so it has no image or audio input.scripts/decision_compare.exs: three new requests: yes/no answer descriptions, options longer than the token limits,and special-token text in the input.
LLAMA_COMMITin theMakefilematches the submodule. The version is 0.8.57. CHANGELOG, README (d1-omnisection with measured answers), release guide (upstream-defect re-check at
23b0202a1) anddecision.exdocs are updated.:mtp_cancel: the known-broken test now crashes the VM in 4 of 6 runs. The previous pin gives the same 4 of 6, so thisbump didn't cause it. Recorded in the comments of
test/mtp_model_test.exsandtest/test_helper.exs.Verification (M1 Max)
mix format,mix dialyzer23b0202a1and compilesscripts/decision_compare.exsagainst upstreamllama-server, built from the same commit (Metal), returns ALL MATCH ontinylaya, tinyopenjev, Clef Flash, d1-3B and d1-omni-600M. Token counts are identical everywhere. On d1-omni, single-question
requests match exactly, and multi-question requests differ by up to 6.6e-4.
d1-omni-600M speed (Q8_0, fully on the GPU): about 14 ms per question once warm (3 questions: 41.6 ms median). The first call
after loading takes 87 ms.
Not run:
rpc_live, and the smoke/MTP suites on a CPU build.