Skip to content

model : enable tensor split for LFM3 - #31431

Merged
CISC merged 1 commit into
ggml-org:masterfrom
tdakhran:tarek/feat/lfm3-enable-tensor-split
Oct 31, 2026
Merged

model : enable tensor split for LFM3#31431
CISC merged 1 commit into
ggml-org:masterfrom
tdakhran:tarek/feat/lfm3-enable-tensor-split

Conversation

@tdakhran

@tdakhran tdakhran commented Oct 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Overview

Enable tensor split support for LFM3

Additional information

Tested on Qualcomm SA8797P

LFM3-1.7B:Q4_0
┌──────────────────────┬────────┬──────────┬─────────┬────────────────┐
│        setup         │  PPL   │ mean KLD │ max KLD │ same top token │
├──────────────────────┼────────┼──────────┼─────────┼────────────────┤
│ 1 NPU                │ 22.026 │ 0.001265 │   0.117 │         98.16% │
├──────────────────────┼────────┼──────────┼─────────┼────────────────┤
│ 2 NPUs, tensor split │ 22.031 │ 0.001260 │   0.112 │         98.20% │
├──────────────────────┼────────┼──────────┼─────────┼────────────────┤
│ 4 NPUs, tensor split │ 22.031 │ 0.001250 │   0.110 │         98.24% │
└──────────────────────┴────────┴──────────┴─────────┴────────────────┘

┌──────────────────────┬────────┬──────┐
│        setup         │ pp1024 │ tg32 │
├──────────────────────┼────────┼──────┤
│ CPU                  │  179.7 │ 43.6 │
├──────────────────────┼────────┼──────┤
│ 1 NPU                │ 3230.8 │ 49.9 │
├──────────────────────┼────────┼──────┤
│ 2 NPUs, tensor split │ 4354.1 │ 60.5 │
├──────────────────────┼────────┼──────┤
│ 4 NPUs, tensor split │ 4695.8 │ 56.4 │
└──────────────────────┴────────┴──────┘

Requirements

@tdakhran
llama : enable tensor split for LFM3
c1cb1d0 
Assisted-by: Claude Opus 5.5
@tdakhran
tdakhran requested a review from CISC as a code owner October 30, 2026 15:38
@tdakhran tdakhran changed the title llama : enable tensor split for LFM3 model : enable tensor split for LFM3 Oct 30, 2026
@CISC
CISC merged commit bc428e4 into ggml-org:master Oct 31, 2026
17 of 21 checks passed
@leehack leehack mentioned this pull request Nov 1, 2026
Open
3 of 6 tasks
huaxel pushed a commit to huaxel/strix-llama.cpp that referenced this pull request Nov 1, 2026
@tdakhran @huaxel
llama : enable tensor split for LFM3 (ggml-org#31431)
67796e0 
Assisted-by: Claude Opus 5.5
@ggerganov ggerganov mentioned this pull request Nov 4, 2026
Merged
edwardyoon pushed a commit to edwardyoon/focus-llama that referenced this pull request Nov 6, 2026
@tdakhran @edwardyoon
llama : enable tensor split for LFM3 (ggml-org#31431)
d1edb99 
Assisted-by: Claude Opus 5.5
(cherry picked from commit bc428e4)
edwardyoon added a commit to edwardyoon/focus-llama that referenced this pull request Nov 6, 2026
@edwardyoon @qwencoder
llama: add LLM_ARCH_LFM3 enum value + arch name mapping
f62dd52 
The fork synced ggml-org#31431 (tensor split for LFM3, which references LLM_ARCH_LFM3 in llama-model.cpp) but not ggml-org#31252 (which adds the enum value), leaving master in a non-building state. Add the enum value + name mapping (matching ggml-org#31252) to unblock the build. The full LFM3 model implementation is not included.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@CISC CISC CISC approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@tdakhran @CISC