[m4zar]

Overview

gemma3n and gemma4 allocate a second copy of token_embd when loading without mmap (-lm none, -lm mlock) and both input and output layers share the same buffer type. This occurs on CPU-only runs, -ngl 0, or partial offload where the output layer stays on the CPU. With default mmap, both tensors simply point to the same mapped pages.

Currently, both models create the tied output before token_embd. The duplicated-tensor lookup in llama_model_loader::create_tensor runs at that point, finds no token_embd yet, and creates a second one. The fix is to create token_embd first. This is the order the other 94 tied-output architectures already use; gemma3n and gemma4 were the only exceptions.

Additional information

Tested with gemma-4-E2B-it Q4_K_M, CPU only, no mmap:

llama-server -m gemma-4-E2B-it-Q4_K_M.gguf -dev none -ngl 0 -lm none --no-repack -lv 4

master cf24317be:  load_tensors:          CPU model buffer size =  3163.73 MiB
this PR:           load_tensors:          CPU model buffer size =  2947.73 MiB

The 216 MiB difference is the exact size of token_embd. On the same model loaded CPU-only, greedy output remains identical to master both with and without mmap.

test-llama-archs currently skips GEMMA4, so there is no test included. With -ngl 99 nothing changes: the output layer is on the GPU and still gets its own copy of token_embd, just like every other tied-output architecture.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES. I used an AI assistant to run and check the measurements and to review the draft. The code and the text are mine.

[nikwen]

/bot review

[ggml-gh-bot]

Automated code review

I reviewed the diff (4-line reorder in src/models/gemma3n.cpp and src/models/gemma4.cpp), the create_tensor/TENSOR_DUPLICATED machinery in src/llama-model-loader.cpp, the surrounding model files, and the test coverage. Checklists run: scope/quick-reject gate, security, general conventions (the only touched areas are src/models/, no new component, so no approach/design pass was needed).

Verification of the core claim

In llama_model_loader::create_tensor (src/llama-model-loader.cpp:1384-1389), a TENSOR_DUPLICATED create looks up the tensor by name in the ctx for the resolved buffer type and reuses it if found. With the old order (tied output created before tok_embd), the lookup missed, so a second token_embd.weight was allocated in the same CPU ctx when the output layer stays on CPU - this matches the reported 216 MiB duplication. With tok_embd created first, the lookup hits whenever both resolve to the same buffer type, and output aliases tok_embd, exactly as in gemma.cpp, gemma2.cpp, and the other tied archs. When the output layer is on a different device, buft_for_tensor maps the duplicated create to LLM_TENSOR_OUTPUT (line 1179) and resolves a different ctx, so the lookup misses and a separate copy is still created - the -ngl 99 behavior is preserved. Accounting is consistent with the shared path used by all other archs (reuse adds neither n_created nor size_data; the weights map is keyed by name, so the weight is read once).

Findings

No blocking issues. The change is minimal, correct, and aligns gemma3n/gemma4 with the convention every other tied-output architecture already follows.

(will slow the review, minor) The PR description says "test-llama-archs currently skips GEMMA4, so there is no test included", but GEMMA3N is not in the skip list (arch_supported in tests/test-llama-archs.cpp:994 only skips GEMMA4/GEMMA4_ASSISTANT). The gemma3n half of this change is therefore exercised by existing CI (the fixture loads a real tied gemma3n GGUF header). No new test is needed, but the description's justification is slightly off, and it's worth confirming the gemma3n test-llama-archs job passes on this branch, since that run covers the new aliasing path (output == tok_embd) end to end on the CPU backend.

Nits (optional)

(1) In both files the retained comment "if output is NULL, init from the input tok embed" sits directly under a create that now reuses tok_embd via the shared lookup; the comment is still accurate enough, but if you touch these lines again, "reuse the input tok embed" would describe the new flow better.

Nice side effect worth knowing for the commit text: the old order also left two distinct tensors with the identical name token_embd.weight in the same ggml ctx, which the reorder eliminates.

This review was generated automatically by pi coding agent using zai-org/GLM-5.3. It may contain mistakes. Maintainers make the final call.

[nikwen]

Good catch, thanks!

[nikwen]

test-llama-archs currently skips GEMMA4

Follow-up PR enabling Gemma 4 there: #31401