ggml-org#27205

Upstream issue draft — ggml-openvino: reshape-aliased split-boundary input bound with source tensor's shape

Target repo: ggml-org/llama.cpp (OpenVINO backend, ggml/src/ggml-openvino/)

Related existing issues (searched 2026-08-16)

• openvino: create_ov_output_tensor uses wrong shape/pointer for passthrough VIEW (op_case=0), breaking GPU plugin input validation ggml-org/llama.cpp#21197 (closed-as-stale 2026-03-30, NO merged fix): the same [GPU] The tensor size is not equal to model bind rejection for passthrough VIEW outputs
  (create_ov_output_tensor uses the view's shape/pointer instead of the
  parent's — op_case=0). Same exception family and same code path family, but
  the VIEW-output case; a commenter reports it is still broken
  ("this is not stale, this is still an issue"). This report covers the
  RESHAPE-as-model-input case on split-boundary graphs, with a verified
  fix (below).
• qwen35moe Bug Report for llama.cpp ggml-org/llama.cpp#25857 (open, 2026-07-18): "qwen35moe Bug Report" — attachment-only, no
  backend/error detail; cannot be matched to this defect.

Title

ggml-openvino: RESHAPE result that crosses a graph-split boundary is bound with
the source tensor's shape → GPU plugin rejects input ("tensor size is not equal
to model") on hybrid MoE models (e.g. qwen35moe 35B)

Summary

On hybrid-attention MoE models (qwen35moe family, e.g. "Ornith 1.0 35B"), every
inference fails at the first compute with an OpenVINO shape mismatch:

[GPU] The tensor size is not equal to model, can't set input tensor with index: 7,
because model input (shape=[1,2,8,1]) and tensor (shape=[1,1,2,8]) are incompatible

(src/plugins/intel_gpu/src/plugin/sync_infer_request.cpp:184, ov::Exception →
ggml_backend_sched_graph_compute_async failed with error -1 → HTTP 500).

The failing input is a RESHAPE result that crosses a subgraph split boundary:
ffn_moe_weights_norm-0 (reshaped) — the ggml_reshape_3d(ffn_moe_weights_norm, 1, n_expert_used, n_tokens) node (n_expert_used=8).

• Compiled model input shape (baked at compile time): [1,2,8,1] (OV order) =
  ggml ne=[1,8,2,1] (the reshape result's shape, 2 = n_tokens).
• Runtime tensor bound: [1,1,2,8] (OV order) = ggml ne=[8,2,1,1] — the
  source 2D tensor's shape (ffn_moe_weights_norm, ne=[8, n_tokens]).

Same element count, same data pointer — only the shape descriptor is wrong.
Cache-INDEPENDENT: fails on the first compute in the same process that just
compiled. Model-agnostic in the sense that it depends on whether a reshape
result lands on a split boundary (shape-sensitive; a smaller model of the same
architecture, ornith:9b, computes fine).

Root cause

ggml_backend_openvino_buffer_init_tensor() (ggml-openvino.cpp) shares the
tensor extra (the ov::Tensor descriptor) from view_src for any tensor
with view_src != nullptr:

// Views share the extra from view_src
if (tensor->view_src != nullptr) {
    GGML_ASSERT(tensor->view_src->buffer->buft == buffer->buft);
    if (tensor->view_src->extra != nullptr) {
        tensor->extra = tensor->view_src->extra;
    }
    return GGML_STATUS_SUCCESS;
}

ggml_new_tensor_impl() sets view_src for reshape results as well as
views. A reshape result therefore inherits the source tensor's ov::Tensor,
whose shape is the source's shape ([8, n_tokens] → OV [1,1,n_tokens,8]).

At inference time convert_ggml_input_to_ov() (utils.cpp) returns the tensor's
extra directly when it exists:

if (ggml_tensor->extra != nullptr && !ggml_decoder->is_splited_model()) {
    ...
    return *tensor_extra->tensor;   // shape = the SOURCE's shape for reshape aliases
}

When the reshape result is a model input (it crossed a scheduler split
boundary — compute_model_inputs() registers it because it is not an
intermediate node of the current subgraph), the compiled model's Parameter has
the reshape result's shape while the runtime binding carries the source's
shape → shape check fails.

The data is identical in both cases (reshape is a memory no-op in ggml:
[8,n_tokens] → [1,8,n_tokens,1] keeps dim0 fastest), so wrapping the same
pointer with the reshape's shape is exactly correct.

Minimal repro

1. Build llama.cpp master with -DGGML_OPENVINO=ON (any recent commit; verified
   on 885c5bb, build b10428) and run llama-server with -dev OPENVINO0 -ngl 99 -lv 5.
2. Load a qwen35moe-arch GGUF where a ffn_moe_weights_norm reshape lands on a
   split boundary (reproduced with "Ornith 1.0 35B", 20G GGUF, -c 16384).
3. Any completion → the ov::Exception above; server returns HTTP 500 "Compute error".

Log signature (before the exception):

W ggml-openvino: cannot determine dynamic dim for RESHAPE node 'conv_states_reshaped-0'
W ggml-openvino: dynamic dim value mismatch for VIEW node 'conv_state_last-0', src[0]: 'conv_input-0'
...
E GGML OpenVINO backend ov::Exception: ... [GPU] The tensor size is not equal to model,
  can't set input tensor with index: 7, because model input (shape=[1,2,8,1]) and tensor
  (shape=[1,1,2,8]) are incompatible

Note: the dynamic-dim warnings for conv-state tensors are benign noise — a
smaller model of the same architecture (ornith:9b) prints the identical
warnings and computes fine. The failure is the reshape-input shape mismatch.

Proposed fix

In ggml_backend_openvino_buffer_init_tensor(), only share the view_src
extra for true views; let reshape results fall through to create an extra with
their own shape:

// Views share the extra from view_src. RESHAPE nodes also alias the source
// data (ggml_new_tensor_impl sets view_src for reshape results), but they have
// a DIFFERENT shape: sharing the source's ov::Tensor would bind inputs with
// the source's shape and fail the plugin's shape check at graph-compute time.
if (tensor->view_src != nullptr && tensor->op == GGML_OP_VIEW) {
    GGML_ASSERT(tensor->view_src->buffer->buft == buffer->buft);
    if (tensor->view_src->extra != nullptr) {
        tensor->extra = tensor->view_src->extra;
    }
    return GGML_STATUS_SUCCESS;
}

(Only GGML_OP_VIEW and GGML_OP_RESHAPE set view_src in ggml.)

Optionally also guard convert_ggml_input_to_ov() against returning an extra
whose shape differs from get_shape(ggml_tensor).

Environment

• llama.cpp 885c5bb (b10428), OpenVINO 2026.4.0-git intel_gpu plugin
• Intel Core Ultra 7 356H (Panther Lake), iGPU, 32 GB shared RAM
• X1 Carbon Gen 14

Verification (with the fix applied) — 2026-08-16, TASK-30 wrap-up

Fixed binary: llama.cpp 885c5bb + the patch below (built 2026-08-16 21:13),
run with -ngl 99 -dev OPENVINO0 -lv 5 --no-cache-prompt, env
GGML_OPENVINO_DEVICE=GPU, GGML_OPENVINO_MEMORY_OPTIMIZE=1, scoped cgroup
(MemoryHigh=29G/MemoryMax=55G/SwapMax=16G, CPUQuota=400%).

• ornith:35b: cold compile completes (574 cache blobs), /health ok, and a full
  completion returns: stopped by limit, n_gen = 8, n_predict = 8 (tokens
  2, <think>, Here, 's, a, thinking...) — direct curl /completion
  → HTTP 200 {"content":"2\n\n<think>\nHere's a thinking","stop":true, "tokens_predicted":8,"tokens_evaluated":4}. Zero ov::Exception /
  Compute error / repack asserts in the log (grep count 0). Runtime memory
  profile clean: model 1452 MiB retained + context 571 + compute 60 on the
  OPENVINO0 device pool; 41/41 layers offloaded to GPU.
• ornith:9b (same architecture): still computes fine (no regression).
• qwen3.5:4b: still computes fine.

Also verified: the patch applies cleanly (git apply --check, exit 0) to a
fresh clone at 885c5bb; the combined patch set reproduces the verified build
exactly.

Two observations for the maintainers, neither blocking:

• A minority of compiled-blob imports fail at plugin compile with could not create a primitive descriptor for the reduction primitive (intel_gpu
  plugin, plugin.cpp:80) while the same graph compiles fine via the
  fresh-compile path — the backend falls back to recompile (warm loads still
  reach /health in ~37s).
• Split-graph blobs are compiled with static token-count shapes; each distinct
  token count gets its own cache blob (by design of the related fingerprint
  fix — see the companion issue draft, upstream-issue-openvino-split-boundary-compute.md).