fix compile error on centos 7.5 by leaout · Pull Request #226 · ggml-org/llama.cpp

leaout · 2023-03-17T07:01:04Z

No description provided.

j-f1 · 2023-03-17T12:50:15Z

 endif

+ifneq ($(CC), gcc)
+	CC=gcc


If the person running make really wants to use a different CC, wouldn’t this override them? Have you tried using CMake instead?

leaout · 2023-03-17T14:45:10Z

Of course cmake worked

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

* llama : expose llama_get_sched() in the C API Add a public C-API accessor returning a context's ggml_backend_sched_t, so downstream callers can enumerate per-backend compute buffer sizes via the existing public ggml_backend_sched_* API (memory observability). Thin wrapper over the existing llama_context::get_sched(); no behavior change. Carried as a titanium-chicken fork patch on top of b9828. * ggml-backend : expose ggml_backend_sched_get_graph() in the C API Returns the scheduler's current compute graph (the one split during ggml_backend_sched_reserve, which llama runs at context construction). Its nodes are in execution order with per-node backend assignments queryable via ggml_backend_sched_get_tensor_backend, enabling read-only compute-order/placement inspection without running a forward pass. Sibling to the existing llama_get_sched() patch (titanium-chicken fork). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * metal: persistent pointer-keyed boundary-event schedule for paged decode (ggml-org#226) Add a persistent, pointer-keyed boundary-event schedule to the Metal backend so an engine can drive async event-fenced KV residency without a per-boundary GPU host drain (the cb_eval ggml_backend_synchronize that dominates paged decode). - ggml_metal_set_boundary_schedule(): register cut nodes (each optionally signalling an event after its command buffer) and wait nodes (each waiting on an event before its segment), keyed by ggml_tensor* POINTER. PERSISTENT: it applies to every graph_compute until cleared (n_cuts==0), so it covers all of ggml_backend_sched's splits within one llama_decode and the reused graph of later tokens; pointer matching is robust to split-vs-full-graph index skew. n_cuts==0 leaves the stock n_cb encode path untouched. - ggml_metal_graph_compute_paged(): sequential per-boundary-committed encode (no dispatch_apply) that, for whichever schedule nodes appear in the graph it is handed, emits encodeSignalEvent after each boundary's command buffer and encodeWaitForEvent before an admitted region's first-reader segment. Reuses ggml_metal_op per segment (fusion preserved). A graph with no schedule nodes encodes as one segment. - Explicit-value event primitives: ggml_metal_event_encode_{signal,wait}_value and host-side ggml_metal_event_host_{signal,wait,value} so an engine reclaim/prefetch thread can coordinate an MTLSharedEvent across threads. - ggml_metal_get_device(): the backend's own device (ggml_metal_device_get() returns a fresh, unrelated device each call, whose events never reach the queue) -- callers must init boundary events on this device. Verified by a standalone probe over a real ggml-metal graph: the segmented encode is bit-identical to the stock encode, every per-boundary signal fires progressively, and an admit wait fences real GPU work until a concurrent host signal from a long-lived thread releases it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PR4CoRiTG8d5YfpjumYJSS * wip(metal): signal boundary events on cmd-buffer completion (ggml-org#226) Adds ggml_metal_event_signal_on_complete: a host reclaim thread waiting the boundary event must see coherent shared-storage KV bytes, so signal from the completion handler rather than mid-stream. Separate WIP commit on top of the seam (542d159) to preserve; squash when formalizing the fork PR. Had zero effect on the paging bit-identity bug (keep-or-drop). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PR4CoRiTG8d5YfpjumYJSS * feat(metal): public boundary-event schedule + event API for paged decode Exposes ggml_backend_metal_set_boundary_schedule plus the ggml_backend_metal_event_* primitives so an engine can drive fusion-preserving, event-fenced KV paging instead of a whole-graph host synchronize at each cut. Claude-Session: https://claude.ai/code/session_01GWUngsbi1CQNi4cBrmNFCG Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Update llama.h ti remove comment * metal: trim declaration comments, drop external references Remove the multi-line block comments above the newly-added forward declarations to match the terse style of the surrounding headers, and strip references to external trackers/repositories from the remaining in-body comments. --------- Co-authored-by: Yi Wen Wong <yiwenwong@Yis-MacBook-Air.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…#8) * metal: persistent pointer-keyed boundary-event schedule for paged decode (ggml-org#226) Add a persistent, pointer-keyed boundary-event schedule to the Metal backend so an engine can drive async event-fenced KV residency without a per-boundary GPU host drain (the cb_eval ggml_backend_synchronize that dominates paged decode). - ggml_metal_set_boundary_schedule(): register cut nodes (each optionally signalling an event after its command buffer) and wait nodes (each waiting on an event before its segment), keyed by ggml_tensor* POINTER. PERSISTENT: it applies to every graph_compute until cleared (n_cuts==0), so it covers all of ggml_backend_sched's splits within one llama_decode and the reused graph of later tokens; pointer matching is robust to split-vs-full-graph index skew. n_cuts==0 leaves the stock n_cb encode path untouched. - ggml_metal_graph_compute_paged(): sequential per-boundary-committed encode (no dispatch_apply) that, for whichever schedule nodes appear in the graph it is handed, emits encodeSignalEvent after each boundary's command buffer and encodeWaitForEvent before an admitted region's first-reader segment. Reuses ggml_metal_op per segment (fusion preserved). A graph with no schedule nodes encodes as one segment. - Explicit-value event primitives: ggml_metal_event_encode_{signal,wait}_value and host-side ggml_metal_event_host_{signal,wait,value} so an engine reclaim/prefetch thread can coordinate an MTLSharedEvent across threads. - ggml_metal_get_device(): the backend's own device (ggml_metal_device_get() returns a fresh, unrelated device each call, whose events never reach the queue) -- callers must init boundary events on this device. Verified by a standalone probe over a real ggml-metal graph: the segmented encode is bit-identical to the stock encode, every per-boundary signal fires progressively, and an admit wait fences real GPU work until a concurrent host signal from a long-lived thread releases it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PR4CoRiTG8d5YfpjumYJSS * wip(metal): signal boundary events on cmd-buffer completion (ggml-org#226) Adds ggml_metal_event_signal_on_complete: a host reclaim thread waiting the boundary event must see coherent shared-storage KV bytes, so signal from the completion handler rather than mid-stream. Separate WIP commit on top of the seam (542d159) to preserve; squash when formalizing the fork PR. Had zero effect on the paging bit-identity bug (keep-or-drop). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PR4CoRiTG8d5YfpjumYJSS * feat(metal): public boundary-event schedule + event API for paged decode Exposes ggml_backend_metal_set_boundary_schedule plus the ggml_backend_metal_event_* primitives so an engine can drive fusion-preserving, event-fenced KV paging instead of a whole-graph host synchronize at each cut. Claude-Session: https://claude.ai/code/session_01GWUngsbi1CQNi4cBrmNFCG Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Update llama.h ti remove comment * metal: trim declaration comments, drop external references Remove the multi-line block comments above the newly-added forward declarations to match the terse style of the surrounding headers, and strip references to external trackers/repositories from the remaining in-body comments. * metal: check for NULL from ggml_metal_buffer_init / _map (#6) ggml_metal_buffer_init returns NULL when the buffer allocation or the residency set fails, but ggml_backend_metal_buffer_type_alloc_buffer passed the result straight to ggml_metal_buffer_is_shared, which dereferences it at ->is_shared (offset 0x10). An allocation failure that ggml already logs and handles became a SIGSEGV. ggml-alloc.c already treats a NULL from alloc_buffer as a recoverable failure, so returning NULL is the expected contract. ggml_backend_metal_device_buffer_mapped had the same unchecked pattern around ggml_metal_buffer_map. That one built a buffer with a NULL context instead of faulting immediately; llama-model.cpp already checks the result for nullptr. Assisted-by: Claude Opus 4.8 Co-authored-by: Yi Wen Wong <yiwenwong@Yis-MacBook-Air.local> * metal: encode window over the paged (boundary-scheduled) compute path While a boundary schedule is set, an optional encode window restricts ggml_metal_graph_compute_paged to the segments intersecting an inclusive [first_node, last_node] range, matched by pointer per graph like the schedule's own cut/wait nodes (a graph containing neither endpoint encodes in full). Optional blit pairs move a boundary activation into the window's first encoded segment (before its ops, after its schedule waits) and out of its last (after its ops) as device-side blits between resident tensors -- no host round trip. Persistent until cleared; cleared state is byte-identical to the pre-window paged path. --------- Co-authored-by: Yi Wen Wong <yiwenwong@Yis-MacBook-Air.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@Chris-behind-door

…gml-org#249) Mirrors ggml-org#226's CUDA CMakeLists addition into ggml-hip: the four mixed f16/bf16 + q8_0 vec instances were dispatched by the runtime selector but absent from the HIP instance list after the ggml-org#244 rebase, so any default HIP build failed to link (undefined ggml_cuda_flash_attn_ext_vec_case references). Reported and locally validated by @Chris-behind-door.

…r, QSA/indexer) The qwen4exp (Qwen3.8-Flash-Next) architecture support, split out of the old block 10 (r109) so the model graph, the sparse-attention op and the hyper-connections no longer share a commit with the slab, the fusion dispatcher and the async input ring. The tree is unchanged: block 10 plus this commit is exactly what the old block 10 carried. Contents: - src/models/qwen4exp.cpp: the model, the HC (hyper-connection) graph, the MTP draft head, the PLE gather, the QSA selection/attention graph (build_qsa_store_k / build_qsa_top_k / build_attn_qsa) and the QSA arm policy. - QSA/indexer: fattn-qsa.{cu,cuh}, fattn-qsa3.cu, indexer-score.{cu,cuh}, indexer-topk.{cu,cuh}, the 4-head WMMA indexer scorer in lightning-indexer.cu, with the CUDA dispatch (GGML_OP_FLASH_ATTN_QSA / INDEXER_TOPK / INDEXER_SCORE / INDEXER_FILL) and their CPU references; the keys-only indexer cache (LLAMA_QSA_KEYS_ONLY, no V tensor at all) in llama-memory-hybrid-idx and the v_enabled parameter on llama_kv_cache. - HC ops GGML_OP_HC_MIX / HC_COMBINE (dispatch, supports_op, the Meta split states, the hc_combine_mix / hc_mix / hc_combine_norm / hc_gate_mix fusion arms and the HC16 planar layout) in hc-mix.{cu,cuh} and hyperconn.{cu,cuh}. - The managed lazy reader (llama-lazy-reader) and the loader's managed_budget (LLAMA_LAZY_BUF_MB, TENSOR_SKIP_MANAGED, lazy_read::add_range) for the PLE n-gram table; off by default (it measured slower than both the mmap and the resident path on gfx1151). - The QSA/HC op definitions in ggml.h / ggml.c, the scheduler op allowlists, the CPU references, and the qwen4exp tensor-split arch gate in llama-arch.cpp (with the ggml-hip CMakeLists publishing GGML_USE_HIP to the main libraries so the dynamic-backend build reads it). - The qwen4exp MTP draft context gets a hybrid-idx memory instead of a plain KV cache (LLAMA_MTP_SPARSE=0 restores the dense draft), and llama_memory_recurrent stops warning about the speculative verify positions for a context whose recurrent filter matches nothing. - Loader/user-path fix: TENSOR_READ_LAZY is only consulted when the loader has files (the llama_model_init_from_user path has nothing to lazy-read from). - The qwen4exp-family fusion arms that live in this block's own TUs, and the graph-optimizer passes they need: the HC_BLK16/RES16 BF16-stream marks and their alloc deps, and the prefill indexer relu-sum alloc dep. - Tests: test-hc-mix and test_flash_attn_qsa in test-backend-ops (the QSA CPU oracle: `test-backend-ops -o FLASH_ATTN_QSA`, 18/18 minimum), test-lazy-reader, and the lazy_buf_size handover in test-llama-archs. Split note (r109): the infrastructure this block is built on -- the slab, the fusion dispatcher TU, the async input ring, the GDN/MMB/HC16 kernel work -- is block 10 and is listed there. A qwen4exp-only change that is implemented inside a general kernel TU (the GDN gate/beta BF16 fusion, the HC16 MMB BF16 streams, the weighted-down reduction, the PLE conv fusion) deliberately stays with that TU in block 10; only the code that needs this block's own symbols is here. - The separate-tensor (pre-ggml-org#31028) HC mixer emits its inject product right behind the down product, i.e. adjacent in the graph, and expands both: the two BF16 GEMMs read the same F32 `xn`, and the CUDA BF16 src1 cache (block 10) only holds one converted tensor. ggml-org#31028's ordering (inject last, after the gate product whose src1 is the silu output, which evicts the entry) converted the whole [hc*n_embd, n_tok] tensor twice per mixer -- measured on Flash-Next GSQ-IQ3_XXS as 2x the F32->BF16 `convert_unary_cont_vec4` launches and +3.4 % prefill when restored (issue ggml-org#226 follow-up).

…r, QSA/indexer) The qwen4exp (Qwen3.8-Flash-Next) architecture support, split out of the old block 10 (r109) so the model graph, the sparse-attention op and the hyper-connections no longer share a commit with the slab, the fusion dispatcher and the async input ring. The tree is unchanged: block 10 plus this commit is exactly what the old block 10 carried. Contents: - src/models/qwen4exp.cpp: the model, the HC (hyper-connection) graph, the MTP draft head, the PLE gather, the QSA selection/attention graph (build_qsa_store_k / build_qsa_top_k / build_attn_qsa) and the QSA arm policy. - QSA/indexer: fattn-qsa.{cu,cuh}, fattn-qsa3.cu, indexer-score.{cu,cuh}, indexer-topk.{cu,cuh}, the 4-head WMMA indexer scorer in lightning-indexer.cu, with the CUDA dispatch (GGML_OP_FLASH_ATTN_QSA / INDEXER_TOPK / INDEXER_SCORE / INDEXER_FILL) and their CPU references; the keys-only indexer cache (LLAMA_QSA_KEYS_ONLY, no V tensor at all) in llama-memory-hybrid-idx and the v_enabled parameter on llama_kv_cache. - HC ops GGML_OP_HC_MIX / HC_COMBINE (dispatch, supports_op, the Meta split states, the hc_combine_mix / hc_mix / hc_combine_norm / hc_gate_mix fusion arms and the HC16 planar layout) in hc-mix.{cu,cuh} and hyperconn.{cu,cuh}. - The managed lazy reader (llama-lazy-reader) and the loader's managed_budget (LLAMA_LAZY_BUF_MB, TENSOR_SKIP_MANAGED, lazy_read::add_range) for the PLE n-gram table; off by default (it measured slower than both the mmap and the resident path on gfx1151). - The QSA/HC op definitions in ggml.h / ggml.c, the scheduler op allowlists, the CPU references, and the qwen4exp tensor-split arch gate in llama-arch.cpp (with the ggml-hip CMakeLists publishing GGML_USE_HIP to the main libraries so the dynamic-backend build reads it). - The qwen4exp MTP draft context gets a hybrid-idx memory instead of a plain KV cache (LLAMA_MTP_SPARSE=0 restores the dense draft), and llama_memory_recurrent stops warning about the speculative verify positions for a context whose recurrent filter matches nothing. - Loader/user-path fix: TENSOR_READ_LAZY is only consulted when the loader has files (the llama_model_init_from_user path has nothing to lazy-read from). - The qwen4exp-family fusion arms that live in this block's own TUs, and the graph-optimizer passes they need: the HC_BLK16/RES16 BF16-stream marks and their alloc deps, and the prefill indexer relu-sum alloc dep. - Tests: test-hc-mix and test_flash_attn_qsa in test-backend-ops (the QSA CPU oracle: `test-backend-ops -o FLASH_ATTN_QSA`, 18/18 minimum), test-lazy-reader, and the lazy_buf_size handover in test-llama-archs. Split note (r109): the infrastructure this block is built on -- the slab, the fusion dispatcher TU, the async input ring, the GDN/MMB/HC16 kernel work -- is block 10 and is listed there. A qwen4exp-only change that is implemented inside a general kernel TU (the GDN gate/beta BF16 fusion, the HC16 MMB BF16 streams, the weighted-down reduction, the PLE conv fusion) deliberately stays with that TU in block 10; only the code that needs this block's own symbols is here. - The separate-tensor (pre-ggml-org#31028) HC mixer emits its inject product right behind the down product, i.e. adjacent in the graph, and expands both: the two BF16 GEMMs read the same F32 `xn`, and the CUDA BF16 src1 cache (block 10) only holds one converted tensor. ggml-org#31028's ordering (inject last, after the gate product whose src1 is the silu output, which evicts the entry) converted the whole [hc*n_embd, n_tok] tensor twice per mixer -- measured on Flash-Next GSQ-IQ3_XXS as 2x the F32->BF16 `convert_unary_cont_vec4` launches and +3.4 % prefill when restored (issue ggml-org#226 follow-up). R175 amendment: the QSA combine launch passes the new `block_q8_1 * q8` argument of `flash_attn_combine_results` (added by block 10's FA-combine Q8_1 publish) as `nullptr` -- QSA's output is not fed to a quantized matmul, so it publishes nothing.

fix compile error on centos 7.5

3877722

j-f1 reviewed Mar 17, 2023

View reviewed changes

leaout closed this Mar 17, 2023

Bearsaerker mentioned this pull request Mar 12, 2025

Eval bug: Gemma 3 extremly slow prompt processing when using quantized kv cache. #12352

Closed

phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026

Fix compilation error with IQK_FA_ALL_QUANTS enabled (ggml-org#226)

2212c1c

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>

yingying0906 mentioned this pull request Jul 3, 2026

QVAC-21396 fix(mtmd/qwen3vl): grid selection rewrite + --image-max-tiles override tetherto/qvac-fabric-llm.cpp#175

Merged

1 of 4 tasks

Navigation Menu

Repository navigation

fix compile error on centos 7.5 - #226

fix compile error on centos 7.5#226
leaout wants to merge 1 commit into
ggml-org:masterfrom
leaout:master

leaout commented Mar 17, 2023

Uh oh!

j-f1 Mar 17, 2023

Uh oh!

leaout commented Mar 17, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

leaout commented Mar 17, 2023

Uh oh!

j-f1 Mar 17, 2023

Choose a reason for hiding this comment

Uh oh!

leaout commented Mar 17, 2023

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants