Skip to content

Fix mlir_translate TypeError + HostShimCAPI.h include path for a newer SDK - #166

Draft
akapoor3518 wants to merge 2 commits into
masterfrom
fix/bump-ggml-tsi-kernel-mlir-translate-fix
Draft

akapoor3518 wants to merge 2 commits into
masterfrom
fix/bump-ggml-tsi-kernel-mlir-translate-fix

Conversation

@akapoor3518

@akapoor3518 akapoor3518 commented Sep 24, 2026 •

Copy link
Copy Markdown

What

Two small, independent fixes needed to get llama.cpp building clean against a newer SDK:

  1. Submodule bump: bumps the ggml-tsi-kernel submodule to pick up a fix that removes
    a now-nonexistent mlir_translate field from the compiler-config JSON templates.
  2. CMakeLists.txt: adds ${RUNTIME_DIR}/include/tsi-rt/shim to the tsavorite backend's
    include dirs.

Why

A build failure was reported:

TypeError: TXECompilerConfig.__new__() got an unexpected keyword argument 'mlir_translate'

Fixed at the source in the ggml-tsi-kernel submodule itself (currently open, undrafted,
awaiting review — not merged, see Scope below).

Fixing just that uncovered a second, separate issue: ggml-tsavorite/include/TestModel.h
does a bare #include "HostShimCAPI.h". That only ever resolved because a compiler
package directory happened to also carry a duplicate copy of this runtime package
header, on every SDK release up to now. A newer SDK's build no longer duplicates that
header — it only exists at ${RUNTIME_DIR}/include/tsi-rt/shim/HostShimCAPI.h, one level
deeper than the existing ${RUNTIME_DIR}/include include dir already covers. Added the
missing dir explicitly so it doesn't depend on the compiler package re-exporting runtime
headers.

Verification (real, on this exact branch, against the newer SDK)

build-fpga

  • Full cmake --build succeeds, 100%. Real aarch64 ELF binaries produced (llama-cli,
    simple-backend-tsi).

build-posix

  • Full cmake --build succeeds, 100%.
  • Ran the resulting binary for real: produced real output and a real GGML perf summary
    showing nonzero TSI_KERNEL-RUN counts for ADD, MUL, RMS_NORM, MUL_MAT,
    SOFT_MAX, GLU — i.e. actual inference dispatched through the TSI OPU kernels, not
    just a clean link.

Sequence used to get here (documented for whoever picks this up next): reproduced the
exact mlir_translate failure first on an unmodified base against this SDK; confirmed it
disappears with fix #1 alone; hit HostShimCAPI.h next; root-caused and fixed with #2;
re-verified both targets build AND run clean from a fully fresh kernel/blob build (a stale
partial blob-build state from an earlier attempt had briefly masked an unrelated local
config issue on the verifier's own environment — not a code issue).

Scope / sequencing

  • Depends on the upstream ggml-tsi-kernel fix landing first. Left as draft until that
    merges — the submodule commit pinned here is that fix's current branch-tip commit, and
    may need re-pointing once it merges (e.g. if squash-merged rather than fast-forwarded).
  • This does not by itself fix all downstream consumers: some repos pin llama.cpp at a
    fixed release tag rather than tracking main, so they won't see this until a new
    release tag is cut that includes it and their own pin is bumped — their normal release
    process. Other repos vendor llama.cpp via a separate sync mechanism and will need
    their own vendor-pin bump once this lands, following the established pattern for prior
    vendor-pin-sync changes. Flagging both as the next real steps, not doing them in this PR.

@akapoor3518 akapoor3518 changed the title Bump ggml-tsi-kernel submodule to fix mlir_translate TypeError against private SDK Bump ggml-tsi-kernel submodule to fix mlir_translate TypeError against a newer SDK Sep 24, 2026
@akapoor3518 akapoor3518 changed the title Bump ggml-tsi-kernel submodule to fix mlir_translate TypeError against a newer SDK Fix mlir_translate TypeError + HostShimCAPI.h include path for private SDK Sep 24, 2026
@akapoor3518 akapoor3518 changed the title Fix mlir_translate TypeError + HostShimCAPI.h include path for private SDK Fix mlir_translate TypeError + HostShimCAPI.h include path for a newer SDK Sep 24, 2026
Anoop Kapoor added 2 commits September 23, 2026 22:59
The upstream ggml-tsi-kernel fix removes the now-nonexistent
mlir_translate field from the three committed compiler-config JSON
templates, fixing:

  TypeError: TXECompilerConfig.__new__() got an unexpected keyword
  argument 'mlir_translate'

against SDK builds whose compiler dropped that field. This bumps the
submodule pointer from 067617c (pre-fix) to 785d8a2 (the fix commit)
so llama.cpp's own kernel/blob build picks it up.

No other files in llama.cpp change.
ggml-tsavorite/include/TestModel.h does a bare `#include "HostShimCAPI.h"`.
That resolved only because ${MLIR_COMPILER_DIR}/include/tsi-rt/shim (the
*compiler* package's install tree) happened to also carry a duplicate copy
of the runtime shim headers on every SDK release up to now. A newer SDK
build no longer duplicates them there as part of an in-progress refactor
-- the header only exists at ${RUNTIME_DIR}/include/tsi-rt/shim/HostShimCAPI.h
(the *runtime* package), which was already added one level too high
(${RUNTIME_DIR}/include).

Add ${RUNTIME_DIR}/include/tsi-rt/shim explicitly so the bare include
resolves regardless of whether the compiler package re-exports it.
Harmless duplicate include dir on SDKs that still do.

Verified end-to-end against the newer SDK build on this exact branch:
- build-fpga: full `cmake --build` succeeds, `llama-cli`/`simple-backend-tsi`
  linked (aarch64 ELF binaries).
- build-posix: full `cmake --build` succeeds, and the resulting llama-cli
  actually runs: real inference against a small test model, with real TSI
  OPU kernel dispatch (ADD/MUL/RMS_NORM/MUL_MAT/SOFT_MAX/GLU all show
  nonzero TSI_KERNEL-RUN counts in the perf summary), not just a clean
  compile.
@akapoor3518
akapoor3518 force-pushed the fix/bump-ggml-tsi-kernel-mlir-translate-fix branch from ceaf854 to 374b8f8 Compare September 24, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant