Repository navigation
Fix mlir_translate TypeError + HostShimCAPI.h include path for a newer SDK - #166
Draft
akapoor3518 wants to merge 2 commits into
Draft
akapoor3518 wants to merge 2 commits into
akapoor3518 wants to merge 2 commits into
Conversation
added 2 commits
September 23, 2026 22:59
The upstream ggml-tsi-kernel fix removes the now-nonexistent mlir_translate field from the three committed compiler-config JSON templates, fixing: TypeError: TXECompilerConfig.__new__() got an unexpected keyword argument 'mlir_translate' against SDK builds whose compiler dropped that field. This bumps the submodule pointer from 067617c (pre-fix) to 785d8a2 (the fix commit) so llama.cpp's own kernel/blob build picks it up. No other files in llama.cpp change.
ggml-tsavorite/include/TestModel.h does a bare `#include "HostShimCAPI.h"`.
That resolved only because ${MLIR_COMPILER_DIR}/include/tsi-rt/shim (the
*compiler* package's install tree) happened to also carry a duplicate copy
of the runtime shim headers on every SDK release up to now. A newer SDK
build no longer duplicates them there as part of an in-progress refactor
-- the header only exists at ${RUNTIME_DIR}/include/tsi-rt/shim/HostShimCAPI.h
(the *runtime* package), which was already added one level too high
(${RUNTIME_DIR}/include).
Add ${RUNTIME_DIR}/include/tsi-rt/shim explicitly so the bare include
resolves regardless of whether the compiler package re-exports it.
Harmless duplicate include dir on SDKs that still do.
Verified end-to-end against the newer SDK build on this exact branch:
- build-fpga: full `cmake --build` succeeds, `llama-cli`/`simple-backend-tsi`
linked (aarch64 ELF binaries).
- build-posix: full `cmake --build` succeeds, and the resulting llama-cli
actually runs: real inference against a small test model, with real TSI
OPU kernel dispatch (ADD/MUL/RMS_NORM/MUL_MAT/SOFT_MAX/GLU all show
nonzero TSI_KERNEL-RUN counts in the perf summary), not just a clean
compile.
akapoor3518
force-pushed
the
fix/bump-ggml-tsi-kernel-mlir-translate-fix
branch
from
September 24, 2026 06:00
ceaf854 to
374b8f8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two small, independent fixes needed to get
llama.cppbuilding clean against a newer SDK:ggml-tsi-kernelsubmodule to pick up a fix that removesa now-nonexistent
mlir_translatefield from the compiler-config JSON templates.${RUNTIME_DIR}/include/tsi-rt/shimto the tsavorite backend'sinclude dirs.
Why
A build failure was reported:
Fixed at the source in the
ggml-tsi-kernelsubmodule itself (currently open, undrafted,awaiting review — not merged, see Scope below).
Fixing just that uncovered a second, separate issue:
ggml-tsavorite/include/TestModel.hdoes a bare
#include "HostShimCAPI.h". That only ever resolved because a compilerpackage directory happened to also carry a duplicate copy of this runtime package
header, on every SDK release up to now. A newer SDK's build no longer duplicates that
header — it only exists at
${RUNTIME_DIR}/include/tsi-rt/shim/HostShimCAPI.h, one leveldeeper than the existing
${RUNTIME_DIR}/includeinclude dir already covers. Added themissing dir explicitly so it doesn't depend on the compiler package re-exporting runtime
headers.
Verification (real, on this exact branch, against the newer SDK)
build-fpgacmake --buildsucceeds, 100%. Real aarch64 ELF binaries produced (llama-cli,simple-backend-tsi).build-posixcmake --buildsucceeds, 100%.showing nonzero
TSI_KERNEL-RUNcounts forADD,MUL,RMS_NORM,MUL_MAT,SOFT_MAX,GLU— i.e. actual inference dispatched through the TSI OPU kernels, notjust a clean link.
Sequence used to get here (documented for whoever picks this up next): reproduced the
exact
mlir_translatefailure first on an unmodified base against this SDK; confirmed itdisappears with fix #1 alone; hit
HostShimCAPI.hnext; root-caused and fixed with #2;re-verified both targets build AND run clean from a fully fresh kernel/blob build (a stale
partial blob-build state from an earlier attempt had briefly masked an unrelated local
config issue on the verifier's own environment — not a code issue).
Scope / sequencing
ggml-tsi-kernelfix landing first. Left as draft until thatmerges — the submodule commit pinned here is that fix's current branch-tip commit, and
may need re-pointing once it merges (e.g. if squash-merged rather than fast-forwarded).
llama.cppat afixed release tag rather than tracking
main, so they won't see this until a newrelease tag is cut that includes it and their own pin is bumped — their normal release
process. Other repos vendor
llama.cppvia a separate sync mechanism and will needtheir own vendor-pin bump once this lands, following the established pattern for prior
vendor-pin-sync changes. Flagging both as the next real steps, not doing them in this PR.