You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
XRTHostRuntime: host BOs created with group_id=0 — amdxdna kernels silently ignore them, large GEMMs wedge the NPU #174
On amdxdna each host BO is bound to its kernel argument slot via the BO group: opcode=0, instr=1, ninstr=2, host buffers from slot 3 on. XRTTensor creates every BO with group_id=0 (default), while kernels declare data-BO groups like 0x10000/0x20000 (mlir-aie's own C++ tests create BOs with kernel.group_id(3 + arg_index)).
Consequences observed with the stock runtime:
Kernels whose data-BO group is non-zero silently ignore group-0 BOs: the ERT command reports ERT_CMD_STATE_COMPLETED but nothing is written (stale/zero outputs, no error).
Large shapes are worse: an INT8 2048³ GEMM with group-0 BOs wedges the whole NPU — dmesg shows IO_PAGE_FAULT + aie2_set_cmd_timeout; every subsequent dispatch returns stale data until a PCI remove/rescan. This retroactively explains the long-standing "GEMM wrong at 512³/1024³, ok at 1024×512×1024" reports.
Caveat found while testing a blanket fix: some IRON ops (Llama prefill) declare invalid-large groups (0x90000..0xD0000) that must NOT be used for BO creation (garbage results), but those kernels work fine with group-0 BOs. A robust fix must only rebind data BOs when kernel.group_id(3+i) is a valid host group (0x10000/0x20000) — that refined patch makes INT8 2048³/4096³ GEMMs exact and keeps Llama working.
Workaround shipped in the live venv runtime (not upstream): host BOs are bound to the kernel's declared group (rebind with device→host sync first, see #3655 fix). Full upstream fix still needed — this issue tracks it.
Activity
bong-water-water-bong commentedon Aug 31, 2026
Workaround shipped in the live venv runtime (not upstream): host BOs are bound to the kernel's declared group (rebind with device→host sync first, see #3655 fix). Full upstream fix still needed — this issue tracks it.