[Bug] Mode B (kernelType=AICPU_CUSTOM): cust_aicpu_sd subprocess cache stale on AICore HBM writes → AICPU handshake deadlock (507018)

Platform

a2a3 (Ascend 910B/C hardware)

Runtime Variant

tensormap_and_ringbuffer

Description

PR #537 migrates the AICPU dispatcher from CANN 6.x RuntimeAicpuKernelLaunchExWithArgs (path A) to CANN 7.0+ rtsBinaryLoadFromFile + rtsLaunchCpuKernel (path B). The motivation is to enable a single host process binding both host_build_graph and tensormap_and_ringbuffer runtimes — path A is blocked by a process-wide one-shot firstCreatSo_ latch inside CANN preinstalled libaicpu_processer.so::BackendServerHandleManager::SaveSoFile, which makes loading a second runtime's inner SO a silent no-op.

In path B, the JSON descriptor uses opKernelLib=AICPUKernel + userDefined="True", which CANN routes to KERNEL_TYPE_AICPU_CUSTOM (4) (cann/runtime/src/runtime/core/src/kernel/program_common.cc). Per ae_so_manager.cc::GetSoPath, KERNEL_TYPE_AICPU_CUSTOM is the only path that searches /home/CustAiCpuUser/cust_aicpu_<dev>_<vf>_<pid>/ (where our uploaded SO actually lands); all other types search /usr/lib64/aicpu_kernels/... which is root-owned and unwritable from a user process. A gate at ae_so_manager.cc:514 (IsCustAicpuSd()) also enforces that KERNEL_TYPE_AICPU_CUSTOM MUST execute inside the aicpu_custom_scheduler subprocess.

Everything routes through correctly: CANN dispatches our Dyn* exports to the cust subprocess, our libsimpler_aicpu_<runtime>.so is dlopen'd, three phases (Null/Init/Run) all reach our code, and SchedulerContext::handshake_all_cores step 1 writes complete to all 9 cores' Handshake slots in shared HBM. AICPU writes are visible to host (verified via aclrtMemcpy DEVICE_TO_HOST readback). AICore stream dispatches, runs past its phase 1, writes aicore_regs_ready=1 back to HBM (also confirmed via host D2H).

The bug: the cust AICPU's L1 cache holds a stale 0 for the aicore_regs_ready field, even though HBM and host both see 1. SchedulerContext::handshake_all_cores step 2 spin loop never observes the update:

// src/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_cold_path.cpp
while (hank->aicore_regs_ready == 0) {}   // ← cust AICPU stuck here, HBM has 1

After 2 s the host aclrtSynchronizeStreamWithTimeout(stream_aicpu_) reports ACL_ERROR_RT_AICPU_EXCEPTION (507018). Mode A (path A) does not exhibit the bug because the main aicpu_scheduler shares a cache coherency domain with AICore; the cust subprocess gets bound (SetAffinity) to a different AICPU cluster whose L1 is not snooped by AICore HBM writes.

User-space workarounds attempted (all fail):

Attempt	Result
`volatile uint32_t` field qualifier	No effect — prevents register reuse, not L1 cache
`__atomic_load_n(..., __ATOMIC_ACQUIRE)` (→ `ldar`)	No effect — only an ordering instruction, still reads L1
`dc civac` (clean + invalidate) in spin loop	Worse — same cache line co-hosts AICPU-written `aicpu_ready/task` and AICore-written `aicore_regs_ready`; civac writes back AICPU's dirty stale view, clobbering AICore's HBM writes
`dc ivac` (invalidate-only) in spin loop	Silently NOP'd from EL0 (SCTLR_EL1.UCI=0 in kernel)

Steps to Reproduce

Apply the PR Feat: AICPU launch via dispatcher bootstrap and per-task rtsLaunchCpuKernel #537 mode B refactor (single SO libsimpler_aicpu_<runtime>.so with merged outer dispatcher + inner runtime kernels; JSON opKernelLib=AICPUKernel + userDefined=True for all three phases — Null/Init/Run; orch SO candidate_dirs[] adds /home/CustAiCpuUser as first entry)

Run the simplest a2a3 onboard example:

python3 -m venv --system-site-packages .venv
source .venv/bin/activate
pip install --no-build-isolation -e .
python examples/a2a3/tensormap_and_ringbuffer/vector_example/test_vector_example.py -p a2a3

Test fails with timeout. Optional D2H diagnostic in device_runner.cpp::run after the timeout shows HBM has the right values:

if (rc == ACL_ERROR_RT_STREAM_SYNC_TIMEOUT || rc == 507018) {
    Handshake h0 = {};
    aclrtMemcpy(&h0, sizeof(h0),
                reinterpret_cast<const uint8_t*>(kernel_args_.args.runtime_args) + offsetof(Runtime, workers),
                sizeof(h0), ACL_MEMCPY_DEVICE_TO_HOST);
    LOG_ERROR("workers[0] readback: aicpu_ready=%u task=0x%lx aicore_regs_ready=%u",
              h0.aicpu_ready, (uint64_t)h0.task, h0.aicore_regs_ready);
}

Output: workers[0] readback: aicpu_ready=1 task=0x... aicore_regs_ready=1 — both handshake bits set in HBM, both visible to host, but cust AICPU never sees the AICore-written aicore_regs_ready=1.

Expected Behavior

tensormap_and_ringbuffer vector example passes end-to-end with kernelType=AICPU_CUSTOM (4) routing. Multi-runtime in a single host process becomes possible (one ChipWorker process binds both host_build_graph and tensormap_and_ringbuffer), which is the entire motivation for path B over path A.

Actual Behavior

=== Runtime: tensormap_and_ringbuffer  Level: 2 ===
  TestVectorExample::default ... [ERROR] run: [device_runner.cpp:797] aclrtSynchronizeStreamWithTimeout (AICPU) failed: 507018
FAILED: run_prepared failed with code 507018

Device log shows our dispatcher and inner kernel running cleanly in the cust subprocess; the deadlock is purely the AICPU-side read-side cache miss on AICore-written HBM data. The bug is not in user-space code and cannot be fixed there: the four standard AArch64 cache-bypass primitives all fail as documented above.

Possible fix directions

(Listed by where the change has to live — none can be done purely in this repo's user-space.)

#	Where	Change
A	CANN device kernel / driver	Enable EL0 `dc ivac` (set `SCTLR_EL1.UCI=1` for the `aicpu_custom_scheduler` process). Smallest change, user-side spin loops can then explicitly invalidate.
B	CANN runtime / driver	Allocate `Handshake` HBM (`runtime->workers`) with non-cacheable / write-through attribute when called from `aicpu_custom_scheduler` context. Slight per-access HBM latency cost.
C	CANN cust scheduler	Pin `aicpu_custom_scheduler` worker threads to the same AICPU cluster as AICore's snoop domain (today `aicpusd_worker.cpp::SetAffinity` binds them to a separate `cpuId=0`).
D	simpler runtime (this repo)	Split `Handshake` so AICPU-written and AICore-written fields live on disjoint cache lines, then make AICPU spin loops `dc civac` only the AICore-written line — but EL0 invalidate semantics still apply, so this only works in combination with A or B. Alternatively, replace the spin-wait protocol with a device event/notify primitive that bypasses shared-memory polling entirely (substantial runtime refactor).

A/B/C are CANN-side; D is user-side but on its own is insufficient. We'd appreciate guidance from the CANN team on whether A or B is feasible for the cust subprocess in a near-term release.

Git Commit ID

ec7363a2eb6fbed4d71f848e1532dbcef7adc6c8

CANN Version

26.0.rc1 (V100R001C10SPC001B257)

Driver Version

26.0.rc1 (ascendhal 7.35.23)

Host Platform

Linux (aarch64)

Additional Context

Related open issues (different surface, may share root once A/B/C lands):

rtStreamSynchronize (AICPU) fails with error 507018 in TensorMap and RingBuffer runtime #84 — 507018 in tensormap_and_ringbuffer (different reproducer)
[sim] dcci lacks cache-line atomicity in simulation mode — handshake store reordering can cause deadlock #266 — cache coherency in handshake on sim
[Bug] Handshake failure on Ascend910B3 during core initialization #480 — handshake failure on 910B3
Multi-cid dispatch broken: two distinct ChipCallables on one chip child either fire wrong kernel or stream-timeout #759 — stream timeout on multi-cid dispatch

Concrete D2H diagnostic + per-iteration markers + cust subprocess routing analysis lived in the long debug session that produced this issue; the architecture diagram and CANN source pointers are in .docs/ISSUE-mode-b-cache-coherency.md of the PR #537 worktree. CANN open-source references:

cann/runtime/src/runtime/core/src/kernel/program_common.cc — opKernelLib → kernelType table
cann/runtime/src/aicpu_sched/aicpu_processer/ae_so_manager.cc — GetSoPath, LoadSo cust-vs-inner routing and the IsCustAicpuSd gate
cann/runtime/src/aicpu_sched/aicpu_schedule/core/aicpusd_worker.cpp — SetAffinity binding worker threads to specific AICPU cores
cann/runtime/src/aicpu_sched/aicpu_schedule/core/aicpusd_cust_so_manager.cpp — cust SO upload to /home/CustAiCpuUser/cust_aicpu_*/

Investigation update — full attempt matrix and working solution

Continued investigation reframes the original diagnosis. The 507018 symptom is real, but the
root cause is not AICPU-side L1 staleness on AICore HBM writes. The cust_aicpu_sd
subprocess (kernelType = AICPU_CUSTOM (4) or CUSTOM_KFC (6)) is architecturally
sandboxed from AICore — it cannot reach the AICore notify/mailbox primitives that the
handshake protocol depends on, regardless of cache attributes on the shared HBM struct.

Stepping back, the host-side knobs that determine routing are three:

cpuKernelMode ∈ {0, 1, 2}: JSON-only registration / JSON+SO upload / LoadFromData
userDefined ∈ {True, False}: cust subprocess vs main aicpu_scheduler
opKernelLib ∈ {AICPUKernel, CUSTAICPUKernel, KFCKernel, ...}: opaque label hashed
by aicpu::GetAicpuKernelType into one of the four kernelType enum values

CANN's four AICPU kernelType values (pkg_inc/runtime/runtime/kernel.h):

`kernelType`	Enum	Process	Dispatcher ABI	SO discovery path	Can talk to AICore
2	`KERNEL_TYPE_AICPU`	main `aicpu_scheduler`	plain	`/usr/lib64/aicpu_kernels/<vfId>/aicpu_kernels_device/`	yes
4	`KERNEL_TYPE_AICPU_CUSTOM`	`cust_aicpu_sd` subprocess	plain	`/home/CustAiCpuUser/cust_aicpu_*/`	no
5	`KERNEL_TYPE_AICPU_KFC`	main `aicpu_scheduler`	KFC (framework core)	same as type 2	yes
6	`KERNEL_TYPE_CUSTOM_KFC`	`cust_aicpu_sd` subprocess	KFC	same as type 4	no

(Path strings confirmed in libtsdclient.so::tsd::TsdPathMgr::BuildKernelSoPath and
tsd::TsdPathMgr::BuildCustAiCpuUserPath; the cust flag is selected by JSON userDefined
and stored in libaicpu_engine_common.so::_cust_aicpu_flag.)

Full attempt matrix

#	Approach	`opKernelLib`	`userDefined`	`cpuKernelMode`	API	Effective `kernelType`	Result	Why
1	Original PR #537 mode B	`AICPUKernel`	`True`	1	`rtsBinaryLoadFromFile` + `rtsLaunchCpuKernel`	4 (AICPU_CUSTOM)	❌ 507018	Cust subprocess cannot reach AICore (originally diagnosed as L1 cache staleness; actual mechanism is the sandbox cutting off the AICore notify path)
2	Same as #1, with `npu-smi set -t custom-op-secverify-mode=0`	`AICPUKernel`	`True`	1	same	4	❌ 507018	npu-smi flag only disables signature verification on unsigned tar.gz — doesn't change routing
3	Mode B, switch to main scheduler with runtime upload	`AICPUKernel`	`False`	1	same	2 (AICPU)	❌ 507018 (kernel never runs)	`cpuKernelMode=1` uploads SO to `/home/CustAiCpuUser/...`, but `userDefined=False` routes to main scheduler which looks in `/usr/lib64/aicpu_kernels/<vfId>/aicpu_kernels_device/` — upload lands in the wrong namespace
4	Direction 1 (current working)	`AICPUKernel`	`False`	0	same	2 (AICPU)	✅ PASS	Pre-deploy `aicpu_simpler.tar.gz` to `\$ASCEND_HOME_PATH/opp/built-in/op_impl/aicpu/kernel/` + ini whitelist entry. Driver extracts to `/usr/lib64/aicpu_kernels/<vfId>/aicpu_kernels_device/` at process startup. `userDefined=False` routes to main scheduler which finds the pre-deployed SO. Multi-runtime works because the dispatcher caches per-runtime inner SOs by `DeviceArgs.aicpu_so_bin` (distinct rtMalloc'd device addresses per ChipWorker → no global one-shot latch).
5	LoadFromData / bisheng-style	(no JSON)	(no JSON)	2	`rtsBinaryLoadFromData` + `rtsRegisterCpuFunc`	? (no JSON to read)	❌ 507018 without magic; ❌ `107000` with magic	CANN validates the binary has `RT_DEV_BINARY_MAGIC_ELF_AICPU = 0x41415243` ("AARC") baked into a specific offset. Only bisheng-compiled `[aicpu]` SOs have it — plain `g++ -shared` doesn't produce that magic.
6	LoadFromData with declared magic	(no JSON)	(no JSON)	2 + `RT_LOAD_BINARY_OPT_MAGIC = 0x41415243`	same	--	❌ `107000` at `rtsBinaryLoadFromData`	Confirms CANN reads magic from the binary itself, not from the option; the option is for cross-check

Cache-bypass diagnostics from the original report (volatile, __atomic_load_n /
ldar, dc civac, dc ivac) are dropped in retrospect — they were chasing a misdiagnosed
mechanism and all failed because the actual blocker is the cust subprocess's process-level
isolation, not cache attributes.

What we landed on

642ac727 — "Feat:
Direction 1 aicpu dispatcher — stable SO + per-runtime kernel upload". Configuration:

opKernelLib = AICPUKernel, userDefined = False, cpuKernelMode = 0 →
kernelType = 2 → main aicpu_scheduler
One stable libsimpler_aicpu_dispatcher.so (no per-runtime suffix, no inner kernel
sources). Dispatcher dlopens per-runtime inner SOs at runtime from
DeviceArgs.aicpu_so_bin (host-uploaded bytes), keyed by device address — distinct
ChipWorkers get distinct cache entries, removing the single-process multi-runtime block.

One-time host setup needed once per machine:

sudo cp build/lib/a2a3/onboard/aicpu_simpler.tar.gz \
  \$ASCEND_HOME_PATH/opp/built-in/op_impl/aicpu/kernel/

sudo tee -a \$ASCEND_HOME_PATH/aarch64-linux/conf/ascend_package_load.ini <<'INI'

name:aicpu_simpler.tar.gz
install_path:2
optional:true
package_path:opp/built-in/op_impl/aicpu/kernel
load_as_per_soc:false
INI

sudo npu-smi set -t custom-op-secverify-mode=0   # unsigned tar.gz

End-to-end examples/a2a3/tensormap_and_ringbuffer/vector_example and the host_build_graph
counterpart both pass; multi-runtime in a single host process is unblocked.

Suggested CANN-side improvements that would let us drop the manual setup

Independent of whether the original A/B/C fix directions for the cust-subprocess case are
pursued, these would simplify the deployment story for users in our position:

A cpuKernelMode that uploads at runtime but lands in the main scheduler's
/usr/lib64/aicpu_kernels/<vfId>/aicpu_kernels_device/ namespace (instead of the cust
/home/CustAiCpuUser/ namespace). Today there is no such mode — runtime upload implies
cust subprocess routing.
Or: document transformer_tile_fwk_aicpu_kernel.tar.gz (referenced inside
libtsdclient.so and routed to aicpu_kernels_device/tile_fwk_machine/) as a public
extension point for tile-framework AICPU kernels, with a stable contract for what goes
in that tar.gz.

补充实验 #7 — CUSTKFCKernel + cpuKernelMode=1

pypto 的参考实现 (framework/src/machine/runtime/load_aicpu_op.cpp::CustomAiCpuSoLoad) 用
opKernelLib="CUSTKFCKernel" + userDefined="False" + cpuKernelMode=1 处理"运行时上传的
自定义 op"，希望复用这条路绕开 tar.gz 预部署。

实测分支：pr-537-custkfc-experiment（从 pr-537-work 拉出，仅改 load_aicpu_op.cpp）。

	配置	结果
#7	`opKernelLib=CUSTKFCKernel`（三相全部）`userDefined=False`，`cpuKernelMode=1`	❌ AICore 超时 `507046`

device plog 关键行：

ConstructAICpuSqeForDavinciTask: kernel_type=6   ← Null/Init/Run 三相都被识别为 type 6
Force cpy args ...                                ← runtime upload 成功，无 "open so failed"
AICore kernel execute failed, task timeout
errorStr: timeout or trap error                   ← AICore 在等 AICPU 的握手信号

含义：CUSTKFCKernel 在 CANN 9.0 里仍然映射到 KERNEL_TYPE_CUSTOM_KFC = 6，还是走
cust_aicpu_sd 子进程。"CUST" 不是上传机制的标记，而是 kernelType 路由的标记，与
userDefined 字段无关。SO 上传成功、AICPU 入口跑起来了，但 cust 子进程依旧无法跟
AICore 完成握手 —— 跟 #1 (opKernelLib=AICPUKernel + userDefined=True) 失败模式一致。

pypto 用 CUSTKFCKernel 跑得通的 control flow ops 大概率不依赖跟 AICore 的实时握手
（它们是 framework 上层的控制逻辑，不是 AICPU↔AICore 同步原语的使用方）。

更新后的结论矩阵

`opKernelLib`	进程	SO 来源	能跟 AICore 通信
`AICPUKernel` (type 2)	主 `aicpu_scheduler`	tar.gz 预部署到 `/usr/lib64/aicpu_kernels/<vfId>/aicpu_kernels_device/`	✅
`CUSTAICPUKernel` (type 4)	`cust_aicpu_sd` 子进程	API 上传到 `/home/CustAiCpuUser/`	❌
`KFCKernel` (type 5)	主 `aicpu_scheduler`	同 type 2	✅
`CUSTKFCKernel` (type 6)	`cust_aicpu_sd` 子进程	API 上传到 `/home/CustAiCpuUser/`	❌

所有"运行时通过 API 上传 SO"的路径（type 4 / type 6）都强制路由到 cust 子进程。
要在主调度器跑，必须经过 tar.gz 预部署机制（type 2 / type 5）。

Direction 1 (AICPUKernel + userDefined=False + cpuKernelMode=0 + tar.gz) 依然是
唯一同时满足"单进程多 runtime"和"AICore 握手"两条要求的路径。

给 CANN 团队的建议（更具体）

CANN 9.0 的路由表把"运行时上传"和"主调度器进程"做成了互斥两个维度。如果能放开这个
限制（例如新增 KERNEL_TYPE_AICPU_KFC 的 cpuKernelMode=1 路径，让 SO 上传到
主调度器的命名空间），框架侧需要 tar.gz 预部署的 ops 都能省掉。具体落点：

在 aicpu::GetAicpuKernelType 的 opKernelLib→kernelType 映射里增加一项：
opKernelLib="KFCKernel" + cpuKernelMode=1 → kernelType=5 + 主调度器
upload path（与 type 6 上传到 cust 命名空间区分）。
或：保留现有的 userDefined 字段做进程选择，但允许 userDefined="False" + cpuKernelMode=1 时 SO 上传到主调度器命名空间（目前这个组合下 SO 仍上传到
/home/CustAiCpuUser/，主调度器找不到）。

Resolution: zero-deploy bootstrap via `libaicpu_extend_kernels.so`

The "Direction 1" tar.gz approach worked but required three manual sudo
steps per host (cp tar to opp, edit ascend_package_load.ini, npu-smi set
custom-op-secverify-mode=0). Final design replaces all of that with an
in-process bootstrap. Single commit 40eb9c7d ("Feat: zero-deploy AICPU
dispatcher — bootstrap via libaicpu_extend_kernels") replaces the prior
Direction 1 commit on pr-537-work.

Key insight

The dispatcher SO runs in the main aicpu_scheduler's sched thread
(HwHiAiUser), which has write permission on
/usr/lib64/aicpu_kernels/0/aicpu_kernels_device/. So the dispatcher
can deploy itself into the main scheduler's preinstall path — no
host-side packaging or sudo needed.

Mechanism (verified end-to-end in device log)

Host LoadAicpuOp::BootstrapDispatcher issues one Mode A call:

rtAicpuKernelLaunchExWithArgs(
  kernelType  = KERNEL_TYPE_AICPU_KFC,
  opName      = "AST_DYN_AICPU",
  soName      = "libaicpu_extend_kernels.so",  // CANN-preinstalled
  kernelName  = "DynTileFwkKernelServerInit",
  args.cfgdata.aicpu_so_bin = device-uploaded dispatcher bytes)

libaicpu_extend_kernels.so::SaveSoFile + SetTileFwkKernelMap + ExecuteFunc forward into our dispatcher's DynTileFwkBackendKernelServerInit.
Our Init detects "I'm being uploaded with myself as payload" by SO size
(RTLD_DEEPBIND makes pointer-identity self-checks unreliable across
independent dlopen instances), then copies the bytes into the main
scheduler's preinstall path:
/usr/lib64/aicpu_kernels/0/aicpu_kernels_device/libsimpler_aicpu_dispatcher.so.
Subsequent Mode B rtsBinaryLoadFromFile with
kernelSo="libsimpler_aicpu_dispatcher.so" resolves the dispatcher
from the same preinstall path. Three exports map to device_runner's
three phases:
- DynTileFwkBackendKernelServerNull — preload inner SO (no invoke)
- DynTileFwkBackendKernelServerInit — invoke inner Init
- DynTileFwkBackendKernelServer — invoke inner Server
Dispatcher caches inner SO handles by aicpu_so_bin device address —
distinct ChipWorker instances get distinct rtMalloc'd addresses,
hence independent cache entries, sidestepping
libaicpu_extend_kernels's firstCreatSo_ one-shot latch.

Necessary dispatcher exports

libaicpu_extend_kernels::SetTileFwkKernelMap dlsyms a fixed set of
symbols on the uploaded SO and dlcloses if any is missing:
StaticTileFwkBackendKernelServer, DynTileFwkBackendKernelServerInit,
DynTileFwkBackendKernelServer. We export all three (Static is a stub)
plus one extra DynTileFwkBackendKernelServerNull used only by Mode B's
Null phase.

One subtle bug found along the way

device_runner's three-phase pattern is Null → Init → Run with the
contract "Null preloads, Init invokes inner Init, Run invokes inner
Server". An earlier draft mapped Null to Server (Exec), which made Run
execute before Init — leaving the kernel in an inconsistent state and
triggering a 50 s spin-wait timeout (looked like the AICore handshake
issue this issue originally tracked). Dedicated DynTileFwkBackendKernelServerNull
entry fixes that cleanly.

Test result

tensormap_and_ringbuffer/vector_example on a2a3 onboard: 8/8 PASSED
end-to-end with zero host-side deploy (no tar.gz, no sudo, no
npu-smi set).

Closing

The original L1-cache-staleness diagnosis in this issue was incorrect —
the cust subprocess path is architecturally cut off from AICore notify
primitives, not blocked by cache attributes. The actual fix is to route
through the main aicpu_scheduler (this commit). All other attempts
documented earlier in this issue (CUSTKFCKernel, LoadFromData, cust
subprocess) confirmed the only viable path is the main scheduler — but
without requiring tar.gz pre-deployment.

Closing this issue.

Resolved by pr-537-work commit 40eb9c7d (zero-deploy bootstrap via libaicpu_extend_kernels). See the comment above for the final architecture.

Feat: migrate AICPU launch to rtsLaunchCpuKernel + zero-deploy dispat…

Add: cann-examples/aicpu-kernel-launch reference tool + mechanisms doc

Add: cann-examples/aicpu-kernel-launch reference tool + mechanisms doc (

Navigation Menu

Repository navigation

[Bug] Mode B (kernelType=AICPU_CUSTOM): cust_aicpu_sd subprocess cache stale on AICore HBM writes → AICPU handshake deadlock (507018) #822

Platform

Runtime Variant

Description

Steps to Reproduce

Expected Behavior

Actual Behavior

Possible fix directions

Git Commit ID

CANN Version

Driver Version

Host Platform

Additional Context

Investigation update — full attempt matrix and working solution

Full attempt matrix

What we landed on

Suggested CANN-side improvements that would let us drop the manual setup

补充实验 #7 — CUSTKFCKernel + cpuKernelMode=1

更新后的结论矩阵

给 CANN 团队的建议（更具体）

Resolution: zero-deploy bootstrap via `libaicpu_extend_kernels.so`

Key insight

Mechanism (verified end-to-end in device log)

Necessary dispatcher exports

One subtle bug found along the way

Test result

Closing

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[Bug] Mode B (kernelType=AICPU_CUSTOM): cust_aicpu_sd subprocess cache stale on AICore HBM writes → AICPU handshake deadlock (507018) #822

Description

Platform

Runtime Variant

Description

Steps to Reproduce

Expected Behavior

Actual Behavior

Possible fix directions

Git Commit ID

CANN Version

Driver Version

Host Platform

Additional Context

Activity

hw-native-sys-bot commented on May 20, 2026

Investigation update — full attempt matrix and working solution

Full attempt matrix

What we landed on

Suggested CANN-side improvements that would let us drop the manual setup

hw-native-sys-bot commented on May 20, 2026

补充实验 #7 — CUSTKFCKernel + cpuKernelMode=1

更新后的结论矩阵

给 CANN 团队的建议（更具体）

hw-native-sys-bot commented on May 20, 2026

Resolution: zero-deploy bootstrap via libaicpu_extend_kernels.so

Key insight

Mechanism (verified end-to-end in device log)

Necessary dispatcher exports

One subtle bug found along the way

Test result

Closing

hw-native-sys-bot commented on May 20, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Resolution: zero-deploy bootstrap via `libaicpu_extend_kernels.so`