Repository navigation
Fix: raise timeout defaults and keep CI fast-fail - #1175
Merged
Merged
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR raises timeout defaults for a2a3 and a5, sets CI environment overrides back to tight values, updates timeout-focused tests, and refreshes related documentation and operational notes. ChangesTimeout defaults, CI overrides, and docs
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request increases the default runtime timeouts (scheduler, op-execute, and stream-sync) to more production-friendly values across onboard and sim platforms, while updating tests, documentation, and CI configurations to support tighter environment-based overrides. Feedback on these changes suggests using ${CMAKE_CURRENT_SOURCE_DIR} instead of ${CMAKE_SOURCE_DIR} in tests/ut/cpp/CMakeLists.txt to avoid potential path resolution issues if the subdirectory is included in a larger build.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/ci.yml (1)
444-447: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winAdd the PTO2 timeout overrides to the UT hardware jobs
st-onboard-a2a3already has the tightPTO2_*env block, but bothut-a2a3andut-a5still run without it. Add the same overrides to each UT job so the hardware tests use the intended fail-fast timeouts instead of the raised defaults.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci.yml around lines 444 - 447, The UT hardware jobs are missing the PTO2 timeout override environment block, so add the same tight PTO2_* env settings used by st-onboard-a2a3 to both ut-a2a3 and ut-a5 in the CI workflow. Update the job definitions for ut-a2a3 and ut-a5 so their hardware tests use the intended fail-fast timeouts instead of the raised defaults.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/ut/cpp/CMakeLists.txt`:
- Around line 443-445: The test_runtime_timeout_config unit test only includes
the a2a3 platform headers, so it misses coverage for the a5 platform defaults.
Update the test setup around test_runtime_timeout_config in the CMake/test code
to add an a5-specific include path or a separate a5 variant that uses
src/a5/platform/include/common/platform_config.h, so both platform_config.h
implementations are exercised.
---
Outside diff comments:
In @.github/workflows/ci.yml:
- Around line 444-447: The UT hardware jobs are missing the PTO2 timeout
override environment block, so add the same tight PTO2_* env settings used by
st-onboard-a2a3 to both ut-a2a3 and ut-a5 in the CI workflow. Update the job
definitions for ut-a2a3 and ut-a5 so their hardware tests use the intended
fail-fast timeouts instead of the raised defaults.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 14eeeb05-b783-4c9f-98b7-5859626fd3d7
📒 Files selected for processing (15)
.claude/rules/running-onboard.md.github/workflows/ci.ymldocs/ci.mddocs/dfx/args-dump.mddocs/troubleshooting/local-timeout-defaults.mdsrc/a2a3/platform/include/common/platform_config.hsrc/a2a3/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.hsrc/a5/platform/include/common/platform_config.hsrc/a5/runtime/tensormap_and_ringbuffer/runtime/scheduler/scheduler_types.hsrc/common/platform/onboard/aicpu/spin_hint.hsrc/common/platform/sim/aicpu/spin_hint.htests/st/aicore_op_timeout/kernels/aic/kernel_hang.cpptests/st/aicore_op_timeout/test_aicore_op_timeout.pytests/ut/cpp/CMakeLists.txttests/ut/cpp/common/test_runtime_timeout_config.cpp
sunkaixuan2018
force-pushed
the
fix-issue-1165
branch
5 times, most recently
from
June 27, 2026 08:18
613112f to
c04b819
Compare
sunkaixuan2018
changed the title
Collaborator
|
Two review notes (otherwise this LGTM — faithfully implements #1165, keeps a2a3/a5 in parity, and the ST Should fix —
|
- Raise a2a3 and a5 onboard timeout defaults for production workloads - Restore CI fast-fail values with PTO2 timeout env overrides - Keep host timeout ordering validation aligned with scheduler overrides - Update docs and timeout config tests for the new defaults
sunkaixuan2018
force-pushed
the
fix-issue-1165
branch
from
June 27, 2026 09:36
c04b819 to
959eb6b
Compare
ChaoZheng109
approved these changes
Jun 27, 2026
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jun 27, 2026
Split the cross-repo qwen3 work into two focused skills: - multi-repo-setup becomes the repo-relationship + per-repo setup/run skill: documents the import stack (pto-isa -> simpler -> pypto -> pypto-lib -> pypto-serving), clones/installs each layer, and hands off to each external repo's own .claude/skills/ + README rather than duplicating their run steps. Adds pypto-serving as a first-class repo. - multi-repo-qwen-serving-setup becomes the concrete qwen3 guide: defers clone/install to multi-repo-setup, then gives the batch-1 / batch-16 run commands, the prefill ring-sizing + op-timeout gotchas behind a generic 507018, how to read the per-token timing layers, and the kernel-level decode_layer scheduler-overhead DFX flow. Corrects the timeout facts to the post-hw-native-sys#1175 compiled defaults (op_execute 45s / stream_sync 50s / scheduler 10s, not the old 3s/4s/2s): a clean ~16.7s batch-16 prefill no longer trips the op-timeout, so the PTO2_OP_EXECUTE_TIMEOUT_US / PTO2_STREAM_SYNC_TIMEOUT_MS env overrides are only needed on a contended box, and breaking the scheduler < op_execute < stream_sync ordering drops the whole override set. Marks the [STRACE] host-trace markers / strace_timing.py / docs/dfx/host-trace.md as pending simpler hw-native-sys#1177 (still OPEN, not landed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3 tasks done
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jun 27, 2026
Split the cross-repo qwen3 work into two focused skills: - multi-repo-setup becomes the repo-relationship + per-repo setup/run skill: documents the import stack (pto-isa -> simpler -> pypto -> pypto-lib -> pypto-serving), clones/installs each layer, and hands off to each external repo's own .claude/skills/ + README rather than duplicating their run steps. Adds pypto-serving as a first-class repo. - multi-repo-qwen-setup is the concrete qwen3 guide, covering BOTH the pypto-serving runner (batch-1 / batch-16 prefill/decode TPOT) and the pypto-lib decode_layer kernel (scheduler-overhead DFX). It defers clone/install to multi-repo-setup, then gives the run commands, the prefill ring-sizing + op-timeout gotchas behind a generic 507018, how to read the per-token timing layers, and the overhead-analysis flow. (Named multi-repo-qwen-setup, not -serving-setup, since it spans the serving runner and the kernel path, not serving alone.) Corrects the timeout facts to the post-hw-native-sys#1175 compiled defaults (op_execute 45s / stream_sync 50s / scheduler 10s, not the old 3s/4s/2s): a clean ~16.7s batch-16 prefill no longer trips the op-timeout, so the PTO2_OP_EXECUTE_TIMEOUT_US / PTO2_STREAM_SYNC_TIMEOUT_MS env overrides are only needed on a contended box, and breaking the scheduler < op_execute < stream_sync ordering drops the whole override set. Marks the [STRACE] host-trace markers / strace_timing.py / docs/dfx/host-trace.md as pending simpler hw-native-sys#1177 (still OPEN, not landed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jun 28, 2026
Split the cross-repo qwen3 work into two focused skills: - multi-repo-setup becomes the repo-relationship + per-repo setup/run skill: documents the import stack (pto-isa -> simpler -> pypto -> pypto-lib -> pypto-serving), clones/installs each layer, and hands off to each external repo's own .claude/skills/ + README rather than duplicating their run steps. Adds pypto-serving as a first-class repo. - multi-repo-qwen-setup is the concrete qwen3 guide, covering BOTH the pypto-serving runner (batch-1 / batch-16 prefill/decode TPOT) and the pypto-lib decode_layer kernel (scheduler-overhead DFX). It defers clone/install to multi-repo-setup, then gives the run commands, the prefill ring-sizing + op-timeout gotchas behind a generic 507018, how to read the per-token timing layers, and the overhead-analysis flow. (Named multi-repo-qwen-setup, not -serving-setup, since it spans the serving runner and the kernel path, not serving alone.) Corrects the timeout facts to the post-hw-native-sys#1175 compiled defaults (op_execute 45s / stream_sync 50s / scheduler 10s, not the old 3s/4s/2s): a clean ~16.7s batch-16 prefill no longer trips the op-timeout, so the PTO2_OP_EXECUTE_TIMEOUT_US / PTO2_STREAM_SYNC_TIMEOUT_MS env overrides are only needed on a contended box, and breaking the scheduler < op_execute < stream_sync ordering drops the whole override set. Marks the [STRACE] host-trace markers / strace_timing.py / docs/dfx/host-trace.md as pending simpler hw-native-sys#1177 (still OPEN, not landed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jun 28, 2026
Split the cross-repo qwen3 work into two focused skills: - multi-repo-setup becomes the repo-relationship + per-repo setup/run skill: documents the import stack (pto-isa -> simpler -> pypto -> pypto-lib -> pypto-serving), clones/installs each layer, and hands off to each external repo's own .claude/skills/ + README rather than duplicating their run steps. Adds pypto-serving as a first-class repo. - multi-repo-qwen-setup is the concrete qwen3 guide, covering BOTH the pypto-serving runner (batch-1 / batch-16 prefill/decode TPOT) and the pypto-lib decode_layer kernel (scheduler-overhead DFX), framed up front as two distinct paths (different repos, entry points, what they measure). It defers clone/install to multi-repo-setup, then gives the run commands, the prefill ring-sizing + op-timeout gotchas behind a generic 507018, how to read the per-token timing layers, and the overhead-analysis flow. The timeout facts use the post-hw-native-sys#1175 compiled defaults (op_execute 45s / stream_sync 50s / scheduler 10s): a clean ~16.7s batch-16 prefill no longer trips the op-timeout, so the PTO2_OP_EXECUTE_TIMEOUT_US / PTO2_STREAM_SYNC_TIMEOUT_MS env overrides are only needed on a contended box, and breaking the scheduler < op_execute < stream_sync ordering drops the whole override set. The per-token timing section uses the [STRACE] host-trace markers + strace_timing.py / docs/dfx/host-trace.md landed in simpler hw-native-sys#1177. The §7 host-instrumentation rebuild uses an $ARCH variable and a find-based copy (python-version-agnostic), matching multi-repo-setup Step 3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jun 28, 2026
Split the cross-repo qwen3 work into two focused skills: - multi-repo-setup becomes the repo-relationship + per-repo setup/run skill: documents the import stack (pto-isa -> simpler -> pypto -> pypto-lib -> pypto-serving), clones/installs each layer, and hands off to each external repo's own .claude/skills/ + README rather than duplicating their run steps. Adds pypto-serving as a first-class repo. - multi-repo-qwen-setup is the concrete qwen3 guide, covering BOTH the pypto-serving runner (batch-1 / batch-16 prefill/decode TPOT) and the pypto-lib decode_layer kernel (scheduler-overhead DFX), framed up front as two distinct paths (different repos, entry points, what they measure). It defers clone/install to multi-repo-setup, then gives the run commands, the prefill ring-sizing + op-timeout gotchas behind a generic 507018, how to read the per-token timing layers, and the overhead-analysis flow. The timeout facts use the post-hw-native-sys#1175 compiled defaults (op_execute 45s / stream_sync 50s / scheduler 10s): a clean ~16.7s batch-16 prefill no longer trips the op-timeout, so the PTO2_OP_EXECUTE_TIMEOUT_US / PTO2_STREAM_SYNC_TIMEOUT_MS env overrides are only needed on a contended box, and breaking the scheduler < op_execute < stream_sync ordering drops the whole override set. The per-token timing section uses the [STRACE] host-trace markers + strace_timing.py / docs/dfx/host-trace.md landed in simpler hw-native-sys#1177. The §7 host-instrumentation rebuild uses an $ARCH variable and a find-based copy (python-version-agnostic), matching multi-repo-setup Step 3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
that referenced
this pull request
Jun 28, 2026
…1178) Split the cross-repo qwen3 work into two focused skills: - multi-repo-setup becomes the repo-relationship + per-repo setup/run skill: documents the import stack (pto-isa -> simpler -> pypto -> pypto-lib -> pypto-serving), clones/installs each layer, and hands off to each external repo's own .claude/skills/ + README rather than duplicating their run steps. Adds pypto-serving as a first-class repo. - multi-repo-qwen-setup is the concrete qwen3 guide, covering BOTH the pypto-serving runner (batch-1 / batch-16 prefill/decode TPOT) and the pypto-lib decode_layer kernel (scheduler-overhead DFX), framed up front as two distinct paths (different repos, entry points, what they measure). It defers clone/install to multi-repo-setup, then gives the run commands, the prefill ring-sizing + op-timeout gotchas behind a generic 507018, how to read the per-token timing layers, and the overhead-analysis flow. The timeout facts use the post-#1175 compiled defaults (op_execute 45s / stream_sync 50s / scheduler 10s): a clean ~16.7s batch-16 prefill no longer trips the op-timeout, so the PTO2_OP_EXECUTE_TIMEOUT_US / PTO2_STREAM_SYNC_TIMEOUT_MS env overrides are only needed on a contended box, and breaking the scheduler < op_execute < stream_sync ordering drops the whole override set. The per-token timing section uses the [STRACE] host-trace markers + strace_timing.py / docs/dfx/host-trace.md landed in simpler #1177. The §7 host-instrumentation rebuild uses an $ARCH variable and a find-based copy (python-version-agnostic), matching multi-repo-setup Step 3.
doraemonmj
pushed a commit
to doraemonmj/simpler_wc
that referenced
this pull request
Jul 1, 2026
- Raise a2a3 and a5 onboard timeout defaults for production workloads - Restore CI fast-fail values with PTO2 timeout env overrides - Keep host timeout ordering validation aligned with scheduler overrides - Update docs and timeout config tests for the new defaults
doraemonmj
pushed a commit
to doraemonmj/simpler_wc
that referenced
this pull request
Jul 1, 2026
…w-native-sys#1178) Split the cross-repo qwen3 work into two focused skills: - multi-repo-setup becomes the repo-relationship + per-repo setup/run skill: documents the import stack (pto-isa -> simpler -> pypto -> pypto-lib -> pypto-serving), clones/installs each layer, and hands off to each external repo's own .claude/skills/ + README rather than duplicating their run steps. Adds pypto-serving as a first-class repo. - multi-repo-qwen-setup is the concrete qwen3 guide, covering BOTH the pypto-serving runner (batch-1 / batch-16 prefill/decode TPOT) and the pypto-lib decode_layer kernel (scheduler-overhead DFX), framed up front as two distinct paths (different repos, entry points, what they measure). It defers clone/install to multi-repo-setup, then gives the run commands, the prefill ring-sizing + op-timeout gotchas behind a generic 507018, how to read the per-token timing layers, and the overhead-analysis flow. The timeout facts use the post-hw-native-sys#1175 compiled defaults (op_execute 45s / stream_sync 50s / scheduler 10s): a clean ~16.7s batch-16 prefill no longer trips the op-timeout, so the PTO2_OP_EXECUTE_TIMEOUT_US / PTO2_STREAM_SYNC_TIMEOUT_MS env overrides are only needed on a contended box, and breaking the scheduler < op_execute < stream_sync ordering drops the whole override set. The per-token timing section uses the [STRACE] host-trace markers + strace_timing.py / docs/dfx/host-trace.md landed in simpler hw-native-sys#1177. The §7 host-instrumentation rebuild uses an $ARCH variable and a find-based copy (python-version-agnostic), matching multi-repo-setup Step 3.
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jul 27, 2026
Fixes hw-native-sys#1484 The example carried pypto codegen last refreshed on 2026-06-25, cited a provenance file upstream had deleted (`models/qwen3/14b/decode_layer.py`), and covered 2 of Qwen3-14B's 40 layers. Re-harvested from `decode_fwd.py`'s `decode_fwd_layers` at pypto-lib `45be52c` with `_CHUNK_NLAYERS = 40`, and the README now records that commit plus the ptoas and pto-isa pins so the next drift is detectable rather than archaeological. Scaling to the full model is nearly free. The generated orchestration loops over layers (`for (int64_t i = 0; i < 40; i += 1)`) instead of unrolling them, so going from 2 to 40 moves a trip count and leaves the kernel set the same size. The lib chunks its dispatches because a 40-layer one used to exceed a 2000 ms stream-sync timeout; simpler raised that to 50 s in hw-native-sys#1175 and a 40-layer step is ~40 ms, so the constraint is gone. What did change is attention. Since pypto-lib hw-native-sys#765 and hw-native-sys#796 it is a CANN FusedInferAttentionScore extern rather than generated kernels, folding per-head QK-norm, RoPE, the paged KV write, the flash inner loop and the online softmax into one mixed AIC+2xAIV task behind an FFTS barrier. Seven kernels go away (`rope_qkv`, `qk_gamma`, `qk_recip`, `fa_work_build`, `fa_fused_aic`/`_aiv`, `online_softmax`, `attn_fence`) and the harvested extern arrives under `kernels/paged_attention_cce/`. Its `vendor/` subtree is CANN source under Huawei copyright; the tree is copied verbatim, because `kernel/fai_body.hpp` reaches its dependencies by relative include and splitting it would mean re-patching on every refresh. Verbatim also means our header, formatting and language lint must leave it alone, hence the carve-out. That extern also moves the paged KV layout from NSND to vLLM's BSND (`[page, token, kv_head, dim]`, so `slot_mapping` is directly the row index), which the golden now models for both the attention read and the KV write-back. The previous harvest needed one hand-edit, replacing a `[[block_local]] static` in `fa_fused_aiv` that the AICore loader rejected. That kernel no longer exists and the current codegen emits no such construct, so this harvest is unmodified codegen throughout. Verification: the case passes on device with output and all 40 layers' KV caches matching the torch reference at RTOL=5e-2 / ATOL=1e-1, in ~5 min wall on an idle die, with no ring-sizing overrides — per-layer intermediates live inside that iteration's scope, so the live set does not grow with layer count. The fixture is 38.05 GiB (24.61 weights + 13.44 paged KV) against 64 GiB of HBM. The l0-swimlane docs used this example as their "real SPMD workload", shrinking the generated fa_fused via `--set-arg 0=96` on its work-item count. The extern takes its work distribution from runtime tiling metadata instead, so no `--set-arg` shrinks it and those references are removed rather than repointed. The dfx buffer-capacity row measured at 2 layers is dropped for the same reason: it would now be a stale number rather than a wrong one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jul 27, 2026
Fixes hw-native-sys#1484 The example carried pypto codegen last refreshed on 2026-06-25, cited a provenance file upstream had deleted (`models/qwen3/14b/decode_layer.py`), and covered 2 of Qwen3-14B's 40 layers. Re-harvested from `decode_fwd.py`'s `decode_fwd_layers` at pypto-lib `45be52c` with `_CHUNK_NLAYERS = 40`, and the README now records that commit plus the ptoas and pto-isa pins so the next drift is detectable rather than archaeological. Scaling to the full model is nearly free. The generated orchestration loops over layers (`for (int64_t i = 0; i < 40; i += 1)`) instead of unrolling them, so going from 2 to 40 moves a trip count and leaves the kernel set the same size. The lib chunks its dispatches because a 40-layer one used to exceed a 2000 ms stream-sync timeout; simpler raised that to 50 s in hw-native-sys#1175 and a 40-layer step is ~40 ms, so the constraint is gone. What did change is attention. Since pypto-lib hw-native-sys#765 and hw-native-sys#796 it is a CANN FusedInferAttentionScore extern rather than generated kernels, folding per-head QK-norm, RoPE, the paged KV write, the flash inner loop and the online softmax into one mixed AIC+2xAIV task behind an FFTS barrier. Seven kernels go away (`rope_qkv`, `qk_gamma`, `qk_recip`, `fa_work_build`, `fa_fused_aic`/`_aiv`, `online_softmax`, `attn_fence`) and the harvested extern arrives under `kernels/paged_attention_cce/`. Its `vendor/` subtree is CANN source under Huawei copyright; the tree is copied verbatim, because `kernel/fai_body.hpp` reaches its dependencies by relative include and splitting it would mean re-patching on every refresh. Verbatim also means our header, formatting and language lint must leave it alone, hence the carve-out. That extern also moves the paged KV layout from NSND to vLLM's BSND (`[page, token, kv_head, dim]`, so `slot_mapping` is directly the row index), which the golden now models for both the attention read and the KV write-back. The previous harvest needed one hand-edit, replacing a `[[block_local]] static` in `fa_fused_aiv` that the AICore loader rejected. That kernel no longer exists and the current codegen emits no such construct, so this harvest is unmodified codegen throughout. Verification: the case passes on device with output and all 40 layers' KV caches matching the torch reference at RTOL=5e-2 / ATOL=1e-1, in ~5 min wall on an idle die, with no ring-sizing overrides — per-layer intermediates live inside that iteration's scope, so the live set does not grow with layer count. The fixture is 38.05 GiB (24.61 weights + 13.44 paged KV) against 64 GiB of HBM. The l0-swimlane docs used this example as their "real SPMD workload", shrinking the generated fa_fused via `--set-arg 0=96` on its work-item count. The extern takes its work distribution from runtime tiling metadata instead, so no `--set-arg` shrinks it and those references are removed rather than repointed. The dfx buffer-capacity row measured at 2 layers is dropped for the same reason: it would now be a stale number rather than a wrong one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Jul 27, 2026
Fixes hw-native-sys#1484 The example carried pypto codegen last refreshed on 2026-06-25, cited a provenance file upstream had deleted (`models/qwen3/14b/decode_layer.py`), and covered 2 of Qwen3-14B's 40 layers. Re-harvested from `decode_fwd.py`'s `decode_fwd_layers` at pypto-lib `45be52c` with `_CHUNK_NLAYERS = 40`, and the README now records that commit plus the ptoas and pto-isa pins so the next drift is detectable rather than archaeological. Scaling to the full model is nearly free. The generated orchestration loops over layers (`for (int64_t i = 0; i < 40; i += 1)`) instead of unrolling them, so going from 2 to 40 moves a trip count and leaves the kernel set the same size. The lib chunks its dispatches because a 40-layer one used to exceed a 2000 ms stream-sync timeout; simpler raised that to 50 s in hw-native-sys#1175 and a 40-layer step is ~40 ms, so the constraint is gone. What did change is attention. Since pypto-lib hw-native-sys#765 and hw-native-sys#796 it is a CANN FusedInferAttentionScore extern rather than generated kernels, folding per-head QK-norm, RoPE, the paged KV write, the flash inner loop and the online softmax into one mixed AIC+2xAIV task behind an FFTS barrier. Seven kernels go away (`rope_qkv`, `qk_gamma`, `qk_recip`, `fa_work_build`, `fa_fused_aic`/`_aiv`, `online_softmax`, `attn_fence`) and the harvested extern arrives under `kernels/vendor/paged_attention_cce/`. That extern needs CANN devkit headers, which nothing could express: every incore compiled against the same fixed include set, even though `KernelCompiler.compile_incore` has always taken `extra_include_dirs`. The scene-test `CALLABLE` now carries a per-incore `extra_include_dirs`, resolved when the kernel is compiled rather than when its module is imported — sim and macOS runners collect a case declaring a CANN dependency without having CANN. Entries may use `$VAR` and may name directories that do not exist here, so one candidate list spans SDK layouts; an unset variable or a set where nothing survived raises, which beats a "file not found" from four includes deep in vendored code. The harvested tree is copied verbatim and must stay that way: `fai_body.hpp` reaches its dependencies by relative include, and reformatting it would make the next refresh diff against our copy instead of against upstream — the exact drift this example exists to expose. It therefore lives under `kernels/vendor/`, and the repo's header, formatting and language lint skip anything below a `vendor/` directory. Keying the carve-out on the directory rather than on this operator's name means the next harvested extern needs no lint change at all. The extern also moves the paged KV layout from NSND to vLLM's BSND (`[page, token, kv_head, dim]`, so `slot_mapping` is directly the row index), which the golden now models for both the attention read and the KV write-back. The previous harvest needed one hand-edit, replacing a `[[block_local]] static` in `fa_fused_aiv` that the AICore loader rejected. That kernel no longer exists and the current codegen emits no such construct, so this harvest is unmodified codegen throughout. At 40 layers the case takes ~5 min against a 38.05 GiB fixture (24.61 weights + 13.44 paged KV, bf16) — over half the onboard sweep's 600 s session budget. It runs as its own CI step on a dedicated device instead, keeping that sweep's short timeout meaningful as a hang detector. Verification: passes on device with output and all 40 layers' KV caches matching the torch reference at RTOL=5e-2 / ATOL=1e-1, with no ring-sizing overrides — per-layer intermediates live inside that iteration's scope, so the live set does not grow with layer count. The l0-swimlane docs used this example as their "real SPMD workload", shrinking the generated fa_fused via `--set-arg 0=96` on its work-item count. The extern takes its work distribution from runtime tiling metadata instead, so no `--set-arg` shrinks it and those references are removed rather than repointed. The dfx buffer-capacity row measured at 2 layers is dropped for the same reason: it would now be a stale number rather than a wrong one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
that referenced
this pull request
Jul 27, 2026
) Fixes #1484 The example carried pypto codegen last refreshed on 2026-06-25, cited a provenance file upstream had deleted (`models/qwen3/14b/decode_layer.py`), and covered 2 of Qwen3-14B's 40 layers. Re-harvested from `decode_fwd.py`'s `decode_fwd_layers` at pypto-lib `45be52c` with `_CHUNK_NLAYERS = 40`, and the README now records that commit plus the ptoas and pto-isa pins so the next drift is detectable rather than archaeological. Scaling to the full model is nearly free. The generated orchestration loops over layers (`for (int64_t i = 0; i < 40; i += 1)`) instead of unrolling them, so going from 2 to 40 moves a trip count and leaves the kernel set the same size. The lib chunks its dispatches because a 40-layer one used to exceed a 2000 ms stream-sync timeout; simpler raised that to 50 s in #1175 and a 40-layer step is ~40 ms, so the constraint is gone. What did change is attention. Since pypto-lib #765 and #796 it is a CANN FusedInferAttentionScore extern rather than generated kernels, folding per-head QK-norm, RoPE, the paged KV write, the flash inner loop and the online softmax into one mixed AIC+2xAIV task behind an FFTS barrier. Seven kernels go away (`rope_qkv`, `qk_gamma`, `qk_recip`, `fa_work_build`, `fa_fused_aic`/`_aiv`, `online_softmax`, `attn_fence`) and the harvested extern arrives under `kernels/vendor/paged_attention_cce/`. That extern needs CANN devkit headers, which nothing could express: every incore compiled against the same fixed include set, even though `KernelCompiler.compile_incore` has always taken `extra_include_dirs`. The scene-test `CALLABLE` now carries a per-incore `extra_include_dirs`, resolved when the kernel is compiled rather than when its module is imported — sim and macOS runners collect a case declaring a CANN dependency without having CANN. Entries may use `$VAR` and may name directories that do not exist here, so one candidate list spans SDK layouts; an unset variable or a set where nothing survived raises, which beats a "file not found" from four includes deep in vendored code. The harvested tree is copied verbatim and must stay that way: `fai_body.hpp` reaches its dependencies by relative include, and reformatting it would make the next refresh diff against our copy instead of against upstream — the exact drift this example exists to expose. It therefore lives under `kernels/vendor/`, and the repo's header, formatting and language lint skip anything below a `vendor/` directory. Keying the carve-out on the directory rather than on this operator's name means the next harvested extern needs no lint change at all. The extern also moves the paged KV layout from NSND to vLLM's BSND (`[page, token, kv_head, dim]`, so `slot_mapping` is directly the row index), which the golden now models for both the attention read and the KV write-back. The previous harvest needed one hand-edit, replacing a `[[block_local]] static` in `fa_fused_aiv` that the AICore loader rejected. That kernel no longer exists and the current codegen emits no such construct, so this harvest is unmodified codegen throughout. At 40 layers the case takes ~5 min against a 38.05 GiB fixture (24.61 weights + 13.44 paged KV, bf16) — over half the onboard sweep's 600 s session budget. It runs as its own CI step on a dedicated device instead, keeping that sweep's short timeout meaningful as a hang detector. Verification: passes on device with output and all 40 layers' KV caches matching the torch reference at RTOL=5e-2 / ATOL=1e-1, with no ring-sizing overrides — per-layer intermediates live inside that iteration's scope, so the live set does not grow with layer count. The l0-swimlane docs used this example as their "real SPMD workload", shrinking the generated fa_fused via `--set-arg 0=96` on its work-item count. The extern takes its work distribution from runtime tiling metadata instead, so no `--set-arg` shrinks it and those references are removed rather than repointed. The dfx buffer-capacity row measured at 2 layers is dropped for the same reason: it would now be a stale number rather than a wrong one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1165
Validation
git diff --checkpython3 -m py_compile tests/st/aicore_op_timeout/test_aicore_op_timeout.pyissue1165 timeout constants ok)test_runtime_timeout_configC++ UT, 9 passedtask-submit:pytest tests/st/aicore_op_timeout/test_aicore_op_timeout.py --platform a2a3 --device $TASK_DEVICE --max-parallel 1 -q -s, 1 passed