Skip to content

Fix: bind runner TLS during arena prewarm - #1401

Merged
ChaoZheng109 merged 1 commit into
hw-native-sys:mainfrom
hashiqiqixian:codex/fix-prewarm-runner-tls
Jul 20, 2026
Merged

ChaoZheng109 merged 1 commit into
hw-native-sys:mainfrom
hashiqiqixian:codex/fix-prewarm-runner-tls

Conversation

@hashiqiqixian

@hashiqiqixian hashiqiqixian commented Jul 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • bind the active DeviceRunner in thread-local storage for the full simpler_init() call
  • clear the binding on every return path with RAIIScopeGuard
  • prevent eager arena prewarming from dereferencing a null runner on sim and onboard
  • add an in-repo a2a3sim TRB regression test for L2 Worker.init(prewarm_config=...)

Root cause

HostApi callbacks recover the current runner through thread-local storage. The eager prewarm path runs during simpler_init(), before the existing register/run entry points establish that binding.

Validation

  • git diff --check
  • clang-format --dry-run --Werror on both modified C++ files
  • Python syntax validation for test_prewarm_config.py
  • Simpler PR CI runs the new test under the existing a2a3sim scene-test job
  • downstream PyPTO CI: bump pinned pto-isa to 96ba706c #2085 previously passed the five system jobs that crashed before the fix

A full local runtime rebuild was not run because the development host had only 3.07 GB available RAM; PR CI provides the runtime build and simulator execution.

Related

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the simpler_init function in both src/common/platform/onboard/host/c_api_shared.cpp and src/common/platform/sim/host/c_api_shared.cpp to establish a thread-local binding for the device runner (g_runner_key) using pthread_setspecific. It also introduces an RAII scope guard to ensure the thread-specific data is cleared upon exiting the function. There are no review comments to address, and I have no additional feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@coderabbitai

coderabbitai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Initialization now scopes the provided device runner in thread-local storage for HostApi callbacks on onboard and simulator paths. A regression test covers L2 worker initialization with prewarm_config for the tensormap-and-ringbuffer runtime.

Changes

Prewarm runner binding

Layer / File(s) Summary
Scoped runner binding
src/common/platform/onboard/host/c_api_shared.cpp, src/common/platform/sim/host/c_api_shared.cpp
Initialization creates the pthread key once, binds the current runner for HostApi callbacks, and clears it through an RAII guard on exit.
Prewarm initialization regression test
tests/st/a2a3/tensormap_and_ringbuffer/test_prewarm_config.py
Adds an A2A3 simulation test that initializes an L2 worker with ring_task_window = 64 and closes the worker in cleanup.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

Poem

A rabbit found a runner thread,
And bound it where the callbacks sped.
Prewarm configs now hop the ring,
While cleanup clips each dangling string.
Hop, hop—initialization sings!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title accurately summarizes the main change: binding runner TLS during arena prewarm.
Description check ✅ Passed The description matches the code and test changes, including TLS binding, RAII cleanup, and the new regression test.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@hashiqiqixian
hashiqiqixian force-pushed the codex/fix-prewarm-runner-tls branch from 0492e7a to 1d91afd Compare July 20, 2026 07:37
@hashiqiqixian

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@ChaoZheng109

Copy link
Copy Markdown
Collaborator

Suggest adding an in-repo regression test. This is a bugfix, but validation currently rests entirely on downstream pypto#2085 — simpler's own CI stays green because nothing in-repo passes prewarm_config into simpler_init (the scene-test framework always calls worker.init() without it). A future refactor of the TLS/prewarm path could silently reintroduce this null-runner segfault with simpler CI still green.

The repro is small (~20–40 lines, no golden/args boilerplate needed) since L2 trb prewarm runs synchronously inside cw.init:

  • Build an L2 Worker(runtime="tensormap_and_ringbuffer", ...), call w.init(prewarm_config=CallConfig(<ring sizing>)), assert it succeeds. Pre-fix this segfaults on the null-runner deref; post-fix it returns cleanly.
  • Must be on the trb runtime — host_build_graph links the weak no-op prewarm_config_impl and won't crash, so it can't catch the regression there. A pure a2a3sim test already covers the sim half of this fix and needs no device lock.
  • prepared_callable/conftest.py already shows the fresh-L2-Worker construction pattern to copy.

If it can't land in this PR, worth filing an issue to track it.

HostApi callbacks recover the active DeviceRunner through thread-local
storage. Bind and clear that runner for the full simpler_init call so
eager arena prewarming cannot dereference a null runner on sim or
onboard.

Add an a2a3sim TRB regression test that initializes a fresh L2 Worker
with non-default ring sizing so this path remains covered in-repo.
@hashiqiqixian
hashiqiqixian force-pushed the codex/fix-prewarm-runner-tls branch from 1d91afd to 7f7082c Compare July 20, 2026 08:52
@hashiqiqixian

Copy link
Copy Markdown
Contributor Author

Added in 7f7082cf. The new tests/st/a2a3/tensormap_and_ringbuffer/test_prewarm_config.py constructs a fresh L2 TRB worker on a2a3sim, sets ring_task_window=64, and calls init(prewarm_config=config). This exercises the synchronous simpler_init prewarm path without device hardware or scene-test callable/golden boilerplate.

@hashiqiqixian

Copy link
Copy Markdown
Contributor Author

CI is green except st-onboard-a5. That job failed during checkout before setup/tests because the self-hosted runner could not remove .pytest_cache/v/cache (EACCES: permission denied). All code-related jobs, including both a2a3sim scene-test jobs with the new regression test, passed. Could a maintainer please rerun the failed job? The PR author account does not have the repository admin permission required by GitHub to rerun it.

@ChaoZheng109
ChaoZheng109 merged commit 8cdb306 into hw-native-sys:main Jul 20, 2026
30 of 31 checks passed
lyfne123 pushed a commit to hw-native-sys/pypto that referenced this pull request Jul 21, 2026
## Summary

- preserve YunjiQin's `4e352b26` prebuilt runtime-arena prewarm commit
from #2068 unchanged
- advance the runtime submodule to the merged Simpler fix from
hw-native-sys/simpler#1401
- rely on Simpler's public `Worker.init()` to perform eager L3+ startup
instead of invoking the private startup hook twice
- clarify the prewarm/dispatch config relationship in both language
versions of the docs

## Root cause

The eager prewarm path calls context-free `HostApi` callbacks during
`simpler_init`. Those callbacks recover the active runner through
`current_runner()`, but the runner TLS was not bound during
initialization. A non-null `prewarm_config` therefore dereferenced a
null runner.

Merged Simpler also makes `Worker.init()` eagerly initialize the full
L3+ hierarchy. PyPTO must not call `_start_hierarchical()` again after
`init()`; doing so attempts to register workers after the scheduler is
initialized.

## Commit structure

1. `4e352b26` — original #2068 commit by YunjiQin
2. `0fb2dfe4` — merged Simpler integration, eager-init compatibility,
and review-driven documentation clarification

## Testing

- [x] `git diff --check`
- [x] Simpler #1401 added an in-repo a2a3sim TRB regression test and was
merged as `8cdb306c`
- [x] the previous PyPTO runtime-fix revision passed the complete CI
matrix, including all five system jobs that failed on #2068
- [x] PyPTO full CI passed against the official merged Simpler commit,
including distributed, device, simulator, and model integration jobs

Local pytest/build was not run because the 16 GB development host had
only 0.89 GB available RAM; PR CI provides the required build and
simulator/device coverage.

## Related PRs

- Merged Simpler runtime fix:
hw-native-sys/simpler#1401
- Built on #2068

---------

Co-authored-by: YunjiQin <a1339924773@gmail.com>
Co-authored-by: ChenShenAi <chenshenai@huawei.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants