Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
fb03b96
Refactor: retire the PTO2 prefix on the resource-shape type
ChaoWao Aug 24, 2026
d84786f
Refactor: retire the PTO2 prefix on the wire task structs
ChaoWao Aug 24, 2026
d6dcc64
Refactor: retire the PTO2 prefix on the shared-memory structs
ChaoWao Aug 24, 2026
9daa24d
Refactor: retire the PTO2 prefix on the scheduler-state structs
ChaoWao Aug 24, 2026
6577ae8
Refactor: retire the PTO2 prefix on the orchestrator-state structs
ChaoWao Aug 24, 2026
0e8e4ab
Refactor: name the chip runtime's tensor map by its context
ChaoWao Aug 24, 2026
016446e
Refactor: name the chip runtime's ready queue by its context
ChaoWao Aug 24, 2026
1dde6df
Refactor: name the chip runtime's task state by its context
ChaoWao Aug 24, 2026
4aff540
Refactor: name the chip runtime's ring types by their context
ChaoWao Aug 24, 2026
c82420d
Refactor: retire the PTO2 prefix on the task allocator
ChaoWao Aug 24, 2026
789a462
Refactor: retire the PTO2 prefix on the fanin and fanout types
ChaoWao Aug 24, 2026
8b434bc
Refactor: retire the PTO2 prefix on the dependency-list pool
ChaoWao Aug 24, 2026
aac8af1
Refactor: retire the PTO2 prefix on the early-dispatch state machines
ChaoWao Aug 24, 2026
3c0c0aa
Refactor: retire the PTO2 prefix on the subtask and dispatch types
ChaoWao Aug 24, 2026
f9ac2e6
Refactor: name the chip runtime's scope capacities by their context
ChaoWao Aug 24, 2026
89db474
Refactor: name the chip runtime's alignment macros by their context
ChaoWao Aug 24, 2026
d7c67d8
Refactor: retire the PTO2 prefix on the remaining runtime constants
ChaoWao Aug 24, 2026
58b6a6f
Fix: point two diagnostics at the environment variables that exist
ChaoWao Aug 24, 2026
3c2aeb6
Refactor: retire the PTO2 brand from comments, logs and docs
ChaoWao Aug 24, 2026
295217f
Docs: record that the PTO2 retirement is down to its Tier-C knobs
ChaoWao Aug 24, 2026
7f16161
Refactor: retire the process-wide ring sizing environment variables
ChaoWao Aug 24, 2026
08589fb
Fix: ship the cmake includes the platform builds need in the wheel
ChaoWao Aug 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
45 changes: 38 additions & 7 deletions .claude/rules/codestyle.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,10 +125,18 @@
The project is **`simpler`** ("Simple Runtime"); its two runtimes are
`host_build_graph` and `tensormap_and_ringbuffer`. **"PTO Runtime",
"PTO Runtime2", and a bare "Runtime2" are not names of anything** — the
`2` never denoted a version any reader can identify. The legacy surface
is far too large to fix at once (snapshot 2026-07-26: ~6.6k `PTO2`
occurrences across 367 files, plus 17 prose "PTO Runtime[2]" mentions),
so it is retired in graded steps. Find the current surface with:
`2` never denoted a version any reader can identify.

**The retirement is complete** (snapshot 2026-08-24: 6.6k occurrences across
367 files on 2026-07-26 → 0 that this repo reads). Nothing here reads a
`PTO2` name any more: the last of them were the `PTO2_RING_*` ring-sizing
environment variables, retired in favour of per-task
`CallConfig.runtime_env`. What is left in the tree is this rule's own
examples, the warning that fires if a caller still exports a retired ring
variable, prose recording that these names are gone, and
`PTO2_MANUAL_MAX_SEQ` — a pypto-lib knob two example READMEs document and
this repo never reads. **So a new `PTO2` match is a defect** — there is no
backlog to work through. Find the surface with:

```bash
grep -rIn -E "PTO Runtime2?|PTO2|pto2_|pto_runtime2" --include=* . | grep -v '^\./\.git/'
Expand All @@ -140,9 +148,23 @@
| ---- | ---- | ------ |
| **A — prose** | Brand mentions in docs, headings, skill descriptions, issue templates, and code comments (`# PTO Runtime2 Profiling Levels`, "the PTO Runtime consists of…") | **Fix on sight, unconditionally.** No compile risk, no contract. Say `simpler`, or name the actual component ("the AICPU orchestrator", "the `tensormap_and_ringbuffer` runtime") when that is what the sentence means. |
| **B — internal identifiers** | `PTO2Foo` types, `pto2_*` functions, internal `PTO2_*` macros and enumerators, `pto_*.h` / `pto_runtime2*.cpp` file names | Rename **only when you are already modifying that code**, per rule 9, and finish the identifier you started (below). |
| **C — external contracts** | `PTO2_RING_*` and other `runtime_env` knobs (documented in `MULTI_RING.md`, set by pypto-lib / pypto-serving), `extern "C"` symbols in `runtime_c_api.h`, on-wire / serialized names | **Exempt until a migration ships.** Renaming these breaks callers in other repos. Ask the user before touching one; land it only with a compatibility alias or a coordinated cross-repo change. |

Two constraints make the difference between progress and churn:
| **C — external contracts** | `runtime_env` knobs, `extern "C"` symbols in `runtime_c_api.h`, on-wire / serialized names | **Exempt until a migration ships.** Renaming these breaks callers in other repos. Ask the user before touching one; land it only with a compatibility alias or a coordinated cross-repo change. |

An environment variable is the worst case in Tier C and the reason it
exists: an unrecognised name is *ignored*, so the rename fails silently
in production instead of at compile time. Three knobs proved the point.
Two were found already dead — a benchmark setting `PTO2_SERIAL_ORCH_SCHED`
against a runtime reading `SIMPLER_TMR_SERIAL_ORCH_SCHED_ENABLE`, and an
error hint naming `PTO2_SCHEDULER_TIMEOUT_MS` instead of
`SIMPLER_SCHEDULER_TIMEOUT_MS`. The third case, `PTO2_RING_*`, is the one
to copy: the knob was not renamed but **removed**, because
`CallConfig.runtime_env` already carried the same sizing per task and was
strictly more expressive. Retiring an env knob outright beats renaming it —
and when you do, the runtime should say so on sight rather than silently
fall back, which is what `warn_on_retired_ring_env()` in each
`host/runtime_maker.cpp` is for.

Three constraints make the difference between progress and churn:

- **One identifier, all its occurrences, one commit.** A rename that
leaves both spellings alive is worse than no rename — the reader now
Expand All @@ -153,6 +175,15 @@
- **The count only goes down.** Introducing a new `PTO2`/`PTO Runtime`
spelling anywhere — including by copying an existing `pto_*.h` to a
new arch — is a defect, not neutral. Rule 9 makes this unconditional.
- **Check the target name against three things, not one.** The bare name
may be taken by the host orchestrator under `src/common/hierarchical/`
(`TensorMap`, `ReadyQueue`, `TaskState`, `HeapRing`, `MAX_SCOPE_DEPTH`
— rule 13 decides who keeps it, and the chip runtime takes `Chip` /
`CHIP_`); by an external header the same translation units include
(CANN defines `ALIGN_UP`); or, for an **object-like macro**, by any
declaration anywhere, because a macro rewrites its identifier textually
rather than colliding where the compiler can say so. Macros therefore
keep a prefix even when the bare name looks free.

Do **not** open a repo-wide mechanical rename PR: it collides with every
in-flight branch, and a 6k-line diff cannot be reviewed for the handful
Expand Down
4 changes: 2 additions & 2 deletions .claude/rules/running-onboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,8 +174,8 @@ for the signature that actually fired:**
| -------------------- | --------- | ---- |
| `FATAL: Task Allocator Deadlock` / `FATAL: Dependency Pool Deadlock` / `FATAL: Fanin Spill Pool Deadlock` | an allocator or pool could not reclaim enough space | This header identifies the blocked resource, not the root cause. Classify it using the structural/timeout line that follows. |
| `Provable head-of-line deadlock` | **proven open-scope structural deadlock** | TRB only: the reclaim head is the oldest task owned by an open scope on that ring. The blocked orchestrator cannot end that scope, so the head cannot become consumed. On A5, classification waits for at least 10 ms without reclaim progress and an exact-watermark publication acknowledgment. |
| `No reclaim progress for ~500 ms` / `cannot reclaim space after ~500 ms` | allocator/pool **reclaim timeout** | TRB only — the 500ms backstop (`PTO2_ALLOC_DEADLOCK_TIMEOUT_CYCLES`) lives in its reclaiming allocators. It proves prolonged lack of reclaim progress, not why progress stopped; check capacity, the dumped head, consumers and scheduler state. |
| `Task Window Exhausted` / `Fanin Capacity Exhausted` / `TensorMap Entry Pool Exhausted` | **capacity**, HBG only | HBG is whole-graph-resident and reclaims nothing mid-run, so this is a sizing verdict reached immediately, not a stall. The line carries used/capacity and the requested amount; raise `PTO2_RING_TASK_WINDOW` / `PTO2_TENSORMAP_POOL_SIZE` or shrink the graph. Inline fanin is hard-capped at `PTO2_MAX_FANIN=128`; HBG has no `PTO2_RING_DEP_POOL`. HBG's graph heap has no knob and cannot exhaust: it is committed after orchestration at the measured size, so a graph too large for the device fails host-side at that commit. |
| `No reclaim progress for ~500 ms` / `cannot reclaim space after ~500 ms` | allocator/pool **reclaim timeout** | TRB only — the 500ms backstop (`ALLOC_DEADLOCK_TIMEOUT_CYCLES`) lives in its reclaiming allocators. It proves prolonged lack of reclaim progress, not why progress stopped; check capacity, the dumped head, consumers and scheduler state. |
| `Task Window Exhausted` / `Fanin Capacity Exhausted` / `TensorMap Entry Pool Exhausted` | **capacity**, HBG only | HBG is whole-graph-resident and reclaims nothing mid-run, so this is a sizing verdict reached immediately, not a stall. The line carries used/capacity and the requested amount; raise `runtime_env.ring_task_window` / `CHIP_TENSORMAP_POOL_SIZE` or shrink the graph. Inline fanin is hard-capped at `CHIP_MAX_FANIN=128`; HBG has no dependency spill pool. HBG's graph heap has no knob and cannot exhaust: it is committed after orchestration at the measured size, so a graph too large for the device fails host-side at that commit. |
| `Timeout (N cycles): producer/consumers ...` | **SPIN** wait on a specific producer/consumer | `runtime_core.cpp`. |
| `HandleTaskTimeout` / `kill aicpu-sd` | **OS op-execute timeout** | STARS/tsdaemon, default 45s (`PLATFORM_OP_EXECUTE_TIMEOUT_US`). **A 45s kill ≠ deadlock** — the op was merely long or stalled. Raise this constant to measure true on-device duration. |
| `log_stall_diagnostics` (cores idle + tasks `state=WAIT fanin 0/N` + `completed` frozen) | **forward-progress stall** | No dedicated detector — intermittent races (often contention-triggered) land here and are reaped only by the op-timeout above. |
Expand Down
13 changes: 9 additions & 4 deletions .claude/skills/multi-repo-qwen-setup/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,10 +161,15 @@ log to tell them apart**:
1. **Heap-exhausted deadlock** (default 256 MB ring): the open prefill scope's
live-set exceeds one ring → device log prints
`FATAL: Task Allocator Deadlock - Heap Exhausted! ... Provable head-of-line`.
Fix: raise the ring. batch-1 needs just over 256 MB; batch-16 needs the
prefill config `PTO2_RING_HEAP=4294967296 PTO2_RING_TASK_WINDOW=131072
PTO2_RING_DEP_POOL=131072` (heap, task-window, and dep-pool all scale with
batch; default `task_window=16384` also overflows at batch-16).
Fix: raise the ring. batch-1 needs just over 256 MB; batch-16 needs
`ring_heap=4294967296`, `ring_task_window=131072`, `ring_dep_pool=131072`
(heap, task-window, and dep-pool all scale with batch; the default
`task_window=16384` also overflows at batch-16). These are per-task values on
`CallConfig.runtime_env` — the runner has to pass them with the prefill call.
The `PTO2_RING_*` environment variables that used to carry them are no longer
read by simpler, and a stale export is silently ignored (simpler warns), so a
runner still exporting them runs on the 256 MB default and deadlocks exactly
as described above.
2. **Op-execute timeout** (slow/contended prefill): device log prints
`HandleTaskTimeout ... timeOut:<N>` and kills `aicpu-sd`. The compiled
defaults (`PLATFORM_OP_EXECUTE_TIMEOUT_US`=45 s,
Expand Down
9 changes: 9 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,15 @@ add_custom_target(build_package_sim
if(SKBUILD_MODE)
install(DIRECTORY ${CMAKE_SOURCE_DIR}/src/
DESTINATION simpler_setup/_assets/src)
# The platform CMakeLists include these through SIMPLER_CMAKE_DIR, which
# runtime_compiler.py resolves to PROJECT_ROOT/cmake — the installed
# simpler_setup/_assets/cmake in a wheel. All four are reachable from a
# platform configure: host_log_sources.cmake and profiling_config.cmake are
# included unconditionally, profiling_config.cmake configures
# profiling_build_config.h.in, and sanitizers.cmake is included when a
# sanitizer build is requested.
install(DIRECTORY ${CMAKE_SOURCE_DIR}/cmake/
DESTINATION simpler_setup/_assets/cmake)
install(DIRECTORY ${CMAKE_SOURCE_DIR}/build/lib/
DESTINATION simpler_setup/_assets/build/lib
OPTIONAL
Expand Down
4 changes: 2 additions & 2 deletions docs/buffer-abi.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ That is self-consistent, but it charges the device for host-side fields:

- **Device cost.** `sizeof(ChipTensor)` is pinned at **128 B** by
`static_assert(… == 128, "ChipTensor must be exactly 2 cache lines (128 bytes)")` and by
`PTO2_TASKPAYLOAD_TENSOR_STRIDE`, which the AICPU scheduler strides the payload
with. A merged struct is ~192–216 B, taking `PTO2TaskPayload` from 4864 B to
`TASKPAYLOAD_TENSOR_STRIDE`, which the AICPU scheduler strides the payload
with. A merged struct is ~192–216 B, taking `TaskPayload` from 4864 B to
6912–7680 B — **+42% to +58% per task slot** — because the payload embeds
`ChipTensor tensors[MAX_TENSOR_ARGS]` **by value** in the device task ring.
- **Zero return for that cost.** Most of what a merge would add has no reader on
Expand Down
8 changes: 4 additions & 4 deletions docs/dfx/dep-gen.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ the record entirely. The device hot path no longer carries fanout;
When it existed, each producer task carried its own
`ChipSwimlaneAicpuTaskRecord.fanout[]`, populated by the AICPU scheduler at the
moment it wired a downstream consumer. If a producer had already finished and
transitioned to `PTO2_TASK_COMPLETED` by the time a later submit wanted to
transitioned to `CHIP_TASK_COMPLETED` by the time a later submit wanted to
register a dependency on it, the consumer's edge had nowhere to go — the
record was sealed, the slot was closed, and the edge was silently dropped.
This was not a bug in fanout itself; fanout is "successors known at runtime",
Expand Down Expand Up @@ -54,7 +54,7 @@ inputs to each submit are captured and the graph is reconstructed afterwards.
- **Host replay (dual-pass, self-checking).** After `reconcile_counters()`
confirms a clean trace (no drops, no leftovers),
`dep_gen_replay_emit_deps_json` runs every record back through *two*
parallel host-resident `PTO2TensorMap` instances that evolve in lockstep:
parallel host-resident `ChipTensorMap` instances that evolve in lockstep:
- **Oracle pass** drives the canonical `compute_task_fanin` template
from `dep_compute.h` and collects the producer-id → `DepFlags`
mapping the runtime would have emitted (flags OR-accumulated per
Expand All @@ -66,7 +66,7 @@ inputs to each submit are captured and the graph is reconstructed afterwards.
Per-record semantics mirror runtime `submit_task` exactly: STEP 1
(explicit deps), STEP 3 (creator retention + tensormap lookup),
STEP 4 (register outputs). Per-successor dedup matches
`PTO2FaninBuilder::append_fanin_or_fail`. After both passes finish per
`FaninBuilder::append_fanin_or_fail`. After both passes finish per
record, the replay asserts the two producer-id → `DepFlags` mappings are
equal (same producers and same per-producer flags); if they diverge,
`deps.json` is not written and the function returns non-zero.
Expand Down Expand Up @@ -398,7 +398,7 @@ list; only the dep_gen replay graph loses the tail.
| AICPU writer | `src/{a2a3,a5}/platform/include/aicpu/dep_gen_collector_aicpu.h`, `src/common/platform/shared/aicpu/dep_gen_collector_aicpu.cpp` | Single-instance write path; weak-fallback exported to host build. Both platforms share the same writer implementation — the writer accesses its own device-side view of shared memory, independent of how host↔device transport is implemented. |
| Host collector | `src/common/platform/include/host/dep_gen_collector.h`, `src/common/platform/shared/host/dep_gen_collector.cpp` | `ProfilerBase<DepGenCollector, DepGenModule>` — drains ring → `records_` vector. On non-SVM platforms it uses the base `alloc_paired_buffer`, which malloc's a host shadow + `copy_to_device`'s it and registers it via `add_malloc_shadow` so teardown can free it; `reconcile_counters` explicitly `copy_from_device`'s the BufferState before reading, and `finalize` lets `BufferPoolManager::clear_mappings()` release all shadows as the single source of truth. |
| Capture call site (device-orch) | `src/{a2a3,a5}/runtime/tensormap_and_ringbuffer/runtime/orchestrator.cpp` `submit_task_common` | One conditional block that snapshots inputs into the ring when `is_dep_gen_enabled()`; fires for both `submit_task` and `submit_dummy_task`. The schema carries `kernel_ids[3] = {aic, aiv0, aiv1}` so the swimlane post-processor can resolve `task_id → kernel` from `deps.json` at level=1 where the AICore record is the sole device-side identity source. Inactive subslots stay at `INVALID_KERNEL_ID = -1`. It also carries the SPMD logical block num (`block_num` on a2a3, `core_num` on a5's launch spec) as `tasks[].block_num`. |
| Replay | `src/{a2a3,a5}/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.{h,cpp}` | Pure CPU; runs dual-pass differential replay — `compute_task_fanin` (oracle) + inlined STEP A/B mirror (annotated) against two `PTO2TensorMap` instances. Emits `deps.json` when both passes agree per record. Platform-agnostic — a5 reuses the a2a3 source verbatim. |
| Replay | `src/{a2a3,a5}/runtime/tensormap_and_ringbuffer/host/dep_gen_replay.{h,cpp}` | Pure CPU; runs dual-pass differential replay — `compute_task_fanin` (oracle) + inlined STEP A/B mirror (annotated) against two `ChipTensorMap` instances. Emits `deps.json` when both passes agree per record. Platform-agnostic — a5 reuses the a2a3 source verbatim. |
| Host-direct capture (host-orch) | `src/a2a3/runtime/host_build_graph/runtime/dep_gen_host_graph.h`, `src/a2a3/runtime/host_build_graph/host/dep_gen_host_graph.cpp` | Task / tensor / edge tables filled from `submit_task_common` + `compute_task_fanin`'s `Annotate` hooks (`src/a2a3/runtime/host_build_graph/runtime/dep_compute.h`), reset per orchestration by `run_host_orchestration`, serialized by the same `deps.json` writer. The runtime translation unit carries weak no-op fallbacks so the AICPU build links without it. |
| Device-runner hookup | `src/{a2a3,a5}/platform/{onboard,sim}/host/device_runner.cpp` | `dep_gen_host_graph_active()` picks the shape: host-orch calls `dep_gen_host_graph_emit(deps_path)` at teardown — reading the thread-local graph its own orchestration built on the same child progress thread — and skips collector init/start/reconcile entirely; device-orch calls `dep_gen_replay_emit_deps_json(records.data(), records.size(), deps_path)` post-`reconcile_counters`. The c_api latches the CallConfig before the bind so host capture is armed before the orchestration it records. |
| Viewer | `simpler_setup/tools/deps_viewer.py` | `deps.json` → text (default) or pan/zoom HTML |
Expand Down
17 changes: 13 additions & 4 deletions docs/dfx/profiling-config-naming.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,19 @@ runtime-specific ones) unambiguous about ownership.

### 4. Everything carries the `SIMPLER_` project prefix

`PTO2_` is the device runtime's internal namespace (89 identifiers:
`CHIP_MAX_RING_DEPTH`, `SIMPLER_ERROR_*`, `RuntimeContext`, …). Configuration
surface exposed to users / CI / external consumers is unified under
`SIMPLER_`, regardless of which internal namespace implements it.
The device runtime's internal names carry no project prefix at all — they are
plain domain names (`RuntimeContext`, `SchedulerState`, `TaskDescriptor`), taking
a `Chip` / `CHIP_` prefix in two cases: where the host orchestrator owns the bare
name (`ChipTensorMap`, `ChipReadyQueue`, `CHIP_MAX_SCOPE_DEPTH`), and for
object-like macros, which capture their identifier textually in every including
translation unit (`CHIP_HEAP_SIZE`, `CHIP_ALIGN_UP`). Configuration surface exposed
to users / CI / external consumers is unified under `SIMPLER_`, and status codes
under `SIMPLER_ERROR_*`.

There is no `PTO2_` configuration surface left. `PTO2_RING_TASK_WINDOW` /
`PTO2_RING_HEAP` / `PTO2_RING_DEP_POOL` were the last of it; ring sizing is per
task now, on `CallConfig.runtime_env`, and the runtime warns if one of those names
is still exported so a stale export cannot quietly become the compile-time default.

## Compile-time gates vs runtime emission

Expand Down
19 changes: 13 additions & 6 deletions docs/dfx/scope-stats.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,19 @@ schema.
The metadata line is also the quickest way to verify per-ring runtime sizing.
For example, after running with:

```python
# In the scene test's CASES entry -- ring sizing is per task, so it travels with
# the case rather than with the process:
"config": {
"runtime_env": {
"ring_task_window": [8192, 16384, 131072, 524288],
"ring_heap": [134217728, 268435456, 402653184, 536870912],
"ring_dep_pool": [4096, 8192, 16384, 32768],
}
}
```

```bash
CASE=...
NAME=...
PTO2_RING_TASK_WINDOW=8192,16384,131072,524288 \
PTO2_RING_HEAP=134217728,268435456,402653184,536870912 \
PTO2_RING_DEP_POOL=4096,8192,16384,32768 \
python "tests/st/${CASE}/test_${NAME}.py" -p a2a3 -d 0 --enable-scope-stats
```

Expand Down Expand Up @@ -295,7 +302,7 @@ the host to chase a device pointer into the orchestration `.so`'s string
table. `on_fatal` sets `header.fatal_latched`, surfacing as
`"fatal": true`; the host still emits whatever records made it.

`ring_id` outside `[0, PTO2_SCOPE_STATS_MAX_RING_DEPTH)` is silently
`ring_id` outside `[0, SCOPE_STATS_MAX_RING_DEPTH)` is silently
dropped. Caps are copied verbatim into the buffer header so the host can
render `used/cap` without a second device→host query.

Expand Down
Loading
Loading