Repository navigation
Add: a switch of its own for the host phase per-event records - #1937
Merged
ChaoWao merged 1 commit intoAug 21, 2026
Merged
Conversation
`SIMPLER_HBG_BIND_BREAKDOWN_ENABLE` gated two unrelated things: the per-kind `LOG_TIMING` breakdown, and — as the producer's arming condition — the per-event record pool behind `host_phase_records.jsonl`. A run that wanted the summed lines while some other diagnostic flag was on had no way to decline the artifact, and the per-event path is the one that appends to a pool under a lock on the path being measured. The producer's arming condition becomes its own variable, `SIMPLER_HBG_HOST_PHASE_RECORDS_ENABLE`, so each of the three views has exactly one switch. `host_phase_timing_enabled()` is renamed `host_phase_breakdown_enabled()`, which is what it now decides alone; the shared `1`/`t`/`T` opt-in parsing moves into one helper. Collection stays armed by either the new variable or a chip-swimlane level of `ORCH_PHASES`, and writing the artifact is unchanged — a collected pass reaches the JSONL whenever the run has an output directory, so a level-4 run still produces both. What changes is that the breakdown alone no longer arms anything. Verified on a2a3 onboard with `tests/st/.../vector_example`, one run per configuration: breakdown variable alone gives 12 `bind phase=` lines and no `host_phase_records.jsonl`; the records variable alone gives the JSONL (16 records) and no lines; `--enable-chip-swimlane 4` with both variables unset gives the JSONL plus `chip_swimlane_records.json`; neither switch gives no output directory at all. hbg scene tests pass (8 L3 cases, 36 passed / 1 skipped L2 hbg, 2 L2 tmr).
|
Warning Review limit reached
Next review available in: 15 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
4 tasks done
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 21, 2026
The host phase pool was built for one writer -- it is the device pool's shape,
where the AICPU alone advances a cursor and the host reads -- and host recording
became multi-threaded under it. A global mutex made that safe: every producer
serialized on one lock to append into one active buffer, and every producer's
counter update landed on one cache line per kind.
The producers are independent. A recording thread records its own Definition and
counts only its own operations; nothing about a record needs coordination. So the
lock and the shared counters bought nothing, and they cost the emitting thread
3.3 us per record at eight producers against 0.1 us at one. All of it runs after
the record's own end timestamp, so it was invisible as anything except a gap
between two consecutive records -- which is exactly how it presented: a swimlane
whose recording lanes looked 45-66% idle, with the idle time attributed to the
workload.
Each producer now claims a whole buffer and fills it alone, so its records are
contiguous and no cache line is shared, and its counters and in-flight claim are
its own lines. The per-record path has no shared write left; the one atomic is a
buffer claim taken once per 512 records. Measured on a 320-core aarch64 host at
20k records per thread:
producers mutex + shared counters per-producer lanes
1 122 ns 48 ns
4 1718 ns 50 ns
8 3233 ns 49 ns
16 7321 ns 49 ns
Flat rather than merely lower, and 48 ns is what the two clock_gettime calls cost
on their own, so the record itself is now free. An intermediate form -- one atomic
slot claim per record into a flat array -- still cost 1.1 us at eight producers,
because a 40-byte record leaves adjacent slots owned by different threads sharing
a line; that is why the buffers are per producer and not merely lock-free.
On dsv4's seven-Definition Graph form, a2a3, `--rounds 5` with the device run
skipped, minimum over the 8 warm passes:
host_orch 1.291 ms -> 0.828 ms -36%
control-plane total 2.998 ms -> 2.379 ms -21%
and in the per-event view the gap between two `record_node` records falls from
1.96-2.30 us to 0.25-0.68 us on every lane. The orchestration itself is unchanged;
this is the instrument getting out of the way of what it was measuring. It also
removes the reason hw-native-sys#1937 gives for wanting the per-event pool separately
switchable -- that path no longer appends under a lock -- though having one switch
per view is worth keeping on its own terms.
Two consequences worth naming. The pool is no longer the device pool's shape, so
it is defined where it is used rather than aliased from the device header. And
`records()` sorts: a buffer holds one producer's records, so concatenating buffers
groups by producer, and sorting at read time -- off any measured path -- gives
every reader one well-defined order.
The buffer count must exceed the producer count or a producer is denied a lane, so
it is 24 (the recorder pool caps at GRAPH_MAX_DEFINITIONS = 16 threads, plus the
submitting thread, plus headroom for chaining) with 512 records each.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 21, 2026
The host phase pool was built for one writer -- it is the device pool's shape,
where the AICPU alone advances a cursor and the host reads -- and host recording
became multi-threaded under it. A global mutex made that safe: every producer
serialized on one lock to append into one active buffer, and every producer's
counter update landed on one cache line per kind.
The producers are independent. A recording thread records its own Definition and
counts only its own operations; nothing about a record needs coordination. So the
lock and the shared counters bought nothing, and they cost the emitting thread
3.3 us per record at eight producers against 0.1 us at one. All of it runs after
the record's own end timestamp, so it was invisible as anything except a gap
between two consecutive records -- which is exactly how it presented: a swimlane
whose recording lanes looked 45-66% idle, with the idle time attributed to the
workload.
Each producer now claims a whole buffer and fills it alone, so its records are
contiguous and no cache line is shared, and its counters and in-flight claim are
its own lines. The per-record path has no shared write left; the one atomic is a
buffer claim taken once per 512 records. Measured on a 320-core aarch64 host at
20k records per thread:
producers mutex + shared counters per-producer lanes
1 122 ns 48 ns
4 1718 ns 50 ns
8 3233 ns 49 ns
16 7321 ns 49 ns
Flat rather than merely lower, and 48 ns is what the two clock_gettime calls cost
on their own, so the record itself is now free. An intermediate form -- one atomic
slot claim per record into a flat array -- still cost 1.1 us at eight producers,
because a 40-byte record leaves adjacent slots owned by different threads sharing
a line; that is why the buffers are per producer and not merely lock-free.
On dsv4's seven-Definition Graph form, a2a3, `--rounds 5` with the device run
skipped, minimum over the 8 warm passes:
host_orch 1.291 ms -> 0.828 ms -36%
control-plane total 2.998 ms -> 2.379 ms -21%
and in the per-event view the gap between two `record_node` records falls from
1.96-2.30 us to 0.25-0.68 us on every lane. The orchestration itself is unchanged;
this is the instrument getting out of the way of what it was measuring. It also
removes the reason hw-native-sys#1937 gives for wanting the per-event pool separately
switchable -- that path no longer appends under a lock -- though having one switch
per view is worth keeping on its own terms.
Two consequences worth naming. The pool is no longer the device pool's shape, so
it is defined where it is used rather than aliased from the device header. And
`records()` sorts: a buffer holds one producer's records, so concatenating buffers
groups by producer, and sorting at read time -- off any measured path -- gives
every reader one well-defined order.
The buffer count must exceed the producer count or a producer is denied a lane, so
it is 24 (the recorder pool caps at GRAPH_MAX_DEFINITIONS = 16 threads, plus the
submitting thread, plus headroom for chaining) with 512 records each.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 21, 2026
Every number in the DSv4 Graph work came from assembling the same handful of switches by hand, and each of the ways to get that wrong is silent. `--rounds > 1` force-disables every diagnostic, so asking for many rounds and a swimlane in one run yields no artifact and no warning. An L3 case run through the module runner's own L3 phase has its child output captured, so the log holds zero `bind phase=` lines and the run still passes. Summing per-phase minima produces a total no pass achieved, and it can invert the sign of a branch comparison. `hbg_bind_measure.sh` has the two shapes as two modes -- `-m stats` for per-phase minima over many rounds, `-m swimlane` for one round plus a timeline -- and makes them mutually exclusive by construction rather than by remembering. Since hw-native-sys#1937 each view also has its own switch, and the two modes need different ones: setting only `SIMPLER_HBG_BIND_BREAKDOWN_ENABLE`, which used to arm both, leaves swimlane mode with no per-event pool and so no artifact. The script sets what its mode needs. Both modes skip the device, so a case whose device execution does not complete still yields its whole host picture and no device time is spent waiting for it to fail. `hbg_phase_stats.py` holds the grouping: `arena_h2d` closes a pass, the control-plane total is summed within a pass before any minimum is taken, and the first pass of each rank is warm-up -- which is why the caller passes `--rounds`, so the parser can infer the rank count and drop one cold pass per rank rather than one in total. It also prints the spread, because a change that lowers the floor while widening the range has made the cost less predictable, and reading only the minimum hides that. Both live in `tools/`, which is not shipped in the wheel, next to `benchmark_rounds.sh`. `docs/dfx/hbg-bind-measurement.md` keeps the recipes -- they are what explains the switches -- and now points at the script; its description of the pool loses the word "rotating", which per-producer buffers made false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 21, 2026
The host phase pool was built for one writer -- it is the device pool's shape,
where the AICPU alone advances a cursor and the host reads -- and host recording
became multi-threaded under it. A global mutex made that safe: every producer
serialized on one lock to append into one active buffer, and every producer's
counter update landed on one cache line per kind.
The producers are independent. A recording thread records its own Definition and
counts only its own operations; nothing about a record needs coordination. So the
lock and the shared counters bought nothing, and they cost the emitting thread
3.3 us per record at eight producers against 0.1 us at one. All of it runs after
the record's own end timestamp, so it was invisible as anything except a gap
between two consecutive records -- which is exactly how it presented: a swimlane
whose recording lanes looked 45-66% idle, with the idle time attributed to the
workload.
Each producer now claims a whole buffer and fills it alone, so its records are
contiguous and no cache line is shared, and its counters and in-flight claim are
its own lines. The per-record path has no shared write left; the one atomic is a
buffer claim taken once per 512 records. Measured on a 320-core aarch64 host at
20k records per thread:
producers mutex + shared counters per-producer lanes
1 122 ns 48 ns
4 1718 ns 50 ns
8 3233 ns 49 ns
16 7321 ns 49 ns
Flat rather than merely lower, and 48 ns is what the two clock_gettime calls cost
on their own, so the record itself is now free. An intermediate form -- one atomic
slot claim per record into a flat array -- still cost 1.1 us at eight producers,
because a 40-byte record leaves adjacent slots owned by different threads sharing
a line; that is why the buffers are per producer and not merely lock-free.
Plain integers in a lane's counters are sound only because a lane has exactly one
writer for a whole pass, so two things enforce that rather than assume it. There
is no shared fallback lane: a producer that cannot get a lane of its own does not
record, and the loss is counted alongside the pool's. And a pass identity is drawn
from a process-wide counter, not per pool -- a producer caches its lane against
the identity of the pass it claimed it for, and since a store is per DeviceRunner,
a new store can occupy a destroyed one's address, where a per-pool count would
hand it an identity the cached lane already matches and the producer would resume
the dead store's offset.
A pass can also close and reopen between claiming a lane and being admitted, and
the new pass re-hands-out lanes, so a record re-checks the identity once admitted
and withdraws if it moved.
On dsv4's seven-Definition Graph form, a2a3, `--rounds 5` with the device run
skipped, minimum over the 8 warm passes:
host_orch 1.291 ms -> 0.828 ms -36%
control-plane total 2.998 ms -> 2.379 ms -21%
and in the per-event view the gap between two `record_node` records falls from
1.96-2.30 us to 0.31-0.68 us on every lane. The orchestration itself is unchanged;
this is the instrument getting out of the way of what it was measuring. It also
removes the reason hw-native-sys#1937 gives for wanting the per-event pool separately
switchable -- that path no longer appends under a lock -- though having one switch
per view is worth keeping on its own terms.
Two consequences worth naming. The pool is no longer the device pool's shape, so
it is defined where it is used rather than aliased from the device header. And
`records()` sorts: a buffer holds one producer's records, so concatenating buffers
groups by producer, and sorting at read time -- off any measured path -- gives
every reader one well-defined order.
The buffer count must exceed the producer count or a producer is denied a lane, so
it is 24 (the recorder pool caps at GRAPH_MAX_DEFINITIONS = 16 threads, plus the
submitting thread, plus headroom for chaining) with 512 records each.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
that referenced
this pull request
Aug 21, 2026
The host phase pool was built for one writer -- it is the device pool's shape,
where the AICPU alone advances a cursor and the host reads -- and host recording
became multi-threaded under it. A global mutex made that safe: every producer
serialized on one lock to append into one active buffer, and every producer's
counter update landed on one cache line per kind.
The producers are independent. A recording thread records its own Definition and
counts only its own operations; nothing about a record needs coordination. So the
lock and the shared counters bought nothing, and they cost the emitting thread
3.3 us per record at eight producers against 0.1 us at one. All of it runs after
the record's own end timestamp, so it was invisible as anything except a gap
between two consecutive records -- which is exactly how it presented: a swimlane
whose recording lanes looked 45-66% idle, with the idle time attributed to the
workload.
Each producer now claims a whole buffer and fills it alone, so its records are
contiguous and no cache line is shared, and its counters and in-flight claim are
its own lines. The per-record path has no shared write left; the one atomic is a
buffer claim taken once per 512 records. Measured on a 320-core aarch64 host at
20k records per thread:
producers mutex + shared counters per-producer lanes
1 122 ns 48 ns
4 1718 ns 50 ns
8 3233 ns 49 ns
16 7321 ns 49 ns
Flat rather than merely lower, and 48 ns is what the two clock_gettime calls cost
on their own, so the record itself is now free. An intermediate form -- one atomic
slot claim per record into a flat array -- still cost 1.1 us at eight producers,
because a 40-byte record leaves adjacent slots owned by different threads sharing
a line; that is why the buffers are per producer and not merely lock-free.
Plain integers in a lane's counters are sound only because a lane has exactly one
writer for a whole pass, so two things enforce that rather than assume it. There
is no shared fallback lane: a producer that cannot get a lane of its own does not
record, and the loss is counted alongside the pool's. And a pass identity is drawn
from a process-wide counter, not per pool -- a producer caches its lane against
the identity of the pass it claimed it for, and since a store is per DeviceRunner,
a new store can occupy a destroyed one's address, where a per-pool count would
hand it an identity the cached lane already matches and the producer would resume
the dead store's offset.
A pass can also close and reopen between claiming a lane and being admitted, and
the new pass re-hands-out lanes, so a record re-checks the identity once admitted
and withdraws if it moved.
On dsv4's seven-Definition Graph form, a2a3, `--rounds 5` with the device run
skipped, minimum over the 8 warm passes:
host_orch 1.291 ms -> 0.828 ms -36%
control-plane total 2.998 ms -> 2.379 ms -21%
and in the per-event view the gap between two `record_node` records falls from
1.96-2.30 us to 0.31-0.68 us on every lane. The orchestration itself is unchanged;
this is the instrument getting out of the way of what it was measuring. It also
removes the reason #1937 gives for wanting the per-event pool separately
switchable -- that path no longer appends under a lock -- though having one switch
per view is worth keeping on its own terms.
Two consequences worth naming. The pool is no longer the device pool's shape, so
it is defined where it is used rather than aliased from the device header. And
`records()` sorts: a buffer holds one producer's records, so concatenating buffers
groups by producer, and sorting at read time -- off any measured path -- gives
every reader one well-defined order.
The buffer count must exceed the producer count or a producer is denied a lane, so
it is 24 (the recorder pool caps at GRAPH_MAX_DEFINITIONS = 16 threads, plus the
submitting thread, plus headroom for chaining) with 512 records each.
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 21, 2026
Every number in the DSv4 Graph work came from assembling the same handful of switches by hand, and each of the ways to get that wrong is silent. `--rounds > 1` force-disables every diagnostic, so asking for many rounds and a swimlane in one run yields no artifact and no warning. An L3 case run through the module runner's own L3 phase has its child output captured, so the log holds zero `bind phase=` lines and the run still passes. Summing per-phase minima produces a total no pass achieved, and it can invert the sign of a branch comparison. `hbg_bind_measure.sh` has the two shapes as two modes -- `-m stats` for per-phase minima over many rounds, `-m swimlane` for one round plus a timeline -- and makes them mutually exclusive by construction rather than by remembering. Since hw-native-sys#1937 each view also has its own switch, and the two modes need different ones: setting only `SIMPLER_HBG_BIND_BREAKDOWN_ENABLE`, which used to arm both, leaves swimlane mode with no per-event pool and so no artifact. The script sets what its mode needs. Both modes skip the device, so a case whose device execution does not complete still yields its whole host picture and no device time is spent waiting for it to fail. `hbg_phase_stats.py` holds the grouping: `arena_h2d` closes a pass, the control-plane total is summed within a pass before any minimum is taken, and the first pass of each rank is warm-up -- which is why the caller passes `--rounds`, so the parser can infer the rank count and drop one cold pass per rank rather than one in total. It also prints the spread, because a change that lowers the floor while widening the range has made the cost less predictable, and reading only the minimum hides that. Both live in `tools/`, which is not shipped in the wheel, next to `benchmark_rounds.sh`. `docs/dfx/hbg-bind-measurement.md` keeps the recipes -- they are what explains the switches -- and now points at the script; its description of the pool loses the word "rotating", which per-producer buffers made false. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 21, 2026
Every number in the DSv4 Graph work came from assembling the same handful of switches by hand, and each of the ways to get that wrong is silent. `--rounds > 1` force-disables every diagnostic, so asking for many rounds and a swimlane in one run yields no artifact and no warning. An L3 case run through the module runner's own L3 phase has its child output captured, so the log holds zero `bind phase=` lines and the run still passes. Summing per-phase minima produces a total no pass achieved, and it can invert the sign of a branch comparison. `hbg_bind_measure.sh` has the two shapes as two modes -- `-m stats` for per-phase minima over many rounds, `-m swimlane` for one round plus a timeline -- and makes them mutually exclusive by construction rather than by remembering. Since hw-native-sys#1937 each view also has its own switch, and the two modes need different ones: setting only `SIMPLER_HBG_BIND_BREAKDOWN_ENABLE`, which used to arm both, leaves swimlane mode with no per-event pool and so no artifact. The script sets what its mode needs. Both modes skip the device, so a case whose device execution does not complete still yields its whole host picture and no device time is spent waiting for it to fail. It also stamps its whole environment as the log's first line, and the parser echoes that above the table. Having a script is not enough on its own: a hand-assembled baseline compared against a scripted measurement produced a wrong number once, because the baseline lacked `TORCH_DEVICE_BACKEND_AUTOLOAD=0` and so alone paid for `torch_npu` grabbing a device on import, and the gap was credited to the branch. `diff <(head -1 a.log) <(head -1 b.log)` now answers "same ruler?" instead of leaving it to memory, and a table pasted out of the parser carries the conditions it was measured under. `hbg_phase_stats.py` holds the grouping: `arena_h2d` closes a pass, the control-plane total is summed within a pass before any minimum is taken, and the first pass of each rank is warm-up -- which is why the caller passes `--rounds`, so the parser can infer the rank count and drop one cold pass per rank rather than one in total. It also prints the spread, because a change that lowers the floor while widening the range has made the cost less predictable, and reading only the minimum hides that. Both live in `tools/`, which is not shipped in the wheel, next to `benchmark_rounds.sh`. `docs/dfx/hbg-bind-measurement.md` keeps the recipes -- they are what explains the switches -- and now points at the script and at the stamp check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 21, 2026
Every number in the DSv4 Graph work came from assembling the same handful of switches by hand, and each of the ways to get that wrong is silent. `--rounds > 1` force-disables every diagnostic, so asking for many rounds and a swimlane in one run yields no artifact and no warning. An L3 case run through the module runner's own L3 phase has its child output captured, so the log holds zero `bind phase=` lines and the run still passes. Summing per-phase minima produces a total no pass achieved, and it can invert the sign of a branch comparison. `hbg_bind_measure.sh` has the two shapes as two modes -- `-m stats` for per-phase minima over many rounds, `-m swimlane` for one round plus a timeline -- and makes them mutually exclusive by construction rather than by remembering. Since hw-native-sys#1937 each view also has its own switch, and the two modes need different ones: setting only `SIMPLER_HBG_BIND_BREAKDOWN_ENABLE`, which used to arm both, leaves swimlane mode with no per-event pool and so no artifact. The script sets what its mode needs. Both modes skip the device, so a case whose device execution does not complete still yields its whole host picture and no device time is spent waiting for it to fail. It also stamps its whole environment as the log's first line, and the parser echoes that above the table. Having a script is not enough on its own: a hand-assembled baseline compared against a scripted measurement produced a wrong number once, because the baseline lacked `TORCH_DEVICE_BACKEND_AUTOLOAD=0` and so alone paid for `torch_npu` grabbing a device on import, and the gap was credited to the branch. `diff <(head -1 a.log) <(head -1 b.log)` now answers "same ruler?" instead of leaving it to memory, and a table pasted out of the parser carries the conditions it was measured under. `hbg_phase_stats.py` holds the grouping: `arena_h2d` closes a pass, the control-plane total is summed within a pass before any minimum is taken, and the first pass of each rank is warm-up -- which is why the caller passes `--rounds`, so the parser can infer the rank count and drop one cold pass per rank rather than one in total. It also prints the spread, because a change that lowers the floor while widening the range has made the cost less predictable, and reading only the minimum hides that. The control-plane set is not fixed, so the total is summed over the phases a run actually has and the absent ones are named. A change can retire a phase outright -- folding the shared-memory image into the arena copy removes `relocate` and `sm_h2d` -- and that is a saving, not a parse failure; refusing to total such a run makes it uncomparable to the run it should be compared against. Absent from only *some* passes is the different case, a truncated log rather than a retired phase, and is warned about separately because it would understate the run. Both live in `tools/`, which is not shipped in the wheel, next to `benchmark_rounds.sh`. `docs/dfx/hbg-bind-measurement.md` keeps the recipes -- they are what explains the switches -- and now points at the script and at the stamp check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 21, 2026
ChaoWao
added a commit
to ChaoWao/simpler-fork
that referenced
this pull request
Aug 21, 2026
Every number in the recent host_build_graph work came from assembling the same handful of switches by hand, and each of the ways to get that wrong is silent. `--rounds > 1` force-disables every diagnostic, so asking for many rounds and a swimlane in one run yields no artifact and no warning. A multi-device case run through the module runner has its child output captured, so the log holds zero `bind phase=` lines and the run still passes. Summing per-phase minima produces a total no pass achieved, and it can invert the sign of a branch comparison. `tools/hbg_bind_measure.sh` has the two shapes as two modes -- `-m stats` for per-phase minima over many rounds, `-m swimlane` for one round plus a timeline -- and makes them mutually exclusive by construction rather than by remembering. Since hw-native-sys#1937 each view also has its own switch, and the two modes need different ones: the breakdown alone leaves swimlane mode with no per-event pool and so no artifact. Both modes skip the device, so a case whose device execution does not complete still yields its whole host picture, and that is also why the script passes `--skip-golden` -- the outputs a golden check would compare are never produced, so a checking case such as qwen otherwise fails at validation with the whole bind measurement already sitting complete in the log. The script stamps its whole environment as the log's first line, which is what lets `diff <(head -1 a.log) <(head -1 b.log)` answer "same ruler?": a hand-assembled baseline lacking `TORCH_DEVICE_BACKEND_AUTOLOAD=0` paid for `torch_npu` grabbing a device that the other arm did not, and the gap was credited to the branch. `tools/hbg_phase_stats.py` holds the grouping: `arena_h2d` closes a pass, the control-plane total is summed within a pass before any minimum is taken, and the first pass of each rank is warm-up -- which is why the caller passes `--rounds`, so the parser infers the rank count and drops one cold pass per rank rather than one in total. The control-plane set is not fixed, since a change can retire a phase outright, so the total covers the phases a run actually has and names the ones absent from every pass; absent from only *some* passes is a truncated log instead, and is warned about separately. The four surfaces divide by reader rather than restating each other. `docs/dfx/hbg-bind-measurement.md` owns what the phases are, why each trap exists, and the comparison methodology, and it now carries both per-case invocations in Recipe A in place of the hand-assembled ones it had; a note at its top says where a new lesson goes, so the next one does not land on all four surfaces at once. The `measure-bind-path` skill is the operational path only -- the two commands, the mode and case choice, and what to check before quoting a number -- since it enters context on every invocation. `tools/README.md` indexes each tool in a few lines and links onward. The tools carry a comment only where a rule has become an invariant their code depends on. Two things the recipe needed were wrong or unstated. `level=3` was claimed for both cases; qwen is `level=2`, and the level is what decides whether the child command has to be built at all, so it is now a row in the case table and the reason `-l ''` appears in qwen's invocation. And the doc calls six rounds the working minimum, so six is the script's default and the count in every example. Both tools live in `tools/`, which is not shipped in the wheel, next to `benchmark_rounds.sh`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SIMPLER_HBG_BIND_BREAKDOWN_ENABLEgated two unrelated things: the per-kindLOG_TIMINGbreakdown, and — as the producer's arming condition — the per-eventrecord pool behind
host_phase_records.jsonl. A run that wanted the summed lineswhile some other diagnostic flag was on had no way to decline the artifact, and
the per-event path is the one that appends to a pool under a lock on the path
being measured.
SIMPLER_HBG_HOST_PHASE_RECORDS_ENABLE, so each of the three views has exactlyone switch.
host_phase_timing_enabled()→host_phase_breakdown_enabled(), which is whatit now decides alone; the shared
1/t/Topt-in parsing moves into onehelper.
level of
ORCH_PHASES, and writing the artifact is unchanged — a collectedpass reaches the JSONL whenever the run has an output directory, so a level-4
run still produces both. What changes is that the breakdown alone no longer
arms anything.
bind phase=lineshost_phase_records.jsonlSIMPLER_HBG_BIND_BREAKDOWN_ENABLE=1SIMPLER_HBG_HOST_PHASE_RECORDS_ENABLE=1--enable-chip-swimlane 4The one behavior change for existing users is the second row: Recipe B in
docs/dfx/hbg-bind-measurement.mdnow exports both variables, and its conditionlist and traps table say which switch owns the artifact.
Docs updated in the same commit: both copies of the runtime's
profiling_levels.md("The two switches" → three, and the arming rule) anddocs/dfx/hbg-bind-measurement.md.Testing
Onboard a2a3,
tests/st/a2a3/host_build_graph/vector_example, one run perconfiguration through
task-submit, re-run after the rename against therebuilt
.so:breakdown variable alone → 12
bind phase=lines + 1host-orch phase=line,no
host_phase_records.jsonlrecords variable alone → JSONL with 16 records, no breakdown lines
--enable-chip-swimlane 4, both variables unset → JSONL andchip_swimlane_records.jsonneither switch, no diagnostic flag → no output directory at all
Hardware tests pass — hbg scene tests on a2a3: 8 L3 cases, 36 passed /
1 skipped (L2
host_build_graph), 2 passed (L2tensormap_and_ringbuffer)Simulation tests — left to CI