diff --git a/src/a2a3/runtime/host_build_graph/docs/SCALAR_DATA_ACCESS.md b/src/a2a3/runtime/host_build_graph/docs/SCALAR_DATA_ACCESS.md index c71dfb5036..e6b75b8e02 100644 --- a/src/a2a3/runtime/host_build_graph/docs/SCALAR_DATA_ACCESS.md +++ b/src/a2a3/runtime/host_build_graph/docs/SCALAR_DATA_ACCESS.md @@ -11,7 +11,7 @@ external tensors; they do not interleave host code with device execution. | ------------ | ----------------- | ----------------- | | External tensor with no submitted producer | Reads the staged host value | Updates the staged host value | | External control/output tensor not referenced by a task | Reads immediately | Writes immediately | -| Runtime-created output | Unsupported: no registered host view | Unsupported: no registered host view | +| Runtime-created output | Unsupported: not a staged tensor | Unsupported: not a staged tensor. Set the value with `TensorCreateInfo::set_initial_value` instead | | Tensor owned by a submitted device task | Unsupported during graph construction | Unsupported during graph construction | | Tensor with an invalid or stale owner task ID | Fails with `INVALID_ARGS` | Fails with `INVALID_ARGS` | @@ -46,9 +46,54 @@ for that producer from the orchestration call cannot make progress. The runtime keeps a timeout as a defensive failure backstop, but it is not a supported synchronization mechanism. -Runtime-created output buffers also live in the graph heap and have no host-view -registration. Even if their address is nonzero, host orchestration must not -dereference or modify them. +Runtime-created output buffers live in the GM heap rather than among the staged +tensors, so `get_tensor_data` / `set_tensor_data` do not resolve them. Host +orchestration must never dereference such an address itself; the initial-value +path below is the one way it reaches those bytes. + +## Initial Values on a Runtime Allocation + +`TensorCreateInfo::set_initial_value(v)` fills the whole buffer the runtime +allocates for that output, before any task can observe it. It is the way to give +a runtime-created buffer a defined starting content — a fixed-size tile whose +producer writes only a prefix, for instance, so its consumer reads `v` in the +remainder rather than whatever the heap last held. + +```cpp +TensorCreateInfo tile(shape, 2, DataType::INT8); +tile.set_initial_value(0); +TaskOutputTensors outs = alloc_tensors(tile); +``` + +The fill is a host-side write reaching device memory, so how it gets there +depends on the platform, but the semantics do not: + +| Platform | How the bytes reach the heap | +| -------- | ---------------------------- | +| a2a3 onboard | The heap is mapped into the host address space (`halHostRegister(DEV_SVM_MAP_HOST)`) and the store lands on it directly — no copy | +| a5 onboard | No host-map path, so the pattern is staged in a bounded buffer and pushed with `copy_to_device` — the same mechanism `set_tensor_data` already uses there | +| simulation | A device pointer is already a host pointer | + +The mapping is made by the first fill that needs it, not at bind: the heap runs +to hundreds of MB and a run that sets no initial value never touches it, so an +orchestration that uses none costs nothing here. + +The write lands at the call that asked for it, never deferred: the heap is +reclaimed and re-let within one orchestration, so two fills can name the same +address and only their issue order is correct. + +One constraint follows from *when* the fill happens: + +- **A Graph body must not ask for one.** Recording addresses an internal node's + outputs in the private range based at `GRAPH_RECORD_VIRTUAL_BASE`, and every + submission of the resulting Definition materializes its own from a heap block + whose prior contents it never reads — so a value written during recording + reaches none of the replays. The recording is marked unsupported and + `graph_commit` latches `PTO2_ERROR_INVALID_ARGS`; the body is not re-run on the + ordinary path, because its outer shell was published before the body was + recorded (see [GRAPH_EXECUTION.md](../../../../common/host_build_graph/docs/GRAPH_EXECUTION.md)). + Allocate the buffer outside the Graph body and pass it across the boundary, or + have a task write the padding the consumer relies on. ## Ownership Validation @@ -71,3 +116,5 @@ recorded. as a device synchronization barrier. - Pass values needed for graph construction as orchestration inputs or scalars. - Keep device-produced values on the device or return them after the run. +- Give a runtime allocation its starting content with + `TensorCreateInfo::set_initial_value`, not with `set_tensor_data`. diff --git a/src/a2a3/runtime/host_build_graph/host/host_tensor_access.cpp b/src/a2a3/runtime/host_build_graph/host/host_tensor_access.cpp index 973b934030..62d849fe89 100644 --- a/src/a2a3/runtime/host_build_graph/host/host_tensor_access.cpp +++ b/src/a2a3/runtime/host_build_graph/host/host_tensor_access.cpp @@ -25,29 +25,51 @@ struct HostTensorRegion { uint64_t dev_base; uint64_t size; + // Null on a GM-heap region until its first fill, and after one whose mapping + // attempt failed. That region is write-only, so `fill` can stage the bytes + // elsewhere and push them; every other region is read back through this view + // and is mapped when it is registered. unsigned char *host_view; // The host view is a copy of the device bytes rather than the device bytes // themselves, so a write reaches the device only once pushed back. bool mirrored; + // Whether a mapping has been attempted for this region. A GM-heap region is + // registered without one — mapping the whole heap costs a `halHostRegister` + // over hundreds of MB (2 GiB on the dsv4 case) that a run setting no initial + // value would never use — so the attempt is deferred to the first fill and, + // either way, made once. + bool map_attempted; + HostTensorRegionKind kind; }; -// One entry per tensor staged for the run being orchestrated. A run stages a -// handful of tensors and orchestration reads are cold-path, so a linear scan -// costs less than the map that would replace it. +// Bytes of the buffer `fill` stages a viewless region's pattern in. One period +// of the pattern is at most 8 bytes, so this holds thousands of repeats and the +// same buffer serves every chunk of every fill in the run. +constexpr uint64_t kFillStagingBytes = 64 * 1024; + +// One entry per tensor staged for the run being orchestrated, plus at most one +// for the GM heap. A run stages a handful of tensors and orchestration reads +// are cold-path, so a linear scan costs less than the map that would replace it. struct HostTensorAccessor::Impl { const HostApi *api; std::vector regions; std::vector mappings; // Bytes covered by `mappings`, i.e. excluding regions serving a fallback view. uint64_t mapped_bytes; + // Allocated on the first fill of a viewless region, and only then: a + // platform that maps the heap never needs it. + std::vector fill_staging; }; -// The region serving the whole of [dev_addr, dev_addr + bytes), or nullptr. -// `*offset` is the span's distance from that region's base. -const HostTensorRegion * -find_region(const std::vector ®ions, uint64_t dev_addr, uint64_t bytes, uint64_t *offset) { - for (const HostTensorRegion ®ion : regions) { - if (dev_addr < region.dev_base) { +// The `kind` region serving the whole of [dev_addr, dev_addr + bytes), or +// nullptr. `*offset` is the span's distance from that region's base. Non-const +// because a GM-heap region binds its host view on the first fill that needs it. +HostTensorRegion *find_region( + std::vector ®ions, HostTensorRegionKind kind, uint64_t dev_addr, uint64_t bytes, + uint64_t *offset +) { + for (HostTensorRegion ®ion : regions) { + if (region.kind != kind || dev_addr < region.dev_base) { continue; } uint64_t off = dev_addr - region.dev_base; @@ -61,7 +83,7 @@ find_region(const std::vector ®ions, uint64_t dev_addr, uin } HostTensorAccessor::HostTensorAccessor(const HostApi *api) : - impl_(new Impl{api, {}, {}, 0}) {} + impl_(new Impl{api, {}, {}, 0, {}}) {} HostTensorAccessor::~HostTensorAccessor() { close(); @@ -69,6 +91,16 @@ HostTensorAccessor::~HostTensorAccessor() { } bool HostTensorAccessor::add(uint64_t dev_base, uint64_t size, void *fallback_host_view) { + return add_region(dev_base, size, fallback_host_view, HostTensorRegionKind::StagedTensor); +} + +bool HostTensorAccessor::add_heap(uint64_t dev_base, uint64_t size) { + return add_region(dev_base, size, nullptr, HostTensorRegionKind::GmHeap); +} + +bool HostTensorAccessor::add_region( + uint64_t dev_base, uint64_t size, void *fallback_host_view, HostTensorRegionKind kind +) { if (impl_->api == nullptr || dev_base == 0 || size == 0) { return false; } @@ -76,6 +108,14 @@ bool HostTensorAccessor::add(uint64_t dev_base, uint64_t size, void *fallback_ho // first: a reallocation that threw there would leak the mapping. impl_->regions.reserve(impl_->regions.size() + 1); impl_->mappings.reserve(impl_->mappings.size() + 1); + // A staged tensor is read through its view, so it is mapped now and must end + // up with one. The GM heap is only ever written, by `fill`, so its mapping + // waits for a fill to need it and the common run — which sets no initial + // value — registers the region without touching the heap at all. + if (kind == HostTensorRegionKind::GmHeap) { + impl_->regions.push_back({dev_base, size, nullptr, false, false, kind}); + return true; + } void *host_view = impl_->api->register_device_memory_to_host(reinterpret_cast(dev_base), size); if (host_view != nullptr) { impl_->mappings.push_back(reinterpret_cast(dev_base)); @@ -86,15 +126,35 @@ bool HostTensorAccessor::add(uint64_t dev_base, uint64_t size, void *fallback_ho if (host_view == nullptr) { return false; } - impl_->regions.push_back( - {dev_base, size, static_cast(host_view), reinterpret_cast(host_view) != dev_base} - ); + const bool mirrored = reinterpret_cast(host_view) != dev_base; + impl_->regions.push_back({dev_base, size, static_cast(host_view), mirrored, true, kind}); return true; } +// Map `region` if that has not been tried yet, so a heap that no fill reaches is +// never mapped. Leaves `host_view` null when the platform has no host-map path, +// which is the caller's cue to stage and push instead. +void HostTensorAccessor::ensure_region_mapped(HostTensorRegion *region) const { + if (region->map_attempted) { + return; + } + region->map_attempted = true; + impl_->mappings.reserve(impl_->mappings.size() + 1); + void *host_view = + impl_->api->register_device_memory_to_host(reinterpret_cast(region->dev_base), region->size); + if (host_view == nullptr) { + return; + } + impl_->mappings.push_back(reinterpret_cast(region->dev_base)); + impl_->mapped_bytes += region->size; + region->host_view = static_cast(host_view); + region->mirrored = reinterpret_cast(host_view) != region->dev_base; +} + bool HostTensorAccessor::read(uint64_t dev_addr, void *dst, uint64_t bytes) const { uint64_t offset = 0; - const HostTensorRegion *region = find_region(impl_->regions, dev_addr, bytes, &offset); + const HostTensorRegion *region = + find_region(impl_->regions, HostTensorRegionKind::StagedTensor, dev_addr, bytes, &offset); if (region == nullptr) { return false; } @@ -104,7 +164,8 @@ bool HostTensorAccessor::read(uint64_t dev_addr, void *dst, uint64_t bytes) cons bool HostTensorAccessor::write(uint64_t dev_addr, const void *src, uint64_t bytes) const { uint64_t offset = 0; - const HostTensorRegion *region = find_region(impl_->regions, dev_addr, bytes, &offset); + const HostTensorRegion *region = + find_region(impl_->regions, HostTensorRegionKind::StagedTensor, dev_addr, bytes, &offset); if (region == nullptr) { return false; } @@ -116,6 +177,75 @@ bool HostTensorAccessor::write(uint64_t dev_addr, const void *src, uint64_t byte return impl_->api->copy_to_device(reinterpret_cast(dev_addr), dst, static_cast(bytes)) == 0; } +// Fill a span of a region that has no host view, by staging one buffer of the +// repeating pattern and pushing it across the span. +// +// The chunk is a whole number of elements, so every push starts on a pattern +// boundary and the staged bytes can be reused verbatim. A span that is not a +// whole number of elements ends on a partial one, which is what a buffer whose +// size is not a multiple of its element width should get. +bool HostTensorAccessor::fill_staged(uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) const { + const uint64_t chunk = (kFillStagingBytes / elem_size) * elem_size; + std::vector &staging = impl_->fill_staging; + // The buffer is reused across fills but the pattern is not: consecutive + // fills differ in value and element width, so it is laid down per call. + staging.resize(static_cast(chunk)); + for (uint64_t written = 0; written < chunk; written += elem_size) { + memcpy(staging.data() + written, &value, static_cast(elem_size)); + } + for (uint64_t pushed = 0; pushed < bytes;) { + const uint64_t span = (bytes - pushed < chunk) ? bytes - pushed : chunk; + if (impl_->api->copy_to_device( + reinterpret_cast(dev_addr + pushed), staging.data(), static_cast(span) + ) != 0) { + return false; + } + pushed += span; + } + return true; +} + +HostTensorFillStatus +HostTensorAccessor::fill(uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) const { + if (bytes == 0) { + return HostTensorFillStatus::Ok; + } + if (elem_size == 0 || elem_size > sizeof(value)) { + return HostTensorFillStatus::BadElementSize; + } + uint64_t offset = 0; + HostTensorRegion *region = find_region(impl_->regions, HostTensorRegionKind::GmHeap, dev_addr, bytes, &offset); + if (region == nullptr) { + return HostTensorFillStatus::NoRegion; + } + ensure_region_mapped(region); + if (region->host_view == nullptr) { + // No host-map path on this platform: stage the pattern and push it. + return fill_staged(dev_addr, bytes, value, elem_size) ? HostTensorFillStatus::Ok : + HostTensorFillStatus::PushFailed; + } + // Seed one element, then double the written prefix into the remainder: the + // span is filled in log2(bytes / elem_size) memcpys rather than one per + // element. A span that is not a whole number of elements ends on a partial + // one, which is what a buffer whose size is not a multiple of its element + // width should get. + unsigned char *dst = region->host_view + offset; + uint64_t seed = (elem_size < bytes) ? elem_size : bytes; + memcpy(dst, &value, static_cast(seed)); + uint64_t filled = seed; + while (filled < bytes) { + uint64_t copy_size = ((bytes - filled) < filled) ? (bytes - filled) : filled; + memcpy(dst + filled, dst, static_cast(copy_size)); + filled += copy_size; + } + if (!region->mirrored) { + return HostTensorFillStatus::Ok; + } + return impl_->api->copy_to_device(reinterpret_cast(dev_addr), dst, static_cast(bytes)) == 0 ? + HostTensorFillStatus::Ok : + HostTensorFillStatus::PushFailed; +} + size_t HostTensorAccessor::mapping_count() const noexcept { return impl_->mappings.size(); } uint64_t HostTensorAccessor::mapped_bytes() const noexcept { return impl_->mapped_bytes; } @@ -136,3 +266,23 @@ bool host_tensor_read(HostTensorAccessor *accessor, uint64_t dev_addr, void *dst bool host_tensor_write(HostTensorAccessor *accessor, uint64_t dev_addr, const void *src, uint64_t bytes) { return accessor != nullptr && accessor->write(dev_addr, src, bytes); } + +HostTensorFillStatus +host_tensor_fill(HostTensorAccessor *accessor, uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) { + if (accessor == nullptr) return HostTensorFillStatus::NoRegion; + return accessor->fill(dev_addr, bytes, value, elem_size); +} + +const char *host_tensor_fill_status_name(HostTensorFillStatus status) { + switch (status) { + case HostTensorFillStatus::Ok: + return "ok"; + case HostTensorFillStatus::BadElementSize: + return "unsupported element width"; + case HostTensorFillStatus::NoRegion: + return "address is not in the GM heap"; + case HostTensorFillStatus::PushFailed: + return "device copy failed"; + } + return "unknown"; +} diff --git a/src/a2a3/runtime/host_build_graph/host/runtime_maker.cpp b/src/a2a3/runtime/host_build_graph/host/runtime_maker.cpp index 8d6892e7f6..a1871cafb9 100644 --- a/src/a2a3/runtime/host_build_graph/host/runtime_maker.cpp +++ b/src/a2a3/runtime/host_build_graph/host/runtime_maker.cpp @@ -525,11 +525,28 @@ struct GraphHostStateBinding { PTO2OrchestratorState &orchestrator; }; +// The accessor lives on run_host_orchestration's frame while `rt` lives in an +// arena that outlives the call, so both pointers to it are cleared on the way +// out rather than left dangling for the next run to find. +struct HostTensorAccessBinding { + HostTensorAccessBinding(PTO2Runtime &rt, HostTensorAccessor &accessor) : + rt(rt) { + rt.tensor_access = &accessor; + rt.orchestrator.tensor_access = &accessor; + } + ~HostTensorAccessBinding() { + rt.tensor_access = nullptr; + rt.orchestrator.tensor_access = nullptr; + } + + PTO2Runtime &rt; +}; + int32_t run_host_orchestration( Runtime *runtime, const HostApi *api, HostTensorAccessor &tensor_access, PTO2Runtime *rt, DeviceArena &host_arena, const PTO2RuntimeArenaLayout &layout, void *device_sm, uint64_t sm_size, void *device_arena, void *gm_heap, - const uint64_t eff_heap_sizes[PTO2_MAX_RING_DEPTH], const uint64_t eff_task_window_sizes[PTO2_MAX_RING_DEPTH], - void *host_orch_func_ptr, const ChipTaskArgs &orch_l2 + uint64_t total_heap_size, const uint64_t eff_heap_sizes[PTO2_MAX_RING_DEPTH], + const uint64_t eff_task_window_sizes[PTO2_MAX_RING_DEPTH], void *host_orch_func_ptr, const ChipTaskArgs &orch_l2 ) { dep_gen_host_graph_begin_capture(); @@ -582,7 +599,15 @@ int32_t run_host_orchestration( return -1; } rt->active_callable_hash = reinterpret_cast(entry_points->entry); - rt->tensor_access = &tensor_access; + HostTensorAccessBinding tensor_access_binding(*rt, tensor_access); + // The GM heap backs every runtime-allocated output, so an initial value the + // orchestration asks for lands here. Registering only records the span; the + // host mapping of it waits for a fill to need one, so a run that sets no + // initial value pays nothing for this. + if (!tensor_access.add_heap(reinterpret_cast(gm_heap), total_heap_size)) { + LOG_ERROR("host-orch: failed to register the GM heap (%" PRIu64 " bytes)", total_heap_size); + return -1; + } // Binds the orchestration .so's own framework_current_runtime, which its // inline rt_submit_* read. The host library links a same-named copy from // orchestration/common.cpp, but nothing outside the .so includes @@ -1043,7 +1068,7 @@ extern "C" int bind_callable_to_runtime_impl( orch_l2.create_from_chip_args(device_args); int32_t total_tasks = run_host_orchestration( runtime, api, tensor_access, rt, host_arena, layout, sm_ptr, sm_size, runtime_arena_dev, gm_heap, - eff_heap_sizes, eff_task_window_sizes, host_orch_func_ptr, orch_l2 + total_heap_size, eff_heap_sizes, eff_task_window_sizes, host_orch_func_ptr, orch_l2 ); // The orchestrator is the only host-view reader; from here the device // owns these buffers, so drop the window on both exits. diff --git a/src/a2a3/runtime/host_build_graph/orchestration/pto_orchestration_api.h b/src/a2a3/runtime/host_build_graph/orchestration/pto_orchestration_api.h index ea4d106108..f57fdda133 100644 --- a/src/a2a3/runtime/host_build_graph/orchestration/pto_orchestration_api.h +++ b/src/a2a3/runtime/host_build_graph/orchestration/pto_orchestration_api.h @@ -518,8 +518,9 @@ static inline T get_tensor_data(const ChipTensor &tensor, uint32_t ndims, const * This API updates the registered host view used to stage an external tensor. * The updated value becomes part of the graph's initial device data. It is not * a synchronization barrier for submitted readers or writers. A tensor with a - * submitted producer, or a runtime-created output with no registered host view, - * is rejected as an invalid argument. + * submitted producer, or a runtime-created output, which is not a staged tensor, + * is rejected as an invalid argument; give a runtime allocation its starting + * content with TensorCreateInfo::set_initial_value instead. */ template static inline void set_tensor_data(const ChipTensor &tensor, uint32_t ndims, const uint32_t indices[], T value) { diff --git a/src/a2a3/runtime/host_build_graph/runtime/host_tensor_access.h b/src/a2a3/runtime/host_build_graph/runtime/host_tensor_access.h index 148feb8f4b..74fbbb82b8 100644 --- a/src/a2a3/runtime/host_build_graph/runtime/host_tensor_access.h +++ b/src/a2a3/runtime/host_build_graph/runtime/host_tensor_access.h @@ -54,6 +54,29 @@ struct HostApi; // common/host_api.h — fwd-declared so this header stays out of platform includes +// What a region backs, and therefore which lookups may resolve to it. The two +// kinds never overlap: a staged tensor is its own device allocation and the GM +// heap is another, so a device address belongs to at most one region. +enum class HostTensorRegionKind : uint8_t { + StagedTensor, + GmHeap, +}; + +// Why a fill did not happen. The cases are distinct enough to lead a reader in +// different directions — a bad element width is a malformed create-info, an +// unresolved span is an address outside the heap, and a failed push is a device +// copy error — so the caller reports which one occurred rather than guessing. +// +// A region without a host view is not among them: that is the ordinary state on +// a platform with no host-map path, where `fill` stages the bytes and pushes +// them, and the outcome is an ordinary Ok or PushFailed. +enum class HostTensorFillStatus : uint8_t { + Ok, + BadElementSize, + NoRegion, + PushFailed, +}; + /** * The registered regions of one orchestration run, and the mappings that run * installed to serve them. @@ -64,14 +87,23 @@ struct HostApi; // common/host_api.h — fwd-declared so this header stays out * are isolated by each owning a separate accessor, which is what makes two runs * unable to see or drop each other's regions. * - * `add` is the only producer of mappings and the only caller of - * `register_device_memory_to_host`; `close` unregisters exactly the mappings + * `add` and `add_heap` are the only producers of mappings and the only callers + * of `register_device_memory_to_host`; `close` unregisters exactly the mappings * this accessor installed and nothing else. Both are reached on every return * path — `close` is idempotent and the destructor calls it — so a mapping * cannot outlive the run that made it. * - * A null `api` makes every `add` fail, so a registered region always implies a - * usable `api`; `write`'s mirror push-back relies on that and does not re-check. + * A null `api` makes every registration fail, so a registered region always + * implies a usable `api`; `write`'s mirror push-back relies on that and does + * not re-check. + * + * Regions come in two kinds and the lookups are kind-scoped. `read` / `write` + * serve the orchestration scalar-access API and see staged-tensor regions only, + * so a device address outside a staged tensor — a GM-heap buffer, a + * pass-through child-memory buffer — resolves to nothing there, which is what + * makes runtime-created outputs unreadable during host orchestration. `fill` + * serves `TensorCreateInfo::set_initial_value`, whose target is always a + * runtime allocation, and sees GM-heap regions only. * * The state lives behind `Impl` because this header is also compiled by the * AICPU build (through `orchestrator_core/pto_runtime2.cpp`, which resolves the @@ -95,9 +127,38 @@ class HostTensorAccessor { * mapping nor a fallback view is available. */ bool add(uint64_t dev_base, uint64_t size, void *fallback_host_view); + + /** + * Register the GM heap `[dev_base, dev_base + size)` as the region serving + * runtime allocations, reachable through `fill` alone. + * + * Recording the span is all this does. Mapping it is deferred to the first + * `fill` that needs one, because the heap runs to hundreds of MB (2 GiB on + * the dsv4 case) and a run that sets no initial value never reads or writes + * it — paying `halHostRegister` over that span at every bind would buy + * nothing. Where no host-map path exists (a5 onboard, whose + * `DeviceRunnerBase` default returns nullptr) the mapping stays absent and + * `fill` stages the bytes and pushes them with `copy_to_device` instead. + */ + bool add_heap(uint64_t dev_base, uint64_t size); + bool read(uint64_t dev_addr, void *dst, uint64_t bytes) const; bool write(uint64_t dev_addr, const void *src, uint64_t bytes) const; + /** + * Fill `[dev_addr, dev_addr + bytes)` with `elem_size`-wide `value`, + * leaving the bytes visible to the device. `bytes` need not be a multiple + * of `elem_size`; the tail is a partial element. + * + * Maps the GM heap on the first call that needs it, so a run that sets no + * initial value never pays for a mapping of the whole heap. + * + * The write lands immediately, never deferred to a flush: the GM heap is + * reclaimed and re-let within one orchestration, so two fills can name the + * same address and only the order they were issued in is correct. + */ + HostTensorFillStatus fill(uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) const; + /** Drop every region and unregister every mapping this accessor installed. */ void close() noexcept; @@ -108,6 +169,10 @@ class HostTensorAccessor { uint64_t mapped_bytes() const noexcept; private: + bool add_region(uint64_t dev_base, uint64_t size, void *fallback_host_view, HostTensorRegionKind kind); + void ensure_region_mapped(struct HostTensorRegion *region) const; + bool fill_staged(uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) const; + struct Impl; Impl *impl_; }; @@ -124,7 +189,19 @@ bool host_tensor_read(HostTensorAccessor *accessor, uint64_t dev_addr, void *dst * Write `bytes` from `src` to device address `dev_addr`, leaving the bytes * visible to the device. * - * @return false when no registered region covers the whole span, or when the + * @return false when no staged-tensor region covers the whole span, or when the * push-back to the device fails. */ bool host_tensor_write(HostTensorAccessor *accessor, uint64_t dev_addr, const void *src, uint64_t bytes); + +/** + * Fill `bytes` at device address `dev_addr` with `elem_size`-wide `value`, + * leaving the bytes visible to the device. + * + * @return what stopped the fill, or `Ok`. + */ +HostTensorFillStatus +host_tensor_fill(HostTensorAccessor *accessor, uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size); + +/** Short, stable label for `status`, for a diagnostic naming what failed. */ +const char *host_tensor_fill_status_name(HostTensorFillStatus status); diff --git a/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/pto_orchestrator.cpp b/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/pto_orchestrator.cpp index cc1c96b904..7df911f609 100644 --- a/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/pto_orchestrator.cpp +++ b/src/a2a3/runtime/host_build_graph/runtime/orchestrator_core/pto_orchestrator.cpp @@ -49,6 +49,7 @@ #include "pto_dep_compute.h" #include "graph_execution.h" #include "graph_host_state.h" +#include "host_tensor_access.h" #include "pto_runtime2_types.h" #include "pto_shared_memory.h" #include "pto_tensormap.h" @@ -79,6 +80,32 @@ __attribute__((weak, visibility("hidden"))) void dep_gen_host_graph_add_tensorma uint64_t, int32_t, const ChipTensor &, const PTO2TensorMapEntry &, OverlapStatus ) {} +// Initial-value fill for a caller that can store to a device address directly. +// It lives here rather than beside the host_tensor_read / host_tensor_write +// fallbacks in pto_runtime2.cpp because apply_initial_values below is its only +// caller, and the unit tests that link this translation unit do not link that +// one. libhost_runtime.so overrides it with host/host_tensor_access.cpp, where +// a device address is not loadable in general. +__attribute__((weak, visibility("hidden"))) HostTensorFillStatus +host_tensor_fill(HostTensorAccessor *, uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) { + if (bytes == 0) return HostTensorFillStatus::Ok; + if (elem_size == 0 || elem_size > sizeof(value)) return HostTensorFillStatus::BadElementSize; + char *dst = reinterpret_cast(dev_addr); + uint64_t seed = (elem_size < bytes) ? elem_size : bytes; + memcpy(dst, &value, seed); + uint64_t filled = seed; + while (filled < bytes) { + uint64_t copy_size = ((bytes - filled) < filled) ? (bytes - filled) : filled; + memcpy(dst + filled, dst, copy_size); + filled += copy_size; + } + return HostTensorFillStatus::Ok; +} + +__attribute__((weak, visibility("hidden"))) const char *host_tensor_fill_status_name(HostTensorFillStatus) { + return "unavailable"; +} + // Scope_stats enable gate, queried via the same predicate idiom as // dep_gen_host_graph_enabled above. The AICPU collector links the strong definition; // host builds fall back to this weak `false`. Gating here still skips the @@ -1141,6 +1168,37 @@ static bool ensure_tensormap_capacity(PTO2OrchestratorState *orch, int32_t neede return false; } +// Write each output create-info's initial value into the buffer just +// materialized for it. `tensors` is the payload's tensor array, parallel to +// `args`, so slot i's buffer address and size are the ones the device will see. +// +// The buffer is in the GM heap, which the host reaches only through the run's +// registered host view; a platform that cannot map the heap latches a fatal +// here rather than storing to a device address. Returns false once it has. +static bool apply_initial_values( + PTO2OrchestratorState *orch, const CoreTaskArgs &args, const ChipTensor *tensors, const char *caller +) { + for (int32_t i = 0; i < args.tensor_count(); i++) { + if (args.tag(i) != TensorArgType::OUTPUT) continue; + const TensorCreateInfo &ci = args.tensor(i).create_info(); + if (!ci.has_initial_value) continue; + const ChipTensor &t = tensors[i]; + const uint64_t elem_size = get_element_size(t.dtype); + const HostTensorFillStatus status = + host_tensor_fill(orch->tensor_access, t.buffer.addr, t.buffer.size, ci.initial_value, elem_size); + if (status != HostTensorFillStatus::Ok) { + orch->report_fatal( + PTO2_ERROR_INVALID_ARGS, caller, + "set_initial_value: %s (addr=%#llx bytes=%llu dtype=%d elem_size=%llu)", + host_tensor_fill_status_name(status), (unsigned long long)t.buffer.addr, + (unsigned long long)t.buffer.size, static_cast(t.dtype), (unsigned long long)elem_size + ); + return false; + } + } + return true; +} + // Shared body for submit_task / submit_dummy_task. Caller has already validated // args.has_error, decided active_mask (empty for dummy), and resolved the per-slot // kernel_ids (all INVALID_KERNEL_ID for dummy). Performs tensormap sync, fanin @@ -1284,6 +1342,9 @@ static TaskOutputTensors submit_task_common( // payload.fanin_local_ids and bumped its last_consumer_local_id; the count is // published in STEP 6 below. payload.init does not touch the fanin region. payload.init(args, result, prepared.alloc_result, layout); + if (!apply_initial_values(orch, args, payload.tensors, __FUNCTION__)) { + return result; + } // Dispatch predicate: resolve the (tensor, indices) to an absolute GM address // now so the scheduler can read it at the dispatch point with a single load, @@ -1693,9 +1754,16 @@ TaskOutputTensors graph_record_submit_node( if (args.tag(i) != TensorArgType::OUTPUT) { slot_tensor.copy(args.tensor(i).ref()); } else { + const TensorCreateInfo &ci = args.tensor(i).create_info(); + // An initial value cannot survive into a replay: recording addresses + // this output in the private range based at GRAPH_RECORD_VIRTUAL_BASE, + // while every submission materializes its own from a heap block whose + // prior contents it never reads. Mark the recording unsupported so + // commit reports it, rather than letting a Definition replay outputs + // the orchestration believes it initialized. + if (ci.has_initial_value) recording.unsupported = true; init_tensor_from_create_info( - slot_tensor, args.tensor(i).create_info(), - reinterpret_cast(packed_base_addr + layout.offsets[i]), layout.buffer_sizes[i] + slot_tensor, ci, reinterpret_cast(packed_base_addr + layout.offsets[i]), layout.buffer_sizes[i] ); slot_tensor.owner_task_id = task_id; } @@ -2216,6 +2284,9 @@ TaskOutputTensors PTO2OrchestratorState::alloc_tensors(const CoreTaskArgs &args) TaskOutputTensors outputs; outputs.set_task_id(prepared.task_id); payload.init(args, outputs, prepared.alloc_result, layout); + if (!apply_initial_values(orch, args, payload.tensors, __FUNCTION__)) { + return TaskOutputTensors{}; + } payload.fanin_count = 0; // hidden-alloc tasks have no producer dependencies CYCLE_COUNT_LAP(g_orch_args_cycle); diff --git a/src/a2a3/runtime/host_build_graph/runtime/pto_orchestrator.h b/src/a2a3/runtime/host_build_graph/runtime/pto_orchestrator.h index df140d0cb6..81bb069b3f 100644 --- a/src/a2a3/runtime/host_build_graph/runtime/pto_orchestrator.h +++ b/src/a2a3/runtime/host_build_graph/runtime/pto_orchestrator.h @@ -39,6 +39,7 @@ #include "pto_types.h" struct GraphHostState; +class HostTensorAccessor; /** * Layout descriptor produced by PTO2OrchestratorState::reserve_layout(). Holds @@ -100,6 +101,13 @@ struct PTO2OrchestratorState { void *gm_heap_base; // Base address of GM heap uint64_t gm_heap_size; // Total size of GM heap (all rings) + // Host views registered for this run, borrowed from PTO2Runtime. The + // orchestrator reaches GM-heap bytes only through this: `gm_heap_base` is a + // device address, loadable by the AICPU but not by the host. Null on the + // AICPU path and whenever the platform cannot map the heap, which makes an + // initial-value fill fail closed rather than store to a device address. + HostTensorAccessor *tensor_access{nullptr}; + // === FATAL ERROR === // Fatal error flag (single-thread access by orchestrator, no atomic needed) // Cross-thread notification uses shared memory orch_error_code (atomic) diff --git a/src/a2a3/runtime/host_build_graph/runtime/tensor_create_info.h b/src/a2a3/runtime/host_build_graph/runtime/tensor_create_info.h index e50cc709e6..4047fa2d12 100644 --- a/src/a2a3/runtime/host_build_graph/runtime/tensor_create_info.h +++ b/src/a2a3/runtime/host_build_graph/runtime/tensor_create_info.h @@ -108,28 +108,15 @@ static_assert(offsetof(TensorCreateInfo, shapes) == offsetof(ChipTensor, shapes) // header) so the create-info dependency stays runtime-only. // ============================================================================ -/// Fill the entire backing buffer of `t` with `initial_value` (doubling memcpy). -inline void fill_tensor_initial_value(ChipTensor &t, uint64_t initial_value) { - always_assert(reinterpret_cast(t.buffer.addr) != nullptr); - uint64_t elem_size = get_element_size(t.dtype); - char *dst = reinterpret_cast(t.buffer.addr); - constexpr uint64_t blk_size = 64; - uint64_t blk = (t.buffer.size < blk_size) ? t.buffer.size : blk_size; - for (uint64_t b = 0; b < blk; b += elem_size) { - memcpy(dst + b, &initial_value, elem_size); - } - uint64_t filled = blk; - while (filled < t.buffer.size) { - uint64_t copy_size = ((t.buffer.size - filled) < filled) ? (t.buffer.size - filled) : filled; - memcpy(dst + filled, dst, copy_size); - filled += copy_size; - } -} - /// Materialize a TensorCreateInfo into `t` (fresh contiguous output). /// Single 64B memcpy covers cache line 1; `ci` pre-initialises start_offset (=0) /// and is_contiguous (=true) in its line-1 slots so they need no reset here. /// Cache line 2 (stride/extent) is computed from `ci.shapes` in a single reverse pass. +/// +/// Metadata only: `ci.initial_value` is not applied here. The buffer lives in +/// the GM heap, which the host orchestrator reaches only through the run's +/// HostTensorAccessor, so the fill belongs to the orchestrator call sites that +/// hold that accessor and can report a failure. inline void init_tensor_from_create_info(ChipTensor &t, const TensorCreateInfo &ci, void *addr, uint64_t buffer_size) { always_assert(ci.ndims > 0 && ci.ndims <= MAX_TENSOR_DIMS); memcpy(&t, &ci, 64); @@ -141,7 +128,4 @@ inline void init_tensor_from_create_info(ChipTensor &t, const TensorCreateInfo & s *= t.shapes[i]; } t.extent_elem_cache = s; - if (ci.has_initial_value) { - fill_tensor_initial_value(t, ci.initial_value); - } } diff --git a/src/a5/runtime/host_build_graph/docs/SCALAR_DATA_ACCESS.md b/src/a5/runtime/host_build_graph/docs/SCALAR_DATA_ACCESS.md index c71dfb5036..e6b75b8e02 100644 --- a/src/a5/runtime/host_build_graph/docs/SCALAR_DATA_ACCESS.md +++ b/src/a5/runtime/host_build_graph/docs/SCALAR_DATA_ACCESS.md @@ -11,7 +11,7 @@ external tensors; they do not interleave host code with device execution. | ------------ | ----------------- | ----------------- | | External tensor with no submitted producer | Reads the staged host value | Updates the staged host value | | External control/output tensor not referenced by a task | Reads immediately | Writes immediately | -| Runtime-created output | Unsupported: no registered host view | Unsupported: no registered host view | +| Runtime-created output | Unsupported: not a staged tensor | Unsupported: not a staged tensor. Set the value with `TensorCreateInfo::set_initial_value` instead | | Tensor owned by a submitted device task | Unsupported during graph construction | Unsupported during graph construction | | Tensor with an invalid or stale owner task ID | Fails with `INVALID_ARGS` | Fails with `INVALID_ARGS` | @@ -46,9 +46,54 @@ for that producer from the orchestration call cannot make progress. The runtime keeps a timeout as a defensive failure backstop, but it is not a supported synchronization mechanism. -Runtime-created output buffers also live in the graph heap and have no host-view -registration. Even if their address is nonzero, host orchestration must not -dereference or modify them. +Runtime-created output buffers live in the GM heap rather than among the staged +tensors, so `get_tensor_data` / `set_tensor_data` do not resolve them. Host +orchestration must never dereference such an address itself; the initial-value +path below is the one way it reaches those bytes. + +## Initial Values on a Runtime Allocation + +`TensorCreateInfo::set_initial_value(v)` fills the whole buffer the runtime +allocates for that output, before any task can observe it. It is the way to give +a runtime-created buffer a defined starting content — a fixed-size tile whose +producer writes only a prefix, for instance, so its consumer reads `v` in the +remainder rather than whatever the heap last held. + +```cpp +TensorCreateInfo tile(shape, 2, DataType::INT8); +tile.set_initial_value(0); +TaskOutputTensors outs = alloc_tensors(tile); +``` + +The fill is a host-side write reaching device memory, so how it gets there +depends on the platform, but the semantics do not: + +| Platform | How the bytes reach the heap | +| -------- | ---------------------------- | +| a2a3 onboard | The heap is mapped into the host address space (`halHostRegister(DEV_SVM_MAP_HOST)`) and the store lands on it directly — no copy | +| a5 onboard | No host-map path, so the pattern is staged in a bounded buffer and pushed with `copy_to_device` — the same mechanism `set_tensor_data` already uses there | +| simulation | A device pointer is already a host pointer | + +The mapping is made by the first fill that needs it, not at bind: the heap runs +to hundreds of MB and a run that sets no initial value never touches it, so an +orchestration that uses none costs nothing here. + +The write lands at the call that asked for it, never deferred: the heap is +reclaimed and re-let within one orchestration, so two fills can name the same +address and only their issue order is correct. + +One constraint follows from *when* the fill happens: + +- **A Graph body must not ask for one.** Recording addresses an internal node's + outputs in the private range based at `GRAPH_RECORD_VIRTUAL_BASE`, and every + submission of the resulting Definition materializes its own from a heap block + whose prior contents it never reads — so a value written during recording + reaches none of the replays. The recording is marked unsupported and + `graph_commit` latches `PTO2_ERROR_INVALID_ARGS`; the body is not re-run on the + ordinary path, because its outer shell was published before the body was + recorded (see [GRAPH_EXECUTION.md](../../../../common/host_build_graph/docs/GRAPH_EXECUTION.md)). + Allocate the buffer outside the Graph body and pass it across the boundary, or + have a task write the padding the consumer relies on. ## Ownership Validation @@ -71,3 +116,5 @@ recorded. as a device synchronization barrier. - Pass values needed for graph construction as orchestration inputs or scalars. - Keep device-produced values on the device or return them after the run. +- Give a runtime allocation its starting content with + `TensorCreateInfo::set_initial_value`, not with `set_tensor_data`. diff --git a/src/a5/runtime/host_build_graph/host/host_tensor_access.cpp b/src/a5/runtime/host_build_graph/host/host_tensor_access.cpp index 973b934030..62d849fe89 100644 --- a/src/a5/runtime/host_build_graph/host/host_tensor_access.cpp +++ b/src/a5/runtime/host_build_graph/host/host_tensor_access.cpp @@ -25,29 +25,51 @@ struct HostTensorRegion { uint64_t dev_base; uint64_t size; + // Null on a GM-heap region until its first fill, and after one whose mapping + // attempt failed. That region is write-only, so `fill` can stage the bytes + // elsewhere and push them; every other region is read back through this view + // and is mapped when it is registered. unsigned char *host_view; // The host view is a copy of the device bytes rather than the device bytes // themselves, so a write reaches the device only once pushed back. bool mirrored; + // Whether a mapping has been attempted for this region. A GM-heap region is + // registered without one — mapping the whole heap costs a `halHostRegister` + // over hundreds of MB (2 GiB on the dsv4 case) that a run setting no initial + // value would never use — so the attempt is deferred to the first fill and, + // either way, made once. + bool map_attempted; + HostTensorRegionKind kind; }; -// One entry per tensor staged for the run being orchestrated. A run stages a -// handful of tensors and orchestration reads are cold-path, so a linear scan -// costs less than the map that would replace it. +// Bytes of the buffer `fill` stages a viewless region's pattern in. One period +// of the pattern is at most 8 bytes, so this holds thousands of repeats and the +// same buffer serves every chunk of every fill in the run. +constexpr uint64_t kFillStagingBytes = 64 * 1024; + +// One entry per tensor staged for the run being orchestrated, plus at most one +// for the GM heap. A run stages a handful of tensors and orchestration reads +// are cold-path, so a linear scan costs less than the map that would replace it. struct HostTensorAccessor::Impl { const HostApi *api; std::vector regions; std::vector mappings; // Bytes covered by `mappings`, i.e. excluding regions serving a fallback view. uint64_t mapped_bytes; + // Allocated on the first fill of a viewless region, and only then: a + // platform that maps the heap never needs it. + std::vector fill_staging; }; -// The region serving the whole of [dev_addr, dev_addr + bytes), or nullptr. -// `*offset` is the span's distance from that region's base. -const HostTensorRegion * -find_region(const std::vector ®ions, uint64_t dev_addr, uint64_t bytes, uint64_t *offset) { - for (const HostTensorRegion ®ion : regions) { - if (dev_addr < region.dev_base) { +// The `kind` region serving the whole of [dev_addr, dev_addr + bytes), or +// nullptr. `*offset` is the span's distance from that region's base. Non-const +// because a GM-heap region binds its host view on the first fill that needs it. +HostTensorRegion *find_region( + std::vector ®ions, HostTensorRegionKind kind, uint64_t dev_addr, uint64_t bytes, + uint64_t *offset +) { + for (HostTensorRegion ®ion : regions) { + if (region.kind != kind || dev_addr < region.dev_base) { continue; } uint64_t off = dev_addr - region.dev_base; @@ -61,7 +83,7 @@ find_region(const std::vector ®ions, uint64_t dev_addr, uin } HostTensorAccessor::HostTensorAccessor(const HostApi *api) : - impl_(new Impl{api, {}, {}, 0}) {} + impl_(new Impl{api, {}, {}, 0, {}}) {} HostTensorAccessor::~HostTensorAccessor() { close(); @@ -69,6 +91,16 @@ HostTensorAccessor::~HostTensorAccessor() { } bool HostTensorAccessor::add(uint64_t dev_base, uint64_t size, void *fallback_host_view) { + return add_region(dev_base, size, fallback_host_view, HostTensorRegionKind::StagedTensor); +} + +bool HostTensorAccessor::add_heap(uint64_t dev_base, uint64_t size) { + return add_region(dev_base, size, nullptr, HostTensorRegionKind::GmHeap); +} + +bool HostTensorAccessor::add_region( + uint64_t dev_base, uint64_t size, void *fallback_host_view, HostTensorRegionKind kind +) { if (impl_->api == nullptr || dev_base == 0 || size == 0) { return false; } @@ -76,6 +108,14 @@ bool HostTensorAccessor::add(uint64_t dev_base, uint64_t size, void *fallback_ho // first: a reallocation that threw there would leak the mapping. impl_->regions.reserve(impl_->regions.size() + 1); impl_->mappings.reserve(impl_->mappings.size() + 1); + // A staged tensor is read through its view, so it is mapped now and must end + // up with one. The GM heap is only ever written, by `fill`, so its mapping + // waits for a fill to need it and the common run — which sets no initial + // value — registers the region without touching the heap at all. + if (kind == HostTensorRegionKind::GmHeap) { + impl_->regions.push_back({dev_base, size, nullptr, false, false, kind}); + return true; + } void *host_view = impl_->api->register_device_memory_to_host(reinterpret_cast(dev_base), size); if (host_view != nullptr) { impl_->mappings.push_back(reinterpret_cast(dev_base)); @@ -86,15 +126,35 @@ bool HostTensorAccessor::add(uint64_t dev_base, uint64_t size, void *fallback_ho if (host_view == nullptr) { return false; } - impl_->regions.push_back( - {dev_base, size, static_cast(host_view), reinterpret_cast(host_view) != dev_base} - ); + const bool mirrored = reinterpret_cast(host_view) != dev_base; + impl_->regions.push_back({dev_base, size, static_cast(host_view), mirrored, true, kind}); return true; } +// Map `region` if that has not been tried yet, so a heap that no fill reaches is +// never mapped. Leaves `host_view` null when the platform has no host-map path, +// which is the caller's cue to stage and push instead. +void HostTensorAccessor::ensure_region_mapped(HostTensorRegion *region) const { + if (region->map_attempted) { + return; + } + region->map_attempted = true; + impl_->mappings.reserve(impl_->mappings.size() + 1); + void *host_view = + impl_->api->register_device_memory_to_host(reinterpret_cast(region->dev_base), region->size); + if (host_view == nullptr) { + return; + } + impl_->mappings.push_back(reinterpret_cast(region->dev_base)); + impl_->mapped_bytes += region->size; + region->host_view = static_cast(host_view); + region->mirrored = reinterpret_cast(host_view) != region->dev_base; +} + bool HostTensorAccessor::read(uint64_t dev_addr, void *dst, uint64_t bytes) const { uint64_t offset = 0; - const HostTensorRegion *region = find_region(impl_->regions, dev_addr, bytes, &offset); + const HostTensorRegion *region = + find_region(impl_->regions, HostTensorRegionKind::StagedTensor, dev_addr, bytes, &offset); if (region == nullptr) { return false; } @@ -104,7 +164,8 @@ bool HostTensorAccessor::read(uint64_t dev_addr, void *dst, uint64_t bytes) cons bool HostTensorAccessor::write(uint64_t dev_addr, const void *src, uint64_t bytes) const { uint64_t offset = 0; - const HostTensorRegion *region = find_region(impl_->regions, dev_addr, bytes, &offset); + const HostTensorRegion *region = + find_region(impl_->regions, HostTensorRegionKind::StagedTensor, dev_addr, bytes, &offset); if (region == nullptr) { return false; } @@ -116,6 +177,75 @@ bool HostTensorAccessor::write(uint64_t dev_addr, const void *src, uint64_t byte return impl_->api->copy_to_device(reinterpret_cast(dev_addr), dst, static_cast(bytes)) == 0; } +// Fill a span of a region that has no host view, by staging one buffer of the +// repeating pattern and pushing it across the span. +// +// The chunk is a whole number of elements, so every push starts on a pattern +// boundary and the staged bytes can be reused verbatim. A span that is not a +// whole number of elements ends on a partial one, which is what a buffer whose +// size is not a multiple of its element width should get. +bool HostTensorAccessor::fill_staged(uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) const { + const uint64_t chunk = (kFillStagingBytes / elem_size) * elem_size; + std::vector &staging = impl_->fill_staging; + // The buffer is reused across fills but the pattern is not: consecutive + // fills differ in value and element width, so it is laid down per call. + staging.resize(static_cast(chunk)); + for (uint64_t written = 0; written < chunk; written += elem_size) { + memcpy(staging.data() + written, &value, static_cast(elem_size)); + } + for (uint64_t pushed = 0; pushed < bytes;) { + const uint64_t span = (bytes - pushed < chunk) ? bytes - pushed : chunk; + if (impl_->api->copy_to_device( + reinterpret_cast(dev_addr + pushed), staging.data(), static_cast(span) + ) != 0) { + return false; + } + pushed += span; + } + return true; +} + +HostTensorFillStatus +HostTensorAccessor::fill(uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) const { + if (bytes == 0) { + return HostTensorFillStatus::Ok; + } + if (elem_size == 0 || elem_size > sizeof(value)) { + return HostTensorFillStatus::BadElementSize; + } + uint64_t offset = 0; + HostTensorRegion *region = find_region(impl_->regions, HostTensorRegionKind::GmHeap, dev_addr, bytes, &offset); + if (region == nullptr) { + return HostTensorFillStatus::NoRegion; + } + ensure_region_mapped(region); + if (region->host_view == nullptr) { + // No host-map path on this platform: stage the pattern and push it. + return fill_staged(dev_addr, bytes, value, elem_size) ? HostTensorFillStatus::Ok : + HostTensorFillStatus::PushFailed; + } + // Seed one element, then double the written prefix into the remainder: the + // span is filled in log2(bytes / elem_size) memcpys rather than one per + // element. A span that is not a whole number of elements ends on a partial + // one, which is what a buffer whose size is not a multiple of its element + // width should get. + unsigned char *dst = region->host_view + offset; + uint64_t seed = (elem_size < bytes) ? elem_size : bytes; + memcpy(dst, &value, static_cast(seed)); + uint64_t filled = seed; + while (filled < bytes) { + uint64_t copy_size = ((bytes - filled) < filled) ? (bytes - filled) : filled; + memcpy(dst + filled, dst, static_cast(copy_size)); + filled += copy_size; + } + if (!region->mirrored) { + return HostTensorFillStatus::Ok; + } + return impl_->api->copy_to_device(reinterpret_cast(dev_addr), dst, static_cast(bytes)) == 0 ? + HostTensorFillStatus::Ok : + HostTensorFillStatus::PushFailed; +} + size_t HostTensorAccessor::mapping_count() const noexcept { return impl_->mappings.size(); } uint64_t HostTensorAccessor::mapped_bytes() const noexcept { return impl_->mapped_bytes; } @@ -136,3 +266,23 @@ bool host_tensor_read(HostTensorAccessor *accessor, uint64_t dev_addr, void *dst bool host_tensor_write(HostTensorAccessor *accessor, uint64_t dev_addr, const void *src, uint64_t bytes) { return accessor != nullptr && accessor->write(dev_addr, src, bytes); } + +HostTensorFillStatus +host_tensor_fill(HostTensorAccessor *accessor, uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) { + if (accessor == nullptr) return HostTensorFillStatus::NoRegion; + return accessor->fill(dev_addr, bytes, value, elem_size); +} + +const char *host_tensor_fill_status_name(HostTensorFillStatus status) { + switch (status) { + case HostTensorFillStatus::Ok: + return "ok"; + case HostTensorFillStatus::BadElementSize: + return "unsupported element width"; + case HostTensorFillStatus::NoRegion: + return "address is not in the GM heap"; + case HostTensorFillStatus::PushFailed: + return "device copy failed"; + } + return "unknown"; +} diff --git a/src/a5/runtime/host_build_graph/host/runtime_maker.cpp b/src/a5/runtime/host_build_graph/host/runtime_maker.cpp index a855343d5b..c228a35f74 100644 --- a/src/a5/runtime/host_build_graph/host/runtime_maker.cpp +++ b/src/a5/runtime/host_build_graph/host/runtime_maker.cpp @@ -570,11 +570,28 @@ struct GraphHostStateBinding { PTO2OrchestratorState &orchestrator; }; +// The accessor lives on run_host_orchestration's frame while `rt` lives in an +// arena that outlives the call, so both pointers to it are cleared on the way +// out rather than left dangling for the next run to find. +struct HostTensorAccessBinding { + HostTensorAccessBinding(PTO2Runtime &rt, HostTensorAccessor &accessor) : + rt(rt) { + rt.tensor_access = &accessor; + rt.orchestrator.tensor_access = &accessor; + } + ~HostTensorAccessBinding() { + rt.tensor_access = nullptr; + rt.orchestrator.tensor_access = nullptr; + } + + PTO2Runtime &rt; +}; + int32_t run_host_orchestration( Runtime *runtime, const HostApi *api, HostTensorAccessor &tensor_access, PTO2Runtime *rt, DeviceArena &host_arena, const PTO2RuntimeArenaLayout &layout, void *device_sm, uint64_t sm_size, void *device_arena, void *gm_heap, - const uint64_t eff_heap_sizes[PTO2_MAX_RING_DEPTH], const uint64_t eff_task_window_sizes[PTO2_MAX_RING_DEPTH], - void *host_orch_func_ptr, const ChipTaskArgs &orch_l2 + uint64_t total_heap_size, const uint64_t eff_heap_sizes[PTO2_MAX_RING_DEPTH], + const uint64_t eff_task_window_sizes[PTO2_MAX_RING_DEPTH], void *host_orch_func_ptr, const ChipTaskArgs &orch_l2 ) { // The dep_gen graph belongs to the orchestration that is about to run. dep_gen_host_graph_begin_capture(); @@ -638,7 +655,15 @@ int32_t run_host_orchestration( return -1; } rt->active_callable_hash = reinterpret_cast(entry_points->entry); - rt->tensor_access = &tensor_access; + HostTensorAccessBinding tensor_access_binding(*rt, tensor_access); + // The GM heap backs every runtime-allocated output, so an initial value the + // orchestration asks for lands here. Registering only records the span; the + // host mapping of it waits for a fill to need one, so a run that sets no + // initial value pays nothing for this. + if (!tensor_access.add_heap(reinterpret_cast(gm_heap), total_heap_size)) { + LOG_ERROR("host-orch: failed to register the GM heap (%" PRIu64 " bytes)", total_heap_size); + return -1; + } // Binds the orchestration .so's own framework_current_runtime, which its // inline rt_submit_* read. The host library links a same-named copy from // orchestration/common.cpp, but nothing outside the .so includes @@ -1105,7 +1130,7 @@ extern "C" int bind_callable_to_runtime_impl( orch_l2.create_from_chip_args(device_args); int32_t total_tasks = run_host_orchestration( runtime, api, tensor_access, rt, host_arena, layout, sm_ptr, sm_size, runtime_arena_dev, gm_heap, - eff_heap_sizes, eff_task_window_sizes, host_orch_func_ptr, orch_l2 + total_heap_size, eff_heap_sizes, eff_task_window_sizes, host_orch_func_ptr, orch_l2 ); // The orchestrator is the only host-view reader; from here the device // owns these buffers, so drop the window on both exits. diff --git a/src/a5/runtime/host_build_graph/orchestration/pto_orchestration_api.h b/src/a5/runtime/host_build_graph/orchestration/pto_orchestration_api.h index ea4d106108..f57fdda133 100644 --- a/src/a5/runtime/host_build_graph/orchestration/pto_orchestration_api.h +++ b/src/a5/runtime/host_build_graph/orchestration/pto_orchestration_api.h @@ -518,8 +518,9 @@ static inline T get_tensor_data(const ChipTensor &tensor, uint32_t ndims, const * This API updates the registered host view used to stage an external tensor. * The updated value becomes part of the graph's initial device data. It is not * a synchronization barrier for submitted readers or writers. A tensor with a - * submitted producer, or a runtime-created output with no registered host view, - * is rejected as an invalid argument. + * submitted producer, or a runtime-created output, which is not a staged tensor, + * is rejected as an invalid argument; give a runtime allocation its starting + * content with TensorCreateInfo::set_initial_value instead. */ template static inline void set_tensor_data(const ChipTensor &tensor, uint32_t ndims, const uint32_t indices[], T value) { diff --git a/src/a5/runtime/host_build_graph/runtime/host_tensor_access.h b/src/a5/runtime/host_build_graph/runtime/host_tensor_access.h index 148feb8f4b..74fbbb82b8 100644 --- a/src/a5/runtime/host_build_graph/runtime/host_tensor_access.h +++ b/src/a5/runtime/host_build_graph/runtime/host_tensor_access.h @@ -54,6 +54,29 @@ struct HostApi; // common/host_api.h — fwd-declared so this header stays out of platform includes +// What a region backs, and therefore which lookups may resolve to it. The two +// kinds never overlap: a staged tensor is its own device allocation and the GM +// heap is another, so a device address belongs to at most one region. +enum class HostTensorRegionKind : uint8_t { + StagedTensor, + GmHeap, +}; + +// Why a fill did not happen. The cases are distinct enough to lead a reader in +// different directions — a bad element width is a malformed create-info, an +// unresolved span is an address outside the heap, and a failed push is a device +// copy error — so the caller reports which one occurred rather than guessing. +// +// A region without a host view is not among them: that is the ordinary state on +// a platform with no host-map path, where `fill` stages the bytes and pushes +// them, and the outcome is an ordinary Ok or PushFailed. +enum class HostTensorFillStatus : uint8_t { + Ok, + BadElementSize, + NoRegion, + PushFailed, +}; + /** * The registered regions of one orchestration run, and the mappings that run * installed to serve them. @@ -64,14 +87,23 @@ struct HostApi; // common/host_api.h — fwd-declared so this header stays out * are isolated by each owning a separate accessor, which is what makes two runs * unable to see or drop each other's regions. * - * `add` is the only producer of mappings and the only caller of - * `register_device_memory_to_host`; `close` unregisters exactly the mappings + * `add` and `add_heap` are the only producers of mappings and the only callers + * of `register_device_memory_to_host`; `close` unregisters exactly the mappings * this accessor installed and nothing else. Both are reached on every return * path — `close` is idempotent and the destructor calls it — so a mapping * cannot outlive the run that made it. * - * A null `api` makes every `add` fail, so a registered region always implies a - * usable `api`; `write`'s mirror push-back relies on that and does not re-check. + * A null `api` makes every registration fail, so a registered region always + * implies a usable `api`; `write`'s mirror push-back relies on that and does + * not re-check. + * + * Regions come in two kinds and the lookups are kind-scoped. `read` / `write` + * serve the orchestration scalar-access API and see staged-tensor regions only, + * so a device address outside a staged tensor — a GM-heap buffer, a + * pass-through child-memory buffer — resolves to nothing there, which is what + * makes runtime-created outputs unreadable during host orchestration. `fill` + * serves `TensorCreateInfo::set_initial_value`, whose target is always a + * runtime allocation, and sees GM-heap regions only. * * The state lives behind `Impl` because this header is also compiled by the * AICPU build (through `orchestrator_core/pto_runtime2.cpp`, which resolves the @@ -95,9 +127,38 @@ class HostTensorAccessor { * mapping nor a fallback view is available. */ bool add(uint64_t dev_base, uint64_t size, void *fallback_host_view); + + /** + * Register the GM heap `[dev_base, dev_base + size)` as the region serving + * runtime allocations, reachable through `fill` alone. + * + * Recording the span is all this does. Mapping it is deferred to the first + * `fill` that needs one, because the heap runs to hundreds of MB (2 GiB on + * the dsv4 case) and a run that sets no initial value never reads or writes + * it — paying `halHostRegister` over that span at every bind would buy + * nothing. Where no host-map path exists (a5 onboard, whose + * `DeviceRunnerBase` default returns nullptr) the mapping stays absent and + * `fill` stages the bytes and pushes them with `copy_to_device` instead. + */ + bool add_heap(uint64_t dev_base, uint64_t size); + bool read(uint64_t dev_addr, void *dst, uint64_t bytes) const; bool write(uint64_t dev_addr, const void *src, uint64_t bytes) const; + /** + * Fill `[dev_addr, dev_addr + bytes)` with `elem_size`-wide `value`, + * leaving the bytes visible to the device. `bytes` need not be a multiple + * of `elem_size`; the tail is a partial element. + * + * Maps the GM heap on the first call that needs it, so a run that sets no + * initial value never pays for a mapping of the whole heap. + * + * The write lands immediately, never deferred to a flush: the GM heap is + * reclaimed and re-let within one orchestration, so two fills can name the + * same address and only the order they were issued in is correct. + */ + HostTensorFillStatus fill(uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) const; + /** Drop every region and unregister every mapping this accessor installed. */ void close() noexcept; @@ -108,6 +169,10 @@ class HostTensorAccessor { uint64_t mapped_bytes() const noexcept; private: + bool add_region(uint64_t dev_base, uint64_t size, void *fallback_host_view, HostTensorRegionKind kind); + void ensure_region_mapped(struct HostTensorRegion *region) const; + bool fill_staged(uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) const; + struct Impl; Impl *impl_; }; @@ -124,7 +189,19 @@ bool host_tensor_read(HostTensorAccessor *accessor, uint64_t dev_addr, void *dst * Write `bytes` from `src` to device address `dev_addr`, leaving the bytes * visible to the device. * - * @return false when no registered region covers the whole span, or when the + * @return false when no staged-tensor region covers the whole span, or when the * push-back to the device fails. */ bool host_tensor_write(HostTensorAccessor *accessor, uint64_t dev_addr, const void *src, uint64_t bytes); + +/** + * Fill `bytes` at device address `dev_addr` with `elem_size`-wide `value`, + * leaving the bytes visible to the device. + * + * @return what stopped the fill, or `Ok`. + */ +HostTensorFillStatus +host_tensor_fill(HostTensorAccessor *accessor, uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size); + +/** Short, stable label for `status`, for a diagnostic naming what failed. */ +const char *host_tensor_fill_status_name(HostTensorFillStatus status); diff --git a/src/a5/runtime/host_build_graph/runtime/orchestrator_core/pto_orchestrator.cpp b/src/a5/runtime/host_build_graph/runtime/orchestrator_core/pto_orchestrator.cpp index cc1c96b904..7df911f609 100644 --- a/src/a5/runtime/host_build_graph/runtime/orchestrator_core/pto_orchestrator.cpp +++ b/src/a5/runtime/host_build_graph/runtime/orchestrator_core/pto_orchestrator.cpp @@ -49,6 +49,7 @@ #include "pto_dep_compute.h" #include "graph_execution.h" #include "graph_host_state.h" +#include "host_tensor_access.h" #include "pto_runtime2_types.h" #include "pto_shared_memory.h" #include "pto_tensormap.h" @@ -79,6 +80,32 @@ __attribute__((weak, visibility("hidden"))) void dep_gen_host_graph_add_tensorma uint64_t, int32_t, const ChipTensor &, const PTO2TensorMapEntry &, OverlapStatus ) {} +// Initial-value fill for a caller that can store to a device address directly. +// It lives here rather than beside the host_tensor_read / host_tensor_write +// fallbacks in pto_runtime2.cpp because apply_initial_values below is its only +// caller, and the unit tests that link this translation unit do not link that +// one. libhost_runtime.so overrides it with host/host_tensor_access.cpp, where +// a device address is not loadable in general. +__attribute__((weak, visibility("hidden"))) HostTensorFillStatus +host_tensor_fill(HostTensorAccessor *, uint64_t dev_addr, uint64_t bytes, uint64_t value, uint64_t elem_size) { + if (bytes == 0) return HostTensorFillStatus::Ok; + if (elem_size == 0 || elem_size > sizeof(value)) return HostTensorFillStatus::BadElementSize; + char *dst = reinterpret_cast(dev_addr); + uint64_t seed = (elem_size < bytes) ? elem_size : bytes; + memcpy(dst, &value, seed); + uint64_t filled = seed; + while (filled < bytes) { + uint64_t copy_size = ((bytes - filled) < filled) ? (bytes - filled) : filled; + memcpy(dst + filled, dst, copy_size); + filled += copy_size; + } + return HostTensorFillStatus::Ok; +} + +__attribute__((weak, visibility("hidden"))) const char *host_tensor_fill_status_name(HostTensorFillStatus) { + return "unavailable"; +} + // Scope_stats enable gate, queried via the same predicate idiom as // dep_gen_host_graph_enabled above. The AICPU collector links the strong definition; // host builds fall back to this weak `false`. Gating here still skips the @@ -1141,6 +1168,37 @@ static bool ensure_tensormap_capacity(PTO2OrchestratorState *orch, int32_t neede return false; } +// Write each output create-info's initial value into the buffer just +// materialized for it. `tensors` is the payload's tensor array, parallel to +// `args`, so slot i's buffer address and size are the ones the device will see. +// +// The buffer is in the GM heap, which the host reaches only through the run's +// registered host view; a platform that cannot map the heap latches a fatal +// here rather than storing to a device address. Returns false once it has. +static bool apply_initial_values( + PTO2OrchestratorState *orch, const CoreTaskArgs &args, const ChipTensor *tensors, const char *caller +) { + for (int32_t i = 0; i < args.tensor_count(); i++) { + if (args.tag(i) != TensorArgType::OUTPUT) continue; + const TensorCreateInfo &ci = args.tensor(i).create_info(); + if (!ci.has_initial_value) continue; + const ChipTensor &t = tensors[i]; + const uint64_t elem_size = get_element_size(t.dtype); + const HostTensorFillStatus status = + host_tensor_fill(orch->tensor_access, t.buffer.addr, t.buffer.size, ci.initial_value, elem_size); + if (status != HostTensorFillStatus::Ok) { + orch->report_fatal( + PTO2_ERROR_INVALID_ARGS, caller, + "set_initial_value: %s (addr=%#llx bytes=%llu dtype=%d elem_size=%llu)", + host_tensor_fill_status_name(status), (unsigned long long)t.buffer.addr, + (unsigned long long)t.buffer.size, static_cast(t.dtype), (unsigned long long)elem_size + ); + return false; + } + } + return true; +} + // Shared body for submit_task / submit_dummy_task. Caller has already validated // args.has_error, decided active_mask (empty for dummy), and resolved the per-slot // kernel_ids (all INVALID_KERNEL_ID for dummy). Performs tensormap sync, fanin @@ -1284,6 +1342,9 @@ static TaskOutputTensors submit_task_common( // payload.fanin_local_ids and bumped its last_consumer_local_id; the count is // published in STEP 6 below. payload.init does not touch the fanin region. payload.init(args, result, prepared.alloc_result, layout); + if (!apply_initial_values(orch, args, payload.tensors, __FUNCTION__)) { + return result; + } // Dispatch predicate: resolve the (tensor, indices) to an absolute GM address // now so the scheduler can read it at the dispatch point with a single load, @@ -1693,9 +1754,16 @@ TaskOutputTensors graph_record_submit_node( if (args.tag(i) != TensorArgType::OUTPUT) { slot_tensor.copy(args.tensor(i).ref()); } else { + const TensorCreateInfo &ci = args.tensor(i).create_info(); + // An initial value cannot survive into a replay: recording addresses + // this output in the private range based at GRAPH_RECORD_VIRTUAL_BASE, + // while every submission materializes its own from a heap block whose + // prior contents it never reads. Mark the recording unsupported so + // commit reports it, rather than letting a Definition replay outputs + // the orchestration believes it initialized. + if (ci.has_initial_value) recording.unsupported = true; init_tensor_from_create_info( - slot_tensor, args.tensor(i).create_info(), - reinterpret_cast(packed_base_addr + layout.offsets[i]), layout.buffer_sizes[i] + slot_tensor, ci, reinterpret_cast(packed_base_addr + layout.offsets[i]), layout.buffer_sizes[i] ); slot_tensor.owner_task_id = task_id; } @@ -2216,6 +2284,9 @@ TaskOutputTensors PTO2OrchestratorState::alloc_tensors(const CoreTaskArgs &args) TaskOutputTensors outputs; outputs.set_task_id(prepared.task_id); payload.init(args, outputs, prepared.alloc_result, layout); + if (!apply_initial_values(orch, args, payload.tensors, __FUNCTION__)) { + return TaskOutputTensors{}; + } payload.fanin_count = 0; // hidden-alloc tasks have no producer dependencies CYCLE_COUNT_LAP(g_orch_args_cycle); diff --git a/src/a5/runtime/host_build_graph/runtime/pto_orchestrator.h b/src/a5/runtime/host_build_graph/runtime/pto_orchestrator.h index 0d24c1babd..051b6c4f0b 100644 --- a/src/a5/runtime/host_build_graph/runtime/pto_orchestrator.h +++ b/src/a5/runtime/host_build_graph/runtime/pto_orchestrator.h @@ -39,6 +39,7 @@ #include "pto_types.h" struct GraphHostState; +class HostTensorAccessor; /** * Layout descriptor produced by PTO2OrchestratorState::reserve_layout(). Holds @@ -100,6 +101,13 @@ struct PTO2OrchestratorState { void *gm_heap_base; // Base address of GM heap uint64_t gm_heap_size; // Total size of GM heap (all rings) + // Host views registered for this run, borrowed from PTO2Runtime. The + // orchestrator reaches GM-heap bytes only through this: `gm_heap_base` is a + // device address, loadable by the AICPU but not by the host. Null on the + // AICPU path and whenever the platform cannot map the heap, which makes an + // initial-value fill fail closed rather than store to a device address. + HostTensorAccessor *tensor_access{nullptr}; + // === FATAL ERROR === // Fatal error flag (single-thread access by orchestrator, no atomic needed) // Cross-thread notification uses shared memory orch_error_code (atomic) diff --git a/src/a5/runtime/host_build_graph/runtime/tensor_create_info.h b/src/a5/runtime/host_build_graph/runtime/tensor_create_info.h index e50cc709e6..4047fa2d12 100644 --- a/src/a5/runtime/host_build_graph/runtime/tensor_create_info.h +++ b/src/a5/runtime/host_build_graph/runtime/tensor_create_info.h @@ -108,28 +108,15 @@ static_assert(offsetof(TensorCreateInfo, shapes) == offsetof(ChipTensor, shapes) // header) so the create-info dependency stays runtime-only. // ============================================================================ -/// Fill the entire backing buffer of `t` with `initial_value` (doubling memcpy). -inline void fill_tensor_initial_value(ChipTensor &t, uint64_t initial_value) { - always_assert(reinterpret_cast(t.buffer.addr) != nullptr); - uint64_t elem_size = get_element_size(t.dtype); - char *dst = reinterpret_cast(t.buffer.addr); - constexpr uint64_t blk_size = 64; - uint64_t blk = (t.buffer.size < blk_size) ? t.buffer.size : blk_size; - for (uint64_t b = 0; b < blk; b += elem_size) { - memcpy(dst + b, &initial_value, elem_size); - } - uint64_t filled = blk; - while (filled < t.buffer.size) { - uint64_t copy_size = ((t.buffer.size - filled) < filled) ? (t.buffer.size - filled) : filled; - memcpy(dst + filled, dst, copy_size); - filled += copy_size; - } -} - /// Materialize a TensorCreateInfo into `t` (fresh contiguous output). /// Single 64B memcpy covers cache line 1; `ci` pre-initialises start_offset (=0) /// and is_contiguous (=true) in its line-1 slots so they need no reset here. /// Cache line 2 (stride/extent) is computed from `ci.shapes` in a single reverse pass. +/// +/// Metadata only: `ci.initial_value` is not applied here. The buffer lives in +/// the GM heap, which the host orchestrator reaches only through the run's +/// HostTensorAccessor, so the fill belongs to the orchestrator call sites that +/// hold that accessor and can report a failure. inline void init_tensor_from_create_info(ChipTensor &t, const TensorCreateInfo &ci, void *addr, uint64_t buffer_size) { always_assert(ci.ndims > 0 && ci.ndims <= MAX_TENSOR_DIMS); memcpy(&t, &ci, 64); @@ -141,7 +128,4 @@ inline void init_tensor_from_create_info(ChipTensor &t, const TensorCreateInfo & s *= t.shapes[i]; } t.extent_elem_cache = s; - if (ci.has_initial_value) { - fill_tensor_initial_value(t, ci.initial_value); - } } diff --git a/src/common/task_interface/tensor.h b/src/common/task_interface/tensor.h index 5b3d4e14c7..dfdafcd1ec 100644 --- a/src/common/task_interface/tensor.h +++ b/src/common/task_interface/tensor.h @@ -243,10 +243,14 @@ struct alignas(64) ChipTensor { void copy(const ChipTensor &other) { init_from(other); } // Materialization from a TensorCreateInfo (runtime-allocated outputs) lives - // in the runtime tensor_create_info.h as the free functions - // init_tensor_from_create_info() / fill_tensor_initial_value(); they operate - // on a ChipTensor& through its public members. Kept out of the wire/host-facing - // ChipTensor so this header has no dependency on the runtime-only create-info. + // in the runtime tensor_create_info.h as the free function + // init_tensor_from_create_info(), which operates on a ChipTensor& through its + // public members. Kept out of the wire/host-facing ChipTensor so this header + // has no dependency on the runtime-only create-info. Applying the create-info's + // initial value is separate and runtime-specific: tensormap_and_ringbuffer + // stores to the buffer directly from the AICPU orchestrator, while + // host_build_graph routes it through the orchestrating host's view of the GM + // heap, which only the orchestrator holds. // ======================================================================== // Address / offset computation diff --git a/tests/st/a2a3/host_build_graph/initial_value/kernels/orchestration/initial_value_orch.cpp b/tests/st/a2a3/host_build_graph/initial_value/kernels/orchestration/initial_value_orch.cpp new file mode 100644 index 0000000000..b767b4d848 --- /dev/null +++ b/tests/st/a2a3/host_build_graph/initial_value/kernels/orchestration/initial_value_orch.cpp @@ -0,0 +1,91 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +/** + * TensorCreateInfo::set_initial_value on the two paths that materialize a + * runtime allocation: alloc_tensors and a submitted task's output. + * + * Both buffers live in the GM heap, which the host orchestrator reaches only + * through the run's registered host view, so a value that arrives at the kernel + * is evidence that view resolved. Each is observed by a kernel rather than by + * get_tensor_data, which cannot read a runtime-created tensor here. + */ + +#include + +#include + +#include "pto_orchestration_api.h" // NOLINT(build/include_subdir) + +#define FUNC_ADD 0 +#define FUNC_ADD_SCALAR 1 + +namespace { + +constexpr float kAllocFill = 7.0F; +constexpr float kDummyFill = 11.0F; + +// out_alloc = kAllocFill + a, out_dummy = kDummyFill. +void initial_value_body(const CoreTaskArgs &args) { + const ChipTensor &a = args.tensor(0).ref(); + const ChipTensor &out_alloc = args.tensor(1).ref(); + const ChipTensor &out_dummy = args.tensor(2).ref(); + + const std::array shape{a.shapes[0]}; + + // alloc_tensors path: the allocation is never written by a task, so the + // adder's first operand is the initial value itself. + TensorCreateInfo filled_info(shape.data(), static_cast(shape.size()), DataType::FLOAT32); + filled_info.set_initial_value(kAllocFill); + TaskOutputTensors filled_outputs = alloc_tensors(filled_info); + ChipTensor filled = filled_outputs.get_ref(0); + + CoreTaskArgs add_args; + add_args.add_input(filled, a); + add_args.add_output(out_alloc); + rt_submit_aiv_task(FUNC_ADD, add_args); + + // Submitted-task path: a dummy task dispatches no kernel, so its output + // also reaches its consumer holding nothing but the initial value. + TensorCreateInfo dummy_info(shape.data(), static_cast(shape.size()), DataType::FLOAT32); + dummy_info.set_initial_value(kDummyFill); + CoreTaskArgs dummy_args; + dummy_args.add_output(dummy_info); + TaskOutputTensors dummy_outputs = rt_submit_dummy_task(dummy_args); + ChipTensor dummied = dummy_outputs.get_ref(0); + + CoreTaskArgs copy_args; + copy_args.add_input(dummied); + copy_args.add_output(out_dummy); + copy_args.add_scalar(0.0F); + rt_submit_aiv_task(FUNC_ADD_SCALAR, copy_args); +} + +} // namespace + +extern "C" { + +__attribute__((visibility("default"))) PTO2OrchestrationConfig aicpu_orchestration_config(const ChipTaskArgs &args) { + (void)args; + return PTO2OrchestrationConfig{ + .expected_arg_count = 3, + }; +} + +__attribute__((visibility("default"))) void aicpu_orchestration_entry(const ChipTaskArgs &args) { + CoreTaskArgs body_args; + body_args.add_input(args.tensor(0).ref()); + body_args.add_output(args.tensor(1).ref()); + body_args.add_output(args.tensor(2).ref()); + initial_value_body(body_args); +} + +} // extern "C" diff --git a/tests/st/a2a3/host_build_graph/initial_value/test_initial_value.py b/tests/st/a2a3/host_build_graph/initial_value/test_initial_value.py new file mode 100644 index 0000000000..b8b32762af --- /dev/null +++ b/tests/st/a2a3/host_build_graph/initial_value/test_initial_value.py @@ -0,0 +1,76 @@ +#!/usr/bin/env python3 +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- +"""A TensorCreateInfo initial value reaches the kernel that consumes the allocation. + +The buffer is in the GM heap, which the host orchestrator can only write through +the host view registered for the run, so a wrong or absent view shows up as +`out_alloc` / `out_dummy` holding whatever the heap last held. +""" + +import torch +from simpler.task_interface import ArgDirection as D + +from simpler_setup import SceneTestCase, TaskArgsBuilder, TensorArg, scene_test + +ALLOC_FILL = 7.0 +DUMMY_FILL = 11.0 + + +@scene_test(level=2, runtime="host_build_graph") +class TestInitialValueHostBuildGraph(SceneTestCase): + RTOL = 1e-5 + ATOL = 1e-5 + + CALLABLE = { + "orchestration": { + "source": "kernels/orchestration/initial_value_orch.cpp", + "function_name": "aicpu_orchestration_entry", + "signature": [D.IN, D.OUT, D.OUT], + }, + "incores": [ + { + "func_id": 0, + "source": "../vector_example/kernels/aiv/kernel_add.cpp", + "core_type": "aiv", + "signature": [D.IN, D.IN, D.OUT], + }, + { + "func_id": 1, + "source": "../vector_example/kernels/aiv/kernel_add_scalar.cpp", + "core_type": "aiv", + "signature": [D.IN, D.OUT], + }, + ], + } + + CASES = [ + { + "name": "alloc_and_submitted_output", + "platforms": ["a2a3sim", "a2a3"], + "config": {"aicpu_thread_num": 4, "block_dim": 3}, + "params": {"shape": (128 * 128,)}, + }, + ] + + def generate_args(self, params): + shape = params["shape"] + return TaskArgsBuilder( + TensorArg("a", torch.full(shape, 2.0, dtype=torch.float32)), + TensorArg("out_alloc", torch.zeros(shape, dtype=torch.float32)), + TensorArg("out_dummy", torch.zeros(shape, dtype=torch.float32)), + ) + + def compute_golden(self, args, params): + args.out_alloc[:] = ALLOC_FILL + args.a + args.out_dummy[:] = DUMMY_FILL + + +if __name__ == "__main__": + SceneTestCase.run_module(__name__) diff --git a/tests/st/a5/host_build_graph/initial_value/kernels/orchestration/initial_value_orch.cpp b/tests/st/a5/host_build_graph/initial_value/kernels/orchestration/initial_value_orch.cpp new file mode 100644 index 0000000000..b767b4d848 --- /dev/null +++ b/tests/st/a5/host_build_graph/initial_value/kernels/orchestration/initial_value_orch.cpp @@ -0,0 +1,91 @@ +/* + * Copyright (c) PyPTO Contributors. + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of + * CANN Open Software License Agreement Version 2.0 (the "License"). + * Please refer to the License for details. You may not use this file except in compliance with the License. + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. + * See LICENSE in the root of the software repository for the full text of the License. + * ----------------------------------------------------------------------------------------------------------- + */ + +/** + * TensorCreateInfo::set_initial_value on the two paths that materialize a + * runtime allocation: alloc_tensors and a submitted task's output. + * + * Both buffers live in the GM heap, which the host orchestrator reaches only + * through the run's registered host view, so a value that arrives at the kernel + * is evidence that view resolved. Each is observed by a kernel rather than by + * get_tensor_data, which cannot read a runtime-created tensor here. + */ + +#include + +#include + +#include "pto_orchestration_api.h" // NOLINT(build/include_subdir) + +#define FUNC_ADD 0 +#define FUNC_ADD_SCALAR 1 + +namespace { + +constexpr float kAllocFill = 7.0F; +constexpr float kDummyFill = 11.0F; + +// out_alloc = kAllocFill + a, out_dummy = kDummyFill. +void initial_value_body(const CoreTaskArgs &args) { + const ChipTensor &a = args.tensor(0).ref(); + const ChipTensor &out_alloc = args.tensor(1).ref(); + const ChipTensor &out_dummy = args.tensor(2).ref(); + + const std::array shape{a.shapes[0]}; + + // alloc_tensors path: the allocation is never written by a task, so the + // adder's first operand is the initial value itself. + TensorCreateInfo filled_info(shape.data(), static_cast(shape.size()), DataType::FLOAT32); + filled_info.set_initial_value(kAllocFill); + TaskOutputTensors filled_outputs = alloc_tensors(filled_info); + ChipTensor filled = filled_outputs.get_ref(0); + + CoreTaskArgs add_args; + add_args.add_input(filled, a); + add_args.add_output(out_alloc); + rt_submit_aiv_task(FUNC_ADD, add_args); + + // Submitted-task path: a dummy task dispatches no kernel, so its output + // also reaches its consumer holding nothing but the initial value. + TensorCreateInfo dummy_info(shape.data(), static_cast(shape.size()), DataType::FLOAT32); + dummy_info.set_initial_value(kDummyFill); + CoreTaskArgs dummy_args; + dummy_args.add_output(dummy_info); + TaskOutputTensors dummy_outputs = rt_submit_dummy_task(dummy_args); + ChipTensor dummied = dummy_outputs.get_ref(0); + + CoreTaskArgs copy_args; + copy_args.add_input(dummied); + copy_args.add_output(out_dummy); + copy_args.add_scalar(0.0F); + rt_submit_aiv_task(FUNC_ADD_SCALAR, copy_args); +} + +} // namespace + +extern "C" { + +__attribute__((visibility("default"))) PTO2OrchestrationConfig aicpu_orchestration_config(const ChipTaskArgs &args) { + (void)args; + return PTO2OrchestrationConfig{ + .expected_arg_count = 3, + }; +} + +__attribute__((visibility("default"))) void aicpu_orchestration_entry(const ChipTaskArgs &args) { + CoreTaskArgs body_args; + body_args.add_input(args.tensor(0).ref()); + body_args.add_output(args.tensor(1).ref()); + body_args.add_output(args.tensor(2).ref()); + initial_value_body(body_args); +} + +} // extern "C" diff --git a/tests/st/a5/host_build_graph/initial_value/test_initial_value.py b/tests/st/a5/host_build_graph/initial_value/test_initial_value.py new file mode 100644 index 0000000000..1d00e1e971 --- /dev/null +++ b/tests/st/a5/host_build_graph/initial_value/test_initial_value.py @@ -0,0 +1,76 @@ +#!/usr/bin/env python3 +# Copyright (c) PyPTO Contributors. +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of +# CANN Open Software License Agreement Version 2.0 (the "License"). +# Please refer to the License for details. You may not use this file except in compliance with the License. +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. +# See LICENSE in the root of the software repository for the full text of the License. +# ----------------------------------------------------------------------------------------------------------- +"""A TensorCreateInfo initial value reaches the kernel that consumes the allocation. + +The buffer is in the GM heap, which the host orchestrator can only write through +the host view registered for the run, so a wrong or absent view shows up as +`out_alloc` / `out_dummy` holding whatever the heap last held. +""" + +import torch +from simpler.task_interface import ArgDirection as D + +from simpler_setup import SceneTestCase, TaskArgsBuilder, TensorArg, scene_test + +ALLOC_FILL = 7.0 +DUMMY_FILL = 11.0 + + +@scene_test(level=2, runtime="host_build_graph") +class TestInitialValueHostBuildGraphA5(SceneTestCase): + RTOL = 1e-5 + ATOL = 1e-5 + + CALLABLE = { + "orchestration": { + "source": "kernels/orchestration/initial_value_orch.cpp", + "function_name": "aicpu_orchestration_entry", + "signature": [D.IN, D.OUT, D.OUT], + }, + "incores": [ + { + "func_id": 0, + "source": "../vector_example/kernels/aiv/kernel_add.cpp", + "core_type": "aiv", + "signature": [D.IN, D.IN, D.OUT], + }, + { + "func_id": 1, + "source": "../vector_example/kernels/aiv/kernel_add_scalar.cpp", + "core_type": "aiv", + "signature": [D.IN, D.OUT], + }, + ], + } + + CASES = [ + { + "name": "alloc_and_submitted_output", + "platforms": ["a5sim", "a5"], + "config": {"aicpu_thread_num": 4, "block_dim": 3}, + "params": {"shape": (128 * 128,)}, + }, + ] + + def generate_args(self, params): + shape = params["shape"] + return TaskArgsBuilder( + TensorArg("a", torch.full(shape, 2.0, dtype=torch.float32)), + TensorArg("out_alloc", torch.zeros(shape, dtype=torch.float32)), + TensorArg("out_dummy", torch.zeros(shape, dtype=torch.float32)), + ) + + def compute_golden(self, args, params): + args.out_alloc[:] = ALLOC_FILL + args.a + args.out_dummy[:] = DUMMY_FILL + + +if __name__ == "__main__": + SceneTestCase.run_module(__name__)