Skip to content

Nested unfold whose object takes a higher-layer product leaves that layer unflushed #955

Description

@olantwin

Disclaimer: This is quite possibly user error, but it seems like something sensible to do in a real use-case and like something that's not conceptually impossible in phlex's conceptual model.

With phlex 0.4.1, an unfold below another unfold never lets the outer layer flush if its object's constructor takes a product from that outer layer as well as its own input. Every node that joins a product from the outer layer keeps it cached until the end of the job, and folds partitioned by the outer layer never emit.

The layer hierarchy is job → spill → a → b. split_a unfolds each spill into a cells, and split_b unfolds each a cell into b cells. split_b's object takes the a-layer product and a spill-layer product. A fold then counts b cells per spill.

Reproducer

repro.cpp (one plugin that provides the spill product and registers the algorithms):

// job -> spill --split_a--> a --split_b--> b
//
// split_b's object takes the a-layer product and the spill-layer product.
// With two_inputs: false it takes only the a-layer product.
#include "phlex/core/product_selector.hpp"
#include "phlex/model/data_cell_index.hpp"
#include "phlex/module.hpp"
#include "phlex/source.hpp"

#include <spdlog/spdlog.h>
#include <utility>

namespace {
// Unfolds into k children.
struct Split {
  explicit Split(int k) : k{k} {}
  int initial_value() const { return 0; }
  int k;
};
// Same, but also receives a product from a higher layer.
struct SplitWithSpill {
  SplitWithSpill(int k, int /*spill*/) : k{k} {}
  int initial_value() const { return 0; }
  int k;
};
} // namespace

PHLEX_REGISTER_PROVIDERS(s, config)
{
  s.provide("provide_n", [](phlex::data_cell_index const&) { return 2; })
    .output_product("src", phlex::experimental::identifier{"n"},
                    phlex::experimental::identifier{"spill"});
}

PHLEX_REGISTER_ALGORITHMS(m, config)
{
  using phlex::product_selector;
  using id = phlex::experimental::identifier;
  product_selector const spill_n{.creator = "src", .layer = id{"spill"}, .suffix = "n"};
  product_selector const a_n{.creator = "split_a", .layer = id{"a"}, .suffix = "n"};
  product_selector const b_n{.creator = "split_b", .layer = id{"b"}, .suffix = "n"};

  // Each spill -> 2 a cells, each carrying 1.
  m.unfold<Split>(
     "split_a", [](Split const& s, int i) { return i < s.k; },
     [](Split const&, int i) { return std::make_pair(i + 1, 1); }, "a",
     phlex::concurrency::unlimited)
    .input_family(spill_n)
    .output_product_suffixes("n");

  // Each a cell -> 1 b cell.
  if (config.get<bool>("two_inputs", true)) {
    m.unfold<SplitWithSpill>(
       "split_b", [](SplitWithSpill const& s, int i) { return i < s.k; },
       [](SplitWithSpill const&, int i) { return std::make_pair(i + 1, 1); }, "b",
       phlex::concurrency::unlimited)
      .input_family(a_n, spill_n)
      .output_product_suffixes("n");
  } else {
    m.unfold<Split>(
       "split_b", [](Split const& s, int i) { return i < s.k; },
       [](Split const&, int i) { return std::make_pair(i + 1, 1); }, "b",
       phlex::concurrency::unlimited)
      .input_family(a_n)
      .output_product_suffixes("n");
  }

  // Count b cells per spill.
  m.fold(
     "count", [](int& c, int) { ++c; }, phlex::concurrency::serial, "spill")
    .input_family(b_n)
    .output_product_suffixes("count");

  m.observe(
     "print", [](int c) { spdlog::info("count = {}", c); }, phlex::concurrency::serial)
    .input_family(product_selector{.creator = "count", .layer = id{"spill"}, .suffix = "count"});
}

CMakeLists.txt:

cmake_minimum_required(VERSION 3.28)
project(nested_unfold LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 23)
add_compile_options(-Wno-missing-field-initializers)
find_package(phlex 0.4.1 REQUIRED)
add_library(repro MODULE repro.cpp)
target_link_libraries(repro PRIVATE phlex::module)

repro.jsonnet (two_inputs: false gives the working variant):

{
  stage: 'repro',
  driver: { cpp: 'generate_layers', layers: { spill: { parent: 'job', total: 3 } } },
  sources: { src: { cpp: 'repro' } },
  modules: { repro: { cpp: 'repro', two_inputs: true } },
}

Run with PHLEX_PLUGIN_PATH=build:$CONDA_PREFIX/lib phlex -c repro.jsonnet.

Output

With two_inputs: true, nothing is printed, and the teardown reports:

[warning] [repro:split_b/spill] Cached messages: 3
[warning] [repro:split_b/spill]   Product for [spill:0]
[warning] [repro:split_b/spill]   Product for [spill:2]
[warning] [repro:split_b/spill]   Product for [spill:1]
[warning] [repro:count/spill] Cached accumulators: 3
[warning] [repro:count/spill]   Partition [spill:0]
[warning] [repro:count/spill]   Partition [spill:2]
[warning] [repro:count/spill]   Partition [spill:1]

With two_inputs: false, each spill prints count = 2 and there are no warnings. The layer counts are the same in both runs (spill: 3, a: 6, b: 6).

What else we saw

In the full application (SHiP's Trout reconstruction), the same pattern also left spill-layer products cached in every other node that joined them: a transform on b that took the spill product, and a spill-layer transform that took a job-layer product ([node/job] Cached messages: 1). With that pattern absent, the same nodes flushed normally.

  • Empty children aren't needed to trigger it. It happens whether or not some a cells produce no b cells.
  • A transform on b that takes the b product and the spill product works, as long as split_b takes only the a product.
  • Our workaround is to do the two-input work in a transform on a and feed its result to a single-input unfold.

phlex 0.4.1 from conda-forge, GCC 15, Linux x86_64.

Activity

  1. added theissue type on Oct 5, 2026
  2. knoepfel commented on Oct 5, 2026

    @knoepfel
    Member

    @olantwin, bug has been confirmed. Diagnosing now.

  3. olantwin commented on Oct 6, 2026

    @olantwin
    ContributorAuthor

    Thanks for the quick response and fix! I'll test the PR on our real-life code.

  4. added 2 commits that reference this issue on Oct 6, 2026
    88b6340
    3f3a2bb
  5. knoepfel commented on Oct 8, 2026

    @knoepfel
    Member

    As discussed here, we have released Phlex v0.4.3 which now throws an exception if multiple inputs with differently-named layers are presented to an unfold node. Phlex v0.5 will contain the formal fix for this problem, which is proposed in PR #956.

  6. added a commit that references this issue on Oct 8, 2026
    c7ac54d
  7. added a commit that references this issue on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions