Skip to content

Allow co-partitioned Partitioning::Range inputs for sort merge joins #23478

Description

@gene-bordegaray

Related:

Is your feature request related to a problem or challenge?

#23451 makes SortMergeJoinExec require co-partitioned children, but compatible range-partitioned inputs still take the conservative hash-repartition path. A sort merge join can process matching partition indexes locally once the inputs have compatible range layouts.

Example

SortMergeJoinExec: join_type=Inner, on=[(range_key@0, range_key@0)]
  SortExec: expr=[range_key@0 ASC], preserve_partitioning=[true]
    DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4)
  SortExec: expr=[range_key@0 ASC], preserve_partitioning=[true]
    DataSourceExec: output_partitioning=Range([range_key@0 ASC], [(10), (20), (30)], 4)

The compatible range inputs are already co-partitioned by partition index. The optimizer should not insert hash repartitions; the per-partition sorts remain necessary when the source does not provide ordering.

Describe the solution you'd like

Opt SortMergeJoinExec into the temporary compatible-range satisfaction path for co-partitioned inputs.

Covered join types:

  • All supported SortMergeJoinExec join types that execute matching partition indexes locally.

Acceptance criteria

  • Compatible range/range inputs avoid hash repartitioning for the covered join types.
  • Incompatible split points, partition counts, sort options, or key expressions still trigger repartitioning.
  • Tests cover inner, left, right, and full joins, including matched and unmatched rows.
  • The plan retains per-partition sorts when input ordering is not otherwise satisfied.

Additional context

This should follow the multi-child requirement model added in #23451 and precede the general consolidation in #23266.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions