Repository navigation

feat: execute scalar Arrow Python UDFs in native Comet - #6130

Open

viirya wants to merge 15 commits into

apache:mainfrom

viirya:codex/native-arrow-python-udf

Commits on Sep 23, 2026
- feat: execute scalar Arrow Python UDFs in native Comet
  viirya
  committed
  dc54a31
- docs: clarify scalar Arrow UDF support starts in Spark 4.1
  viirya
  committed
  27871a5
- fix: align native Arrow UDF semantics and CI coverage
  viirya
  committed
  fabcae3
- fix: avoid Arrow UDF polling spin and narrow schema support
  viirya
  committed
  e337202
- Fix embedded Python symbols and verify Arrow input types
  viirya
  committed
  6cef96b
- Document embedded Python build and deployment requirements
  viirya
  committed
  ac0f784
Commits on Sep 24, 2026
- Merge upstream main into native Arrow UDF branch
  viirya
  committed
  b8cc409
Commits on Sep 25, 2026
- Fix native Arrow UDF environment and batch semantics
  viirya
  committed
  4830cee
- Honor Arrow Python UDF byte batch limit
  viirya
  committed
  c73d8b5
Commits on Sep 28, 2026
- Skip empty Arrow UDF batches and protect plan metadata
  viirya
  committed
  67a9f77
- Merge upstream main into native Arrow UDF branch
  viirya
  committed
  de71a8e
Commits on Sep 29, 2026
- Fix native Arrow UDF subqueries, file fallback, and exchange reuse
  viirya
  committed
  b0dab2f
Commits on Oct 6, 2026
- Merge upstream main and guard Arrow UDF accumulators
  viirya
  committed
  d6588a6
Commits on Oct 9, 2026
- Address native Arrow UDF review: archives, results, logging, metrics
  viirya
  committed
  794b276
Commits on Oct 10, 2026
- Fix Arrow UDF tracebacks before Python 3.12 and serialize imports
  viirya
  committed
  9391442

Navigation Menu

Repository navigation

feat: execute scalar Arrow Python UDFs in native Comet - #6130

feat: execute scalar Arrow Python UDFs in native Comet#6130
viirya wants to merge 15 commits into
apache:mainfrom
viirya:codex/native-arrow-python-udf

Commits on Sep 23, 2026

feat: execute scalar Arrow Python UDFs in native Comet

docs: clarify scalar Arrow UDF support starts in Spark 4.1

fix: align native Arrow UDF semantics and CI coverage

fix: avoid Arrow UDF polling spin and narrow schema support

Fix embedded Python symbols and verify Arrow input types

Document embedded Python build and deployment requirements

Commits on Sep 24, 2026

Merge upstream main into native Arrow UDF branch

Commits on Sep 25, 2026

Fix native Arrow UDF environment and batch semantics

Honor Arrow Python UDF byte batch limit

Commits on Sep 28, 2026

Skip empty Arrow UDF batches and protect plan metadata

Merge upstream main into native Arrow UDF branch

Commits on Sep 29, 2026

Fix native Arrow UDF subqueries, file fallback, and exchange reuse

Commits on Oct 6, 2026

Merge upstream main and guard Arrow UDF accumulators

Commits on Oct 9, 2026

Address native Arrow UDF review: archives, results, logging, metrics

Commits on Oct 10, 2026

Fix Arrow UDF tracebacks before Python 3.12 and serialize imports