You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support Parquet bloom filter pruning on read #2841
Is your feature request related to a problem or challenge?
Right now ArrowReader only prunes row groups using min/max stats. But for lookups like trace_id = '...' (where values are basically random 128-bit ids), min/max overlaps every row group and nothing gets pruned. The files do have bloom filters on those columns, the reader just never looks at them.
Writing bloom filters already works, you can turn them on per column via WriterProperties. It's purely the read side that's missing.
Describe the solution you'd like
When a predicate has an eq or IN check on a column with a bloom_filter_offset, load the filter and skip any row group that comes back negative. Might make sense as an opt-in on the scan builder, since it's an extra read per row group checked.
#2398 already did this, but it got closed by the stale bot before anyone reviewed it. @dannycjones suggested to file a feature request, so here it is.
A few other places already do this if it helps: Iceberg Java has ParquetBloomRowGroupFilter, DataFusion does it by default (bloom_filter_on_read), and both Trino and DuckDB check bloom filters on read too.
"Hi @dannycjones and @xanderbailey! I saw on #2398 that @xanderbailey mentioned wanting to continue work on this once reopened. I'm a new contributor looking to help out—is @xanderbailey still working on reopening/rebasing #2398, or is this open for someone else to pick up? Happy to help test or co-author if needed!"
Activity
manishgaurav84 commentedon Jul 21, 2026
"Hi @dannycjones and @xanderbailey! I saw on #2398 that @xanderbailey mentioned wanting to continue work on this once reopened. I'm a new contributor looking to help out—is @xanderbailey still working on reopening/rebasing #2398, or is this open for someone else to pick up? Happy to help test or co-author if needed!"
xanderbailey commentedon Jul 26, 2026
Hi there, yes happy to reopen this and see if we can get it over the line. @CTTY are you about to reopen closed PRs or should I just open a new one?