Problem
The ScalarBound enum in src/lib.rs (marked TODO at line ~78) only handles three coordinate types:
Int64 (integer coordinates)
Float64 (lat/lon coordinates)
TimestampNanos (datetime64[ns] coordinates)
xarray supports a much wider range of coordinate types. Unrecognized types silently fall through with no partition pruning.
Missing types
UInt8, UInt16, UInt32, UInt64 (unsigned integers)
Int8, Int16, Int32 (smaller integer widths)
Float32 (single-precision float)
- Timedelta / duration types
- String / categorical coordinates (e.g. station names, model names)
- Boolean coordinates
Work required
- Extend
ScalarBound with variants for the missing types
- Update
compare_to_scalar to handle each new variant against matching DataFusion ScalarValue types
- Update
python_to_scalar_bound to extract these types from Python objects
- Add
partition_metadata() conversion in df.py for new types
- Add tests covering each new coordinate type
Acceptance criteria
- Partition pruning works correctly for each supported xarray coordinate dtype
- Unsupported types degrade gracefully (no pruning, no error)
Problem
The
ScalarBoundenum insrc/lib.rs(markedTODOat line ~78) only handles three coordinate types:Int64(integer coordinates)Float64(lat/lon coordinates)TimestampNanos(datetime64[ns] coordinates)xarray supports a much wider range of coordinate types. Unrecognized types silently fall through with no partition pruning.
Missing types
UInt8,UInt16,UInt32,UInt64(unsigned integers)Int8,Int16,Int32(smaller integer widths)Float32(single-precision float)Work required
ScalarBoundwith variants for the missing typescompare_to_scalarto handle each new variant against matching DataFusionScalarValuetypespython_to_scalar_boundto extract these types from Python objectspartition_metadata()conversion indf.pyfor new typesAcceptance criteria