Optionally retain per-file column statistics on FileScanTask #3422

Is your feature request related to a problem or challenge?

ManifestEntryContext::into_file_scan_task drops the data file's column statistics when it builds a FileScanTask. The task has no lower or upper bounds and no value, null or NaN counts, so an engine reading the tasks cannot reason about a file's key range without going back to the manifests.

Sorted scans need this. apache/datafusion-iceberg#55 keeps a sorted scan's order by merging one reader per file, which opens every file at once. With per-file bounds on the sort columns it could read files whose key ranges don't overlap one after another, and only merge where they do. apache/datafusion-comet#5343 needs the same information.

This is the "Add opt-in file statistics retention to scan tasks" item in #3343. In #607 there was pushback on putting statistics on every task, because it makes serialized tasks larger when they are shipped to workers and because it could expose key_metadata. So this has to be opt-in, and it should add nothing when it isn't used.

Describe the solution you'd like

Add TableScanBuilder::with_column_stats(columns), mirroring Java's Scan.includeColumnStats(Collection<String>). The names resolve to field ids when the scan is built, using the scan's case sensitivity, and an unknown name fails the way select does.

Each planned FileScanTask would then expose column_stats(), returning a new FileScanTaskColumnStats that holds the five maps (value counts, null value counts, NaN value counts, lower bounds and upper bounds) filtered to the requested field ids. The maps would match the shape of FileMetrics in #3347 so the two convert directly. The docs would state what the statistics mean: bounds are inclusive and may be truncated, any of them can be missing, they describe the whole file without adjusting for deletes, filters or splits, and they are typed with the schema the manifest was written with.

When nothing is requested, column_stats() is None, planning copies nothing, and serialization leaves the field out, so serialized tasks stay byte-identical to today. No other DataFile fields, including key_metadata, are exposed.

Willingness to contribute

I can contribute to this feature independently