Core: Reduce visibility on scan planning response builder for specsById and deleteFiles

[dramaticlly]

Reduces visibility on the scan-planning response internals that were deprecated in 1.11.0 with visibility to be reduced in 1.12.0.

.palantir/revapi.yml: add 9 entries with

• 4 noLongerDeprecated
• 4 visibilityReduced
• 1 method.removed.

1. Reduce visibility

The specs map and derived delete files are serialization internals, not part of the response payload:

• specsById() on the response and on Builder → protected
• Builder.withSpecsById(Map) → protected
• Builder.deleteFiles() → protected
• Builder.withDeleteFiles(List) removed — delete files are always derived from the tasks that reference them

Since withSpecsById is no longer publicly callable, servers need another way to supply the specs and response copiers need a way to carry them across:

• builder(Map<Integer, PartitionSpec>) on PlanTableScanResponse, FetchPlanningResultResponse and FetchScanTasksResponse
• toBuilder() on PlanTableScanResponse and FetchPlanningResultResponse

CatalogHandlers passes table.specs() to builder(...) at the three scan-planning call sites. RESTServerCatalogAdapter uses toBuilder() to inject storage credentials rather than rebuilding field by field.

2. Clear derived delete files when file scan tasks are cleared

BaseScanTaskResponse.Builder derives deleteFiles from fileScanTasks, but only refreshed the derived set when the incoming list was non-null:

this.fileScanTasks = tasks;
if (fileScanTasks != null) {
  this.deleteFiles = DeleteFileSet.of(...);
}

So withFileScanTasks(null) after a non-null call left the previously derived delete files behind, and the response then failed its own validate():

IllegalArgumentException: Invalid response: deleteFiles should only be returned
with fileScanTasks that reference them

even though the caller had explicitly cleared the tasks. This is latent on main but becomes load-bearing in commit 2, where toBuilder() makes builder reuse routine — TestRESTScanPlanning now relies on exactly this path to turn a COMPLETED response into a FAILED one.

Covered by two new tests at different entry points: TestFetchScanTasksResponseParser.clearingFileScanTasksAlsoClearsDerivedDeleteFiles (direct builder) and TestPlanTableScanResponseParser.toBuilderClearsDeleteFilesWhenClearingFileScanTasks (via toBuilder()). Both fail without the fix.

Callout

1. I've added toBuilder() as a new public API for out-of-package org.apache.iceberg.rest caller to copy a response with the specs.

2. Note withCredentials appends rather than replaces, so toBuilder().withCredentials(extra) preserves any existing credentials; the TestRESTScanPlanning simplification is behavior-preserving.

Split out of #16449 to reduce reviewer burden.

AI Disclosure

Model: Claude Opus 5 (1M context)
Platform/Tool: Claude Code
Human Oversight: reviewed
Prompt Summary: split #16449 into smaller self-contained PRs; verify each group compiles and tests green standalone

[dramaticlly]

FYI @amogh-jahagirdar if you can help check against #14838
and also @singhpk234 for #14485

[RussellSpitzer]

Why is this changing?

[dramaticlly]

thanks @RussellSpitzer, basically we need a way to clean the deleteFiles after the withDeleteFiles setter has been removed.

Given the delete files can only be derived from from fileScanTasks, use withFileScanTasks(null) would allow clear the internal state of this.deleteFiles.

[amogh-jahagirdar]

Note withCredentials appends rather than replaces, so toBuilder().withCredentials(extra) preserves any existing credentials; the TestRESTScanPlanning simplification is behavior-preserving.

This is actually quite surprising imo. I know it's an existing behavior but I feel like that's definitley not expected from a builder API; I forget if this was intentional or not, maybe @singhpk234 remembers. I guess it's not an issue hit in practice but I feel like we should have this replace credentials, and if we have a need for adding, we could do an addCredentials.

[amogh-jahagirdar]

I think independently this change makes sense though the withCredentials behavior (which I get is existing) is a bit surprising. that's something we can take separately though since it is existing behavior.

[dramaticlly]

Note withCredentials appends rather than replaces, so toBuilder().withCredentials(extra) preserves any existing credentials; the TestRESTScanPlanning simplification is behavior-preserving.

This is actually quite surprising imo. I know it's an existing behavior but I feel like that's definitley not expected from a builder API; I forget if this was intentional or not, maybe @singhpk234 remembers. I guess it's not an issue hit in practice but I feel like we should have this replace credentials, and if we have a need for adding, we could do an addCredentials.

Thanks @amogh-jahagirdar , once we figured out the expected behaviour, happy to follow up and give a pass for all builder method deal with collections.

[amogh-jahagirdar]

I'll go ahead and merge. Thanks @dramaticlly !

[amogh-jahagirdar]

and thanks @uros-b @RussellSpitzer for reviewing!