Context. #6031 adds CometS3LocationScopedCredentialProvider: the Parquet path's LocationScopedObjectStore keeps the provider's policy locations as a snapshot and fetches them again only when S3 returns 403 (review).
Problem. A provider with no policy for the path it is asked about usually throws from getCredentialsForPath instead of returning a credential that S3 then rejects. The bridge turns that exception into object_store::Error::Generic, which does not trigger the refresh. Two ways this shows up:
- The provider has no bucket-wide policy and throws for
/, then adds warehouse/finance mid-job. Reads under it route to / and keep failing, so the new location is never picked up.
- The provider folds
warehouse/finance into warehouse and throws for the old location. Later reads under it keep failing, while a freshly built store would serve the same path from /warehouse.
The store lives as long as the executor, so both stay broken until a restart. The user guide says a location added while a job runs is picked up, and suggests keying the provider's cache on the location, which leads naturally to that throw.
Options.
- (a) Treat a credential failure from a location's store like a 403: mark the bridge's errors with a dedicated error type the store can recognize, and run the same refresh and single retry.
- (b) Require
getCredentialsForPath to answer for / and for locations the provider has dropped, and say so in the Javadoc and user guide.
Proposal. (a). It keeps the documented behavior and a simple vendor contract, and it changes only the implementation. If we choose (b) instead, it should land before CometS3LocationScopedCredentialProvider first ships, since the versioning policy treats a change in a method's meaning as breaking.
Context. #6031 adds
CometS3LocationScopedCredentialProvider: the Parquet path'sLocationScopedObjectStorekeeps the provider's policy locations as a snapshot and fetches them again only when S3 returns 403 (review).Problem. A provider with no policy for the path it is asked about usually throws from
getCredentialsForPathinstead of returning a credential that S3 then rejects. The bridge turns that exception intoobject_store::Error::Generic, which does not trigger the refresh. Two ways this shows up:/, then addswarehouse/financemid-job. Reads under it route to/and keep failing, so the new location is never picked up.warehouse/financeintowarehouseand throws for the old location. Later reads under it keep failing, while a freshly built store would serve the same path from/warehouse.The store lives as long as the executor, so both stay broken until a restart. The user guide says a location added while a job runs is picked up, and suggests keying the provider's cache on the location, which leads naturally to that throw.
Options.
getCredentialsForPathto answer for/and for locations the provider has dropped, and say so in the Javadoc and user guide.Proposal. (a). It keeps the documented behavior and a simple vendor contract, and it changes only the implementation. If we choose (b) instead, it should land before
CometS3LocationScopedCredentialProviderfirst ships, since the versioning policy treats a change in a method's meaning as breaking.