fix: don't let S3A committer, delete and read defaults block native Iceberg writes

[andygrove]

Which issue does this PR close?

No issue. This follows up on #6441.

Rationale for this change

#6441 makes a native Iceberg write to S3 fall back to Spark when the Hadoop configuration has an fs.s3a.* setting the native writer doesn't support. It already ignores Hadoop's built-in defaults and a few S3A read settings that Spark seeds into every session. Some other settings are also common in S3 deployments, and any one of them makes every native S3 write fall back:

• Since Spark 4.1, SparkContext sets fs.s3a.committer.magic.enabled=true and fs.s3a.committer.name=magic for every application when spark-hadoop-cloud is on the classpath (SPARK-47618).
• Many deployments also set fs.s3a.committer.threads, fs.s3a.experimental.input.fadvise (Hadoop's S3A docs recommend random for columnar formats) or fs.s3a.bulk.delete.page.size cluster-wide.

The fallback reason looks like this:

unsupported Hadoop S3A settings: fs.s3a.bulk.delete.page.size, fs.s3a.committer.magic.enabled, fs.s3a.committer.name, fs.s3a.committer.threads, fs.s3a.experimental.input.fadvise

None of these settings affects an Iceberg data-file write:

• The native writer doesn't go through S3A, so S3A's delete batching doesn't apply.
• fadvise is a read hint.
• Iceberg commits through table metadata, not through a Hadoop output committer.

What changes are included in this PR?

The five keys are added to IgnoredHadoopS3Keys in CometIcebergNativeWrite, alongside the Spark-seeded read settings, with a comment explaining why.

How are these changes tested?

There's a new test in CometIcebergWriteDetectionSuite. It checks that these settings produce no unsupported keys, and that a setting the native writer can't honor (fs.s3a.encryption.algorithm) is still reported next to them. The whole suite passes locally.

[manuzhang]

LGTM overall. Some minor questions.

1. Do we plan to ignore more common configs?
2. Do we need to update user and contributor guide as well?

[andygrove]

Not a broader list, no. I've only added keys that S3A wouldn't apply to an Iceberg data-file write and that arrive as cluster-wide defaults rather than a per-job choice: Spark 4.1 sets the two committer keys itself whenever spark-hadoop-cloud is on the classpath, and the other three are common platform-level tuning. Anything S3A does apply to a write that the native writer never sees (encryption, ACLs, storage class, signing, the credentials provider) has to keep falling back, which is what #6441 is for. I'd rather add a key when someone hits it blocking native writes than guess at the long tail. Is there one you've run into?

And yes to the guides. Both name the ignored keys (vectored-read and downgrade.syncable.exceptions), so they're stale after this change. I put the update for both in a separate docs-only PR, #6828, to avoid re-running CI here. It should merge after this one.

[andygrove]

Thanks @manuzhang @snmvaughan