Skip to content

HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling - #10996

Merged
ChenSammi merged 2 commits into
apache:masterfrom
priyeshkaratha:HDDS-16158
Aug 13, 2026
Merged

HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling#10996
ChenSammi merged 2 commits into
apache:masterfrom
priyeshkaratha:HDDS-16158

Conversation

@priyeshkaratha

@priyeshkaratha priyeshkaratha commented Aug 12, 2026

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

sendDeleteKeysRequestAndClearList has two related issues that prevent delete-key requests from being handled correctly in production, particularly when ACLs are enabled and when FSO trash handling is configured.

First, sendDeleteKeysRequestAndClearList submits the Ratis request without executing preExecute under the appropriate UGI context. As a result, userInfo.userName remains blank. When ACLs are enabled, resolveBucketLink attempts to create a UGI from the blank user information and fails with UNAUTHORIZED.

Second, ozoneTrash is captured during KeyManager construction, before OzoneManager.startTrashEmptier() is invoked. Since all OzoneManager startup paths call keyManager.start() before starting the trash emptier, ozoneManager.getOzoneTrash() is null when the constructor runs. The field is never updated afterward, causing the FSO moveToTrash path in handleAndClearFullList to remain unreachable in production.

Fix

  • Wrap createClientRequest, preExecute, and submitRequest inside:
    ugi.doAs(UserGroupInformation.createRemoteUser(bucketOwner)), following the existing moveToTrash pattern.
  • Handle IOException from ugi.doAs() and createClientRequest() by converting it to ServiceException, alongside the existing InterruptedException handling.
  • Remove the constructor-time assignment of ozoneTrash.
  • Add getEffectiveOzoneTrash(), which returns the test-injected ozoneTrash when present, otherwise retrieves ozoneManager.getOzoneTrash() at call time after the trash emptier has started.
  • Update handleAndClearFullList to use getEffectiveOzoneTrash().
  • Preserve existing test behavior through setOzoneTrash(...).

This ensures the request is executed with the correct user context and that the FSO trash moveToTrash path is correctly reached in production.

What is the link to the Apache JIRA

HDDS-16158

How was this patch tested?

Tested manually in cluster with security and acl enabled. Also verified using integration test created using claude.

@priyeshkaratha
priyeshkaratha marked this pull request as ready for review August 12, 2026 04:17
ChenSammi
ChenSammi reviewed Aug 12, 2026
View reviewed changes
Comment thread ...zone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java
this.listMaxSize = size;
}

@VisibleForTesting

@ChenSammi ChenSammi Aug 12, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is removed on purpose?

@priyeshkaratha priyeshkaratha Aug 12, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function no longer used anywhere. So I just removed it.

@priyeshkaratha

priyeshkaratha commented Aug 12, 2026

Copy link
Copy Markdown
Member Author

@ChenSammi Thanks for reviewing the changes. I have addressed your comments.

ChenSammi
ChenSammi approved these changes Aug 13, 2026
View reviewed changes

@ChenSammi ChenSammi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @priyeshkaratha . The patch LGTM.

@ChenSammi
ChenSammi merged commit 749da66 into apache:master Aug 13, 2026
45 checks passed
errose28 added a commit to errose28/ozone that referenced this pull request Aug 14, 2026
@errose28
Merge branch 'master' into HDDS-14496-zdu-master-50978e3ad7
42c6bbc 
* master: (196 commits)
  HDDS-15422. Stream read seek should not close stream (apache#10415)
  HDDS-16089. Improve MPU table evaluation efficiency in KeyLifeCycleService (apache#10949)
  HDDS-16169. Add old updateContainerState to handle upgrade path. (apache#11012)
  HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling (apache#10996)
  HDDS-15911. Recon AI Assistant: add conversation memory for follow-up questions. (apache#10808)
  HDDS-15356. Make multi-buffer chunk checksum allocation-free (apache#10350)
  HDDS-15895. Make ServerNotLeaderException suggested leader flow IPv6-safe (apache#10807)
  HDDS-16118. Release snapshot DB handles before KeyDeletingService submits OM requests (apache#10987)
  HDDS-16125. Copy bucket snapshot in quota repair response to fix cache aliasing (apache#11004)
  HDDS-8082. Check for unnecessary newlines in config defaults (apache#10840)
  HDDS-15826. Recon: add cycle guard to NSSummary /du tree walks (apache#10723)
  HDDS-11620. Log SCM finalization completion (apache#10963)
  HDDS-16117. Failed FSO multipart complete leaks bucket namespace quota (apache#10984)
  HDDS-15961. Resolve linked bucket source properties consistently (apache#10859)
  HDDS-16072. Implement GetObjectAttributes: ETag, ObjectSize, StorageClass, and ObjectParts count (apache#10930).
  HDDS-16114. Avoid misleading Ozone mount warning in acceptance test results (apache#10979)
  HDDS-15761. Create test for OMMXBean (apache#10960)
  HDDS-15542. Closed-container reconciliation advances BCSID past a hole, masking missing chunks (apache#10496)
  HDDS-15776. Make S3 Gateway VirtualHostStyleFilter IPv6-safe (apache#10809)
  HDDS-15775. Support IPv6 literal authorities in OzoneFS URI parsing (apache#10933)
  ...

Files with conflicts or requiring updates:
---
hadoop-hdds/common/src/main/resources/ozone-default.xml
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/HeartbeatEndpointTask.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
hadoop-ozone/cli-debug/src/test/java/org/apache/hadoop/ozone/debug/om/TestContainerToKeyMapping.java
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestDNDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestScmDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/service/TestBlockDeletionService.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java
hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartRequestTests.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/InitialConstraintUpgradeAction.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/UnhealthyContainerReplicaMismatchAction.java
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientManager.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HDDSVersion.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/states/endpoint/TestHeartbeatEndpointTaskDnsRefresh.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineManagerImpl.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/node/TestDecommissionAndMaintenance.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationDeleteRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationSetRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleSaveScanStateRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/OMVersionManagerTestUtils.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/protocolPB/TestOzoneManagerRequestHandler.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/ReconUpgradeAction.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 17, 2026
@errose28
Merge branch 'master' into HDDS-14496-zdu-master-50978e3ad7
6f92947 
* master: (196 commits)
  HDDS-15422. Stream read seek should not close stream (apache#10415)
  HDDS-16089. Improve MPU table evaluation efficiency in KeyLifeCycleService (apache#10949)
  HDDS-16169. Add old updateContainerState to handle upgrade path. (apache#11012)
  HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling (apache#10996)
  HDDS-15911. Recon AI Assistant: add conversation memory for follow-up questions. (apache#10808)
  HDDS-15356. Make multi-buffer chunk checksum allocation-free (apache#10350)
  HDDS-15895. Make ServerNotLeaderException suggested leader flow IPv6-safe (apache#10807)
  HDDS-16118. Release snapshot DB handles before KeyDeletingService submits OM requests (apache#10987)
  HDDS-16125. Copy bucket snapshot in quota repair response to fix cache aliasing (apache#11004)
  HDDS-8082. Check for unnecessary newlines in config defaults (apache#10840)
  HDDS-15826. Recon: add cycle guard to NSSummary /du tree walks (apache#10723)
  HDDS-11620. Log SCM finalization completion (apache#10963)
  HDDS-16117. Failed FSO multipart complete leaks bucket namespace quota (apache#10984)
  HDDS-15961. Resolve linked bucket source properties consistently (apache#10859)
  HDDS-16072. Implement GetObjectAttributes: ETag, ObjectSize, StorageClass, and ObjectParts count (apache#10930).
  HDDS-16114. Avoid misleading Ozone mount warning in acceptance test results (apache#10979)
  HDDS-15761. Create test for OMMXBean (apache#10960)
  HDDS-15542. Closed-container reconciliation advances BCSID past a hole, masking missing chunks (apache#10496)
  HDDS-15776. Make S3 Gateway VirtualHostStyleFilter IPv6-safe (apache#10809)
  HDDS-15775. Support IPv6 literal authorities in OzoneFS URI parsing (apache#10933)
  ...

Files with conflicts or requiring updates:
---
hadoop-hdds/common/src/main/resources/ozone-default.xml
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/HeartbeatEndpointTask.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
hadoop-ozone/cli-debug/src/test/java/org/apache/hadoop/ozone/debug/om/TestContainerToKeyMapping.java
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestDNDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestScmDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/service/TestBlockDeletionService.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java
hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartRequestTests.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/InitialConstraintUpgradeAction.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/UnhealthyContainerReplicaMismatchAction.java
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientManager.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HDDSVersion.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/states/endpoint/TestHeartbeatEndpointTaskDnsRefresh.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineManagerImpl.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/node/TestDecommissionAndMaintenance.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestFinalizeBlock.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmStartupInvalidApparentVersion.java
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationDeleteRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationSetRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleSaveScanStateRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/OMVersionManagerTestUtils.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/protocolPB/TestOzoneManagerRequestHandler.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMLayoutFeature.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/ReconUpgradeAction.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 17, 2026
@errose28
Merge branch 'master' into HDDS-14496-zdu-master-50978e3ad7
b13a863 
* master: (196 commits)
  HDDS-15422. Stream read seek should not close stream (apache#10415)
  HDDS-16089. Improve MPU table evaluation efficiency in KeyLifeCycleService (apache#10949)
  HDDS-16169. Add old updateContainerState to handle upgrade path. (apache#11012)
  HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling (apache#10996)
  HDDS-15911. Recon AI Assistant: add conversation memory for follow-up questions. (apache#10808)
  HDDS-15356. Make multi-buffer chunk checksum allocation-free (apache#10350)
  HDDS-15895. Make ServerNotLeaderException suggested leader flow IPv6-safe (apache#10807)
  HDDS-16118. Release snapshot DB handles before KeyDeletingService submits OM requests (apache#10987)
  HDDS-16125. Copy bucket snapshot in quota repair response to fix cache aliasing (apache#11004)
  HDDS-8082. Check for unnecessary newlines in config defaults (apache#10840)
  HDDS-15826. Recon: add cycle guard to NSSummary /du tree walks (apache#10723)
  HDDS-11620. Log SCM finalization completion (apache#10963)
  HDDS-16117. Failed FSO multipart complete leaks bucket namespace quota (apache#10984)
  HDDS-15961. Resolve linked bucket source properties consistently (apache#10859)
  HDDS-16072. Implement GetObjectAttributes: ETag, ObjectSize, StorageClass, and ObjectParts count (apache#10930).
  HDDS-16114. Avoid misleading Ozone mount warning in acceptance test results (apache#10979)
  HDDS-15761. Create test for OMMXBean (apache#10960)
  HDDS-15542. Closed-container reconciliation advances BCSID past a hole, masking missing chunks (apache#10496)
  HDDS-15776. Make S3 Gateway VirtualHostStyleFilter IPv6-safe (apache#10809)
  HDDS-15775. Support IPv6 literal authorities in OzoneFS URI parsing (apache#10933)
  ...

Files with conflicts or requiring updates:
---
hadoop-hdds/common/src/main/resources/ozone-default.xml
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/HeartbeatEndpointTask.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
hadoop-ozone/cli-debug/src/test/java/org/apache/hadoop/ozone/debug/om/TestContainerToKeyMapping.java
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestDNDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestScmDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/service/TestBlockDeletionService.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java
hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartRequestTests.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/InitialConstraintUpgradeAction.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/UnhealthyContainerReplicaMismatchAction.java
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientManager.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HDDSVersion.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/states/endpoint/TestHeartbeatEndpointTaskDnsRefresh.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineManagerImpl.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/node/TestDecommissionAndMaintenance.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestFinalizeBlock.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmStartupInvalidApparentVersion.java
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationDeleteRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationSetRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleSaveScanStateRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/OMVersionManagerTestUtils.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/protocolPB/TestOzoneManagerRequestHandler.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMLayoutFeature.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/ReconUpgradeAction.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 19, 2026
@errose28
Merge branch 'master' into HDDS-14496-zdu
8b993e0 
* master: (196 commits)
  HDDS-15422. Stream read seek should not close stream (apache#10415)
  HDDS-16089. Improve MPU table evaluation efficiency in KeyLifeCycleService (apache#10949)
  HDDS-16169. Add old updateContainerState to handle upgrade path. (apache#11012)
  HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling (apache#10996)
  HDDS-15911. Recon AI Assistant: add conversation memory for follow-up questions. (apache#10808)
  HDDS-15356. Make multi-buffer chunk checksum allocation-free (apache#10350)
  HDDS-15895. Make ServerNotLeaderException suggested leader flow IPv6-safe (apache#10807)
  HDDS-16118. Release snapshot DB handles before KeyDeletingService submits OM requests (apache#10987)
  HDDS-16125. Copy bucket snapshot in quota repair response to fix cache aliasing (apache#11004)
  HDDS-8082. Check for unnecessary newlines in config defaults (apache#10840)
  HDDS-15826. Recon: add cycle guard to NSSummary /du tree walks (apache#10723)
  HDDS-11620. Log SCM finalization completion (apache#10963)
  HDDS-16117. Failed FSO multipart complete leaks bucket namespace quota (apache#10984)
  HDDS-15961. Resolve linked bucket source properties consistently (apache#10859)
  HDDS-16072. Implement GetObjectAttributes: ETag, ObjectSize, StorageClass, and ObjectParts count (apache#10930).
  HDDS-16114. Avoid misleading Ozone mount warning in acceptance test results (apache#10979)
  HDDS-15761. Create test for OMMXBean (apache#10960)
  HDDS-15542. Closed-container reconciliation advances BCSID past a hole, masking missing chunks (apache#10496)
  HDDS-15776. Make S3 Gateway VirtualHostStyleFilter IPv6-safe (apache#10809)
  HDDS-15775. Support IPv6 literal authorities in OzoneFS URI parsing (apache#10933)
  ...

Files with conflicts or requiring updates:
---
hadoop-hdds/common/src/main/resources/ozone-default.xml
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/HeartbeatEndpointTask.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
hadoop-ozone/cli-debug/src/test/java/org/apache/hadoop/ozone/debug/om/TestContainerToKeyMapping.java
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestDNDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestScmDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/service/TestBlockDeletionService.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java
hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartRequestTests.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/InitialConstraintUpgradeAction.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/UnhealthyContainerReplicaMismatchAction.java
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientManager.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HDDSVersion.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/states/endpoint/TestHeartbeatEndpointTaskDnsRefresh.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineManagerImpl.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/node/TestDecommissionAndMaintenance.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestFinalizeBlock.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmStartupInvalidApparentVersion.java
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationDeleteRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationSetRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleSaveScanStateRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/OMVersionManagerTestUtils.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/protocolPB/TestOzoneManagerRequestHandler.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMLayoutFeature.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/ReconUpgradeAction.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 25, 2026
@errose28
Merge branch 'master' into HDDS-14496-zdu
6b19241 
* master: (196 commits)
  HDDS-15422. Stream read seek should not close stream (apache#10415)
  HDDS-16089. Improve MPU table evaluation efficiency in KeyLifeCycleService (apache#10949)
  HDDS-16169. Add old updateContainerState to handle upgrade path. (apache#11012)
  HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling (apache#10996)
  HDDS-15911. Recon AI Assistant: add conversation memory for follow-up questions. (apache#10808)
  HDDS-15356. Make multi-buffer chunk checksum allocation-free (apache#10350)
  HDDS-15895. Make ServerNotLeaderException suggested leader flow IPv6-safe (apache#10807)
  HDDS-16118. Release snapshot DB handles before KeyDeletingService submits OM requests (apache#10987)
  HDDS-16125. Copy bucket snapshot in quota repair response to fix cache aliasing (apache#11004)
  HDDS-8082. Check for unnecessary newlines in config defaults (apache#10840)
  HDDS-15826. Recon: add cycle guard to NSSummary /du tree walks (apache#10723)
  HDDS-11620. Log SCM finalization completion (apache#10963)
  HDDS-16117. Failed FSO multipart complete leaks bucket namespace quota (apache#10984)
  HDDS-15961. Resolve linked bucket source properties consistently (apache#10859)
  HDDS-16072. Implement GetObjectAttributes: ETag, ObjectSize, StorageClass, and ObjectParts count (apache#10930).
  HDDS-16114. Avoid misleading Ozone mount warning in acceptance test results (apache#10979)
  HDDS-15761. Create test for OMMXBean (apache#10960)
  HDDS-15542. Closed-container reconciliation advances BCSID past a hole, masking missing chunks (apache#10496)
  HDDS-15776. Make S3 Gateway VirtualHostStyleFilter IPv6-safe (apache#10809)
  HDDS-15775. Support IPv6 literal authorities in OzoneFS URI parsing (apache#10933)
  ...

Files with conflicts or requiring updates:
---
hadoop-hdds/common/src/main/resources/ozone-default.xml
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/HeartbeatEndpointTask.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
hadoop-ozone/cli-debug/src/test/java/org/apache/hadoop/ozone/debug/om/TestContainerToKeyMapping.java
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestDNDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestScmDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/service/TestBlockDeletionService.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java
hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartRequestTests.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/InitialConstraintUpgradeAction.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/UnhealthyContainerReplicaMismatchAction.java
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientManager.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HDDSVersion.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/states/endpoint/TestHeartbeatEndpointTaskDnsRefresh.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineManagerImpl.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/node/TestDecommissionAndMaintenance.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestFinalizeBlock.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmStartupInvalidApparentVersion.java
hadoop-ozone/mini-cluster/src/main/java/org/apache/hadoop/ozone/UniformDatanodesFactory.java
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationDeleteRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationSetRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleSaveScanStateRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/OMVersionManagerTestUtils.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/protocolPB/TestOzoneManagerRequestHandler.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMLayoutFeature.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/ReconUpgradeAction.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 25, 2026
@errose28
Merge branch 'master' into HDDS-14496-zdu
71a27e3 
* master: (196 commits)
  HDDS-15422. Stream read seek should not close stream (apache#10415)
  HDDS-16089. Improve MPU table evaluation efficiency in KeyLifeCycleService (apache#10949)
  HDDS-16169. Add old updateContainerState to handle upgrade path. (apache#11012)
  HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling (apache#10996)
  HDDS-15911. Recon AI Assistant: add conversation memory for follow-up questions. (apache#10808)
  HDDS-15356. Make multi-buffer chunk checksum allocation-free (apache#10350)
  HDDS-15895. Make ServerNotLeaderException suggested leader flow IPv6-safe (apache#10807)
  HDDS-16118. Release snapshot DB handles before KeyDeletingService submits OM requests (apache#10987)
  HDDS-16125. Copy bucket snapshot in quota repair response to fix cache aliasing (apache#11004)
  HDDS-8082. Check for unnecessary newlines in config defaults (apache#10840)
  HDDS-15826. Recon: add cycle guard to NSSummary /du tree walks (apache#10723)
  HDDS-11620. Log SCM finalization completion (apache#10963)
  HDDS-16117. Failed FSO multipart complete leaks bucket namespace quota (apache#10984)
  HDDS-15961. Resolve linked bucket source properties consistently (apache#10859)
  HDDS-16072. Implement GetObjectAttributes: ETag, ObjectSize, StorageClass, and ObjectParts count (apache#10930).
  HDDS-16114. Avoid misleading Ozone mount warning in acceptance test results (apache#10979)
  HDDS-15761. Create test for OMMXBean (apache#10960)
  HDDS-15542. Closed-container reconciliation advances BCSID past a hole, masking missing chunks (apache#10496)
  HDDS-15776. Make S3 Gateway VirtualHostStyleFilter IPv6-safe (apache#10809)
  HDDS-15775. Support IPv6 literal authorities in OzoneFS URI parsing (apache#10933)
  ...

Files with conflicts or requiring updates:
---
hadoop-hdds/common/src/main/resources/ozone-default.xml
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/HeartbeatEndpointTask.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/replication/TestReplicationSupervisor.java
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/server/upgrade/TestScmVersionManager.java
hadoop-ozone/cli-debug/src/test/java/org/apache/hadoop/ozone/debug/om/TestContainerToKeyMapping.java
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/OmUtils.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestDNDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/upgrade/TestScmDataDistributionFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/service/TestBlockDeletionService.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/shell/TestOzoneShellHA.java
hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/s3/multipart/S3MultipartRequestTests.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/InitialConstraintUpgradeAction.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/UnhealthyContainerReplicaMismatchAction.java
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientManager.java
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HDDSVersion.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/states/endpoint/TestHeartbeatEndpointTaskDnsRefresh.java
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/upgrade/TestDatanodeVersionManager.java
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestPipelineManagerImpl.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/hdds/scm/node/TestDecommissionAndMaintenance.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOMUpgradeFinalization.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestFinalizeBlock.java
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/om/TestOmStartupInvalidApparentVersion.java
hadoop-ozone/mini-cluster/src/main/java/org/apache/hadoop/ozone/UniformDatanodesFactory.java
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/service/KeyLifecycleService.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationDeleteRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleConfigurationSetRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/request/lifecycle/TestOMLifecycleSaveScanStateRequest.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/OMVersionManagerTestUtils.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/protocolPB/TestOzoneManagerRequestHandler.java
hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/upgrade/TestOMLayoutFeature.java
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/upgrade/ReconUpgradeAction.java
errose28 added a commit to errose28/ozone that referenced this pull request Aug 26, 2026
@errose28
Merge branch 'HDDS-14496-zdu' into worktree/om-finalize-with-old-clis
01d39ab 
* HDDS-14496-zdu: (200 commits)
  HDDS-16190. Do not use unified version framework in OM until ZDU is finalized. (apache#11025)
  HDDS-16154. Datanode should assign DatanodeDetails#currentVersion based on its apparent version (apache#11006)
  HDDS-16153. DatanodeDetails should use HDDSVersion instead of int (apache#10994)
  HDDS-15549. Recon's INITIAL_VERSION should not have an upgrade action (apache#10935)
  HDDS-15422. Stream read seek should not close stream (apache#10415)
  HDDS-16089. Improve MPU table evaluation efficiency in KeyLifeCycleService (apache#10949)
  HDDS-16169. Add old updateContainerState to handle upgrade path. (apache#11012)
  HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling (apache#10996)
  HDDS-15911. Recon AI Assistant: add conversation memory for follow-up questions. (apache#10808)
  HDDS-15356. Make multi-buffer chunk checksum allocation-free (apache#10350)
  HDDS-15895. Make ServerNotLeaderException suggested leader flow IPv6-safe (apache#10807)
  HDDS-16118. Release snapshot DB handles before KeyDeletingService submits OM requests (apache#10987)
  HDDS-16125. Copy bucket snapshot in quota repair response to fix cache aliasing (apache#11004)
  HDDS-8082. Check for unnecessary newlines in config defaults (apache#10840)
  HDDS-15826. Recon: add cycle guard to NSSummary /du tree walks (apache#10723)
  HDDS-11620. Log SCM finalization completion (apache#10963)
  HDDS-16117. Failed FSO multipart complete leaks bucket namespace quota (apache#10984)
  HDDS-15961. Resolve linked bucket source properties consistently (apache#10859)
  HDDS-16072. Implement GetObjectAttributes: ETag, ObjectSize, StorageClass, and ObjectParts count (apache#10930).
  HDDS-16114. Avoid misleading Ozone mount warning in acceptance test results (apache#10979)
  ...

Conflicts:
hadoop-ozone/interface-client/src/main/proto/OmClientProtocol.proto
errose28 added a commit to errose28/ozone that referenced this pull request Aug 26, 2026
@errose28
Merge branch 'HDDS-14496-zdu' into worktree/failing-dn-version
e46408b 
* HDDS-14496-zdu: (197 commits)
  HDDS-16190. Do not use unified version framework in OM until ZDU is finalized. (apache#11025)
  HDDS-15422. Stream read seek should not close stream (apache#10415)
  HDDS-16089. Improve MPU table evaluation efficiency in KeyLifeCycleService (apache#10949)
  HDDS-16169. Add old updateContainerState to handle upgrade path. (apache#11012)
  HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling (apache#10996)
  HDDS-15911. Recon AI Assistant: add conversation memory for follow-up questions. (apache#10808)
  HDDS-15356. Make multi-buffer chunk checksum allocation-free (apache#10350)
  HDDS-15895. Make ServerNotLeaderException suggested leader flow IPv6-safe (apache#10807)
  HDDS-16118. Release snapshot DB handles before KeyDeletingService submits OM requests (apache#10987)
  HDDS-16125. Copy bucket snapshot in quota repair response to fix cache aliasing (apache#11004)
  HDDS-8082. Check for unnecessary newlines in config defaults (apache#10840)
  HDDS-15826. Recon: add cycle guard to NSSummary /du tree walks (apache#10723)
  HDDS-11620. Log SCM finalization completion (apache#10963)
  HDDS-16117. Failed FSO multipart complete leaks bucket namespace quota (apache#10984)
  HDDS-15961. Resolve linked bucket source properties consistently (apache#10859)
  HDDS-16072. Implement GetObjectAttributes: ETag, ObjectSize, StorageClass, and ObjectParts count (apache#10930).
  HDDS-16114. Avoid misleading Ozone mount warning in acceptance test results (apache#10979)
  HDDS-15761. Create test for OMMXBean (apache#10960)
  HDDS-15542. Closed-container reconciliation advances BCSID past a hole, masking missing chunks (apache#10496)
  HDDS-15776. Make S3 Gateway VirtualHostStyleFilter IPv6-safe (apache#10809)
  ...

Conflicts:
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/states/endpoint/HeartbeatEndpointTask.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ChenSammi ChenSammi ChenSammi approved these changes

Assignees

No one assigned

Labels

AI-gen s3-lifecycle HDDS-8342 s3 S3 Gateway

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@priyeshkaratha @ChenSammi