Skip to content

HDDS-16036. SCM Support allocate Container by StoragePolicy - #10946

Open
xichen01 wants to merge 5 commits into
apache:HDDS-11233from
xichen01:HDDS-16036
Open

HDDS-16036. SCM Support allocate Container by StoragePolicy#10946
xichen01 wants to merge 5 commits into
apache:HDDS-11233from
xichen01:HDDS-16036

Conversation

@xichen01

@xichen01 xichen01 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

  • Support allocate Container by StoragePolicy in the SCM side
  • SCM side allocateBlock interface add parameter StoragePolicy and allowFallbackStoragePolicy
    • After this PR, allocateBlock must past the parameter StoragePolicy and allowFallbackStoragePolicy.

For the old client the StoragePolicy will be null, the SCM will use OzoneStoragePolicy.getDefaultPolicy() as the default StoragePolicy for the old client,
The default policy can be configured by Configuration ozone.default.storagepolicy the default value is WARM

FYI:

Releated doc and discuss: #6989
A preview version for the full "Ozone Storage Policy" feature: https://github.com/ivandika3/ozone/tree/refs/heads/backport-storage-policy-storage-class

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16036

How was this patch tested?

New added tests

@xichen01
xichen01 marked this pull request as draft August 4, 2026 11:55
@devmadhuu
devmadhuu requested a review from ivandika3 August 4, 2026 14:02
@devmadhuu

Copy link
Copy Markdown
Contributor

@xichen01 is this PR still in progress ?

@xichen01
xichen01 marked this pull request as ready for review August 4, 2026 17:16
@xichen01

xichen01 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

@xichen01 is this PR still in progress ?

I'm just fixing up the test, it's ready for review.

@xichen01

xichen01 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

@amaliujia @greenwich @chungen0126 @ivandika3 @peterxcli @devmadhuu Please help to review

@devmadhuu

Copy link
Copy Markdown
Contributor

Added HDDS-16116 for rebasing HDDS-11233 feature branch with master. Keep hold till rebase finished.

@devmadhuu
devmadhuu force-pushed the HDDS-11233 branch 2 times, most recently from c6cc43d to 8decb32 Compare August 10, 2026 09:52

@chungen0126 chungen0126 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xichen01 for working on this.

Comment on lines 717 to 718
public static final String OZONE_DEFAULT_STORAGE_TIER_KEY =
"ozone.default.storageTier";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to change this to ozone.server.default.storage.tier.

public static final String OZONE_DEFAULT_STORAGE_TIER_DEFAULT =
StorageTier.DISK.toString();

public static final String OZONE_DEFAULT_STORAGE_POLICY_KEY = "ozone.default.storagepolicy";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to change this to ozone.server.default.storage.policy.

</property>

<property>
<name>ozone.default.storagepolicy</name>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to change this to ozone.server.default.storage.policy.

Comment on lines -64 to -71
@Deprecated
default List<AllocatedBlock> allocateBlock(long size, int numBlocks,
ReplicationType type, ReplicationFactor factor, String owner,
ExcludeList excludeList) throws IOException, TimeoutException {
return allocateBlock(size, numBlocks, ReplicationConfig
.fromProtoTypeAndFactor(type, factor), owner, excludeList);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since ScmBlockLocationProtocol is a public RPC protocol interface, removing this method changes the interface signatures and may break binary/backward compatibility for older clients or dependencies that haven't been recompiled against the new version (causing NoSuchMethodError at runtime).

Is there a reason to remove this deprecated default method in this PR, or should we retain it for backward compatibility?

@devmadhuu

Copy link
Copy Markdown
Contributor

@xichen01 I updated the shared feature branch HDDS-11233 on apache/ozone master to bring it up to date with master. The branch now = the original feature commits (unchanged SHAs) with a single Merge 'upstream/master' commit on top. New tip: 8decb32. Kindly do merge of this feature branch to your PR branch and resolve the conflicts.

@devmadhuu

Copy link
Copy Markdown
Contributor

@xichen01 I updated the shared feature branch HDDS-11233 on apache/ozone master to bring it up to date with master. The branch now = the original feature commits (unchanged SHAs) with a single Merge 'upstream/master' commit on top. New tip: 8decb32. Kindly do merge of this feature branch to your PR branch and resolve the conflicts.

Apologies for this confusion. Yes I rebased and force pushed the HDDS-11233 , but we need to follow the merge process instead of rebase for feature branches and raise PRs and follow the normal PR process. So I reverted the earlier forced push and now raised this PR #10983
Kindly wait till this PR gets merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants