fix(moore): select MCCL bfloat16 architecture - #69
Merged
Conversation
voltjia
force-pushed
the
fix/moore-mccl-bf16-architecture
branch
from
September 7, 2026 04:04
f74ce80 to
559f7f8
Compare
53 tasks
- Move macro-based architecture utilities into `cmake/architecture` and keep MUSA-specific resolution in `MusaArchitecture.cmake`. - The Moore build path now loads the MUSA helper only when `WITH_MOORE` is enabled - Architecture resolution is delegated to a single helper after SDK discovery.
Ziminli
force-pushed
the
fix/moore-mccl-bf16-architecture
branch
from
September 8, 2026 10:42
559f7f8 to
ca71487
Compare
… `cmake/architecture/MusaArchitecture.cmake`
Ziminli
approved these changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moore MCCL exposes
mcclBfloat16only whenMARCH_TYPEdescribes a capable target architecture. InfiniCCL previously passed neitherMARCH_TYPEnor an offload architecture tomcc, so BF16 collectives returnedinfinicclNotSupportedeven on S5000 hardware where MCCL supports them.This PR adds explicit and auto-detected MUSA architecture configuration, passes the matching compiler flags, and keeps mixed-architecture builds fail-closed. The architecture handling is organized under reusable CMake helpers so future SDKs with macro-defined capability surfaces can follow the same pattern without duplicating the common logic.
Changes
Architecture selection
MUSA_ARCHITECTURES, accepting forms such as31,mp_31, and3.1.TORCH_MUSA_ARCH_LIST, then native detection throughmusaGetDeviceProperties.cmake/architecture/MusaArchitecture.cmake, keeping the root Moore CMake block focused on SDK and library discovery.cmake/architecture/MacroArchitecture.cmakefor normalization, conservative macro selection, compiler-option generation, and target-scoped application.MCCL capability mapping
--offload-arch=mp_*flag per configured target.MARCH_TYPEfrom the least capable target becausemccl.hexposes one global data-type table; this prevents a mixed fat binary from advertising BF16 on an unsupported target.Platform and Backend Affected
Platform
Backend
Performance Impact
N/A. This changes build-time target selection and enables an existing MCCL data type.
Known Issues & Future Work
MUSA_ARCHITECTURESorTORCH_MUSA_ARCH_LISTexplicitly.Test Results
Test Involved Platform
Test Involved Backend
Validated on Moore Threads S5000 with MUSA 4.3.6:
This commit was also combined with the open AllGather/Send/Recv stack at
e8ccc0cbfor InfiniLM validation. Qwen3-32B TP4 and Baichuan2-7B TP2 paged FlashAttention graph inference both passed after BF16 AllReduce became available.Moore MCCL + OpenMPI (single node):
ccl_mpi_hybrid_all_reduce.log
Moore MCCL (single node):
ccl_all_reduce.log
Checklist
Title, Branch, and Commits
masterwith no merge commit.Scope and Design
General Code Hygiene
CONTRIBUTING.md§Code/General).CONTRIBUTING.md§Code/General).the `AllReduce` implementation) (CONTRIBUTING.md§Code/General).CONTRIBUTING.md§Code/General).CONTRIBUTING.md§Code/General; §Python).C++ Specific
N/A. No repository C++ source file is modified; the native detector is generated by CMake during configuration.
Python Specific
N/A. No Python file is modified.
Testing
Build, CI, and Tooling
WITH_MOORE=ON.Documentation
Security and Safety