Repository navigation
musa: build the docker image and CI container from the MUSA SDK images - #29624
Merged
JohannesGaessler merged 4 commits intoSep 29, 2026
Merged
Conversation
Use registry.mthreads.com/mcconline/musa_sdk:5.2.0-{devel,runtime}-ubuntu22.04-s5000
instead of registry.mthreads.com/mcconline/inference/pytorch:2.9.1.post1-py3.10-musa5.2.0-mp31-devel-ubuntu22.04-amd64
for the MUSA docker image and the MUSA CI container, and let the runtime stage use the
runtime image instead of reusing the devel one, which drops the MUSA toolchain from the
published images.
github-actions
Bot
added
documentation
musa_sdk:5.2.0-*-s5000 does not ship the cub and thrust headers that the MUSA backend builds against, and its runtime image does not register /usr/local/musa/lib with the dynamic loader. Install both header packages in the build stage and in the MUSA CI container, and write the loader path in the runtime stage.
The MUSA SDK images do not install libmthreads-compute, which provides libmusa.so.1 in /usr/lib/x86_64-linux-gnu, so linking anything against the MUSA backend fails.
The MUSA runtime image does not install libmthreads-compute, so the published images would have no libmusa.so.1 at run time.
yeahdongcn
marked this pull request as ready for review
September 29, 2026 07:04
yeahdongcn
requested review from
a team,
ggerganov and
ngxson
as code owners
September 29, 2026 07:04
Collaborator
Author
CISC
approved these changes
Sep 29, 2026
JohannesGaessler
approved these changes
Sep 29, 2026
edwardyoon
pushed a commit
to edwardyoon/focus-llama
that referenced
this pull request
Oct 7, 2026
ggml-org#29624) * musa: build the docker image and CI container from the MUSA SDK images Use registry.mthreads.com/mcconline/musa_sdk:5.2.0-{devel,runtime}-ubuntu22.04-s5000 instead of registry.mthreads.com/mcconline/inference/pytorch:2.9.1.post1-py3.10-musa5.2.0-mp31-devel-ubuntu22.04-amd64 for the MUSA docker image and the MUSA CI container, and let the runtime stage use the runtime image instead of reusing the devel one, which drops the MUSA toolchain from the published images. * musa: install the MUSA headers and loader path the SDK images omit musa_sdk:5.2.0-*-s5000 does not ship the cub and thrust headers that the MUSA backend builds against, and its runtime image does not register /usr/local/musa/lib with the dynamic loader. Install both header packages in the build stage and in the MUSA CI container, and write the loader path in the runtime stage. * musa: install libmthreads-compute for the MUSA runtime library The MUSA SDK images do not install libmthreads-compute, which provides libmusa.so.1 in /usr/lib/x86_64-linux-gnu, so linking anything against the MUSA backend fails. * musa: install libmthreads-compute in the runtime stages The MUSA runtime image does not install libmthreads-compute, so the published images would have no libmusa.so.1 at run time. --------- Co-authored-by: yeahdongcn <yeahdongcn@users.noreply.github.com> (cherry picked from commit 18b74ff)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
The MUSA SDK images are published now, so use them instead of the PyTorch image:
registry.mthreads.com/mcconline/musa_sdk:5.2.0-devel-ubuntu22.04-s5000— docker image build stage and MUSA CI containerregistry.mthreads.com/mcconline/musa_sdk:5.2.0-runtime-ubuntu22.04-s5000— runtime stage#29481 based both stages on the PyTorch image because it was the only one available, so the runtime stage pulled in the devel image and the published images carry the MUSA toolchain that only the build stage needs.
The SDK images do not ship everything the PyTorch image provides, so the MUSA backend needs a few packages on top:
musa-mualg-5-2andmusa-muthrust-5-2— the CUB and Thrust headers thatggml-cudasources includelibmthreads-compute— provideslibmusa.so.1in/usr/lib/x86_64-linux-gnu, required for linking and at run time/etc/ld.so.conf.d/musa-runtime.conf— the runtime image does not register/usr/local/musa/lib, wherelibmusart.so.5livesTesting Done
musajob, which builds inside the new devel container: pass, 16m30s — https://github.com/ggml-org/llama.cpp/actions/runs/36516634750/job/109240484839 (cudaandhippass as well)Publish Docker imagefor MUSA only, on a fork test branch, from this branch's Dockerfile: all three images built and pushed — https://github.com/yeahdongcn/llama.cpp/actions/runs/36526644817full, revisionc6e3901f0, 12.3 GiB) on an MTT S5000, driver 5.2.0-server, on one leased card, with Qwen3-0.6B-Q4_K_M:ldconfig -presolveslibmusart.so.5,libmublas.so.1andlibmusa.so.1, andldd /app/libggml-musa.soreports no missing libraryllama-bench -ngl 999 -dev MUSA0 -sm none: pp512 8064.34 ± 139.44 t/s, tg128 143.11 ± 0.09 t/sllama-server:/healthok and/v1/chat/completionsanswersParis.layer 0 is assigned to device MUSA0 but Flash Attention is assigned to device CPU, so these are functional numbers on a single non-exclusive card, not a performance measurementAdditional information
docs/docker.mdneeds no change: it only refers to the Dockerfile, not to a base image.Requirements