[jintakhan]

Reopening #768 after upstream's force push.
Dockerfile for building ROCm-based Docker images.
ROCm version pinned to 7.2.4 for compatibility; 7.14.1 and 10.0.0 also verified.

[luckylinux]

Did you try to build the Image ?

When I tried using my own Dockerfile (#1203) using the setup Functions instead of the direct cmake Commands it would just refuse to build llama.cpp IIRC. cmake doesn't build in ROCM Support. I did my test only on the gfx1030.

[jintakhan]

@Niko1221 PR has been updated addressing your requested changes. Header updated; docker-entrypoint.sh to add backend arg.
I've also attached the docker run log below. Tested running on gfx1201. Web UI accessible, engine functional.

Docker run logs

docker run --rm --device=/dev/kfd --device=/dev/dri --group-add video -p 8080:8080 --ulimit memlock=-1 -v strata-data:/data -e MODEL=IQ2_XS strata Setting up iq2_xs: downloading the model (~70 GB; the engine is already in the image). Strata - Qwen3.8-Flash-Next on a normal PC (a GPU + system RAM + CPU)

=== Step 1: checking your PC ===
Your AMD GPUs:
GPU 0: AMD Radeon RX 7900 series (gfx1100), 20 GB VRAM - can be used
GPU 1: AMD Radeon RX 9070 series / AI PRO R9700 (gfx1201), 32 GB VRAM - can be used
[ok] GPU: AMD Radeon RX 9070 series / AI PRO R9700 (gfx1201), 31.9 GB VRAM, gfx1201 (AMD: docs/AMD_HIP.md)
[ok] RAM: 94 GB
[ok] CPU: AMD Ryzen 9 5950X 16-Core Processor (AVX2)

=== Step 2: your choices ===
[ok] model: Qwen3.8-Flash-Next

1. Q2_0 2-bit, the fastest; download 66 GB, uses ~34 GB of RAM
2. IQ2_XS 2-bit i-quant, a little better quality, close in speed; download 68 GB, uses ~36 GB of RAM
3. IQ3_XXS 3-bit i-quant, better quality, slower (more CPU work per token); download 76 GB, uses ~43 GB of RAM
4. IQ3_S 3.5-bit i-quant, the best quality (matches the full model), the slowest; needs a 64 GB PC with little else running; download 84 GB, uses ~50 GB of RAM
   [ok] size: IQ2_XS
   [ok] context: 32768 tokens
   [ok] KV cache: 8-bit
   [ok] images: off

EXPERIMENTAL - speed projection: a small control vector applied while the model runs (layers 4-44).
It changes how the model answers: its package describes it as a refusal-direction projection (the
model declines far fewer requests). Off unless you choose it; when on, the web app can switch it off
per chat. Details: data/experimental-speed-projection/README.md
[ok] experimental speed projection: off

=== Step 3: Python packages ===
Installing numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil ...

/opt/strata/.venv/bin/python -m pip install --quiet --disable-pip-version-check numpy==2.5.3; python_version >= "3.12" numpy==2.4.6; python_version == "3.11" numpy==2.2.6; python_version < "3.11" jinja2==3.1.6 regex==2026.9.10 pyyaml==6.0.3 tqdm==4.70.1 requests==2.34.2 cmake==4.4.3 ninja==1.13.2 pillow==12.3.0 psutil==7.2.2 markupsafe==3.0.3 certifi==2026.7.22 charset-normalizer==3.5.1 idna==3.20 urllib3==2.8.0 colorama==0.4.6; sys_platform == "win32"
[ok] numpy, jinja2, regex, pyyaml, tqdm, requests, cmake, ninja, pillow, psutil installed

=== Step 4: the Strata engine ===
[ok] llama.cpp 3cf0325 (gguf-py, ggml, mtmd)
[ok] engine already built for this PC
[ok] engine: /opt/strata/engine/strata

=== Step 5: downloading Qwen3.8-Flash-Next IQ2_XS ===
The model files go in /data/models/IQ2_XS
Files you already have: put them here with their original names (Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf, Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf), or use --gguf-dir .
Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf: 39.18 / 39.23 GB (100%)
[ok] Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf downloaded
Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf: 28.66 / 28.80 GB (99%)
[ok] Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf downloaded
[ok] model files present

=== Step 6: preparing the model for Strata ===

/opt/strata/.venv/bin/python /opt/strata/tools/iq_pack.py --gguf /data/models/IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf --out /data/packs/iq2_xs
model shards: Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf, Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf
index.txt: 1079 tensors, 302 served natively, 0 in extra.bin, arena 1.43 GiB
tokenizer/: vocab 248320, merges 247587, pre qwen35, specials {'tokenizer.ggml.eos_token_id': 248046, 'tokenizer.ggml.padding_token_id': 248044, 'tokenizer.ggml.bos_token_id': 248044}
[ok] model prepared: /data/packs/iq2_xs
The MTP draft layer (speculative decoding, ~2x faster output) comes from the original Qwen checkpoint:
only its ~5 GB of MTP tensors are downloaded.
/opt/strata/.venv/bin/python /opt/strata/tools/mtp_fetch.py fetch --out /data/mtp

MTP block in the BF16 checkpoint

31 tensors, 5.214 GB, in 28 shards.

tensor	dtype	shape	MB
mtp.fc_embedding.weight	BF16	2560x2560	13.1
mtp.fc_hidden.weight	BF16	2560x2560	13.1
mtp.hyper_connection_mixer.hc_norm.weight	BF16	10240	0.0
mtp.hyper_connection_mixer.input_mix_weight_down.weight	BF16	320x10240	6.6
mtp.hyper_connection_mixer.input_mix_weight_up.weight	BF16	10240x320	6.6
mtp.layers.0.attn_hyper_connection.block_inject_weight.weight	BF16	4x10240	0.1
mtp.layers.0.attn_hyper_connection.hc_norm.weight	BF16	10240	0.0
mtp.layers.0.attn_hyper_connection.input_mix_weight_down.weight	BF16	320x10240	6.6
mtp.layers.0.attn_hyper_connection.input_mix_weight_up.weight	BF16	10240x320	6.6
mtp.layers.0.mlp.experts.down_proj	BF16	512x2560x640	1677.7
mtp.layers.0.mlp.experts.gate_up_proj	BF16	512x1280x2560	3355.4
mtp.layers.0.mlp.gate.weight	BF16	512x2560	2.6
mtp.layers.0.mlp.shared_expert.down_proj.weight	BF16	2560x640	3.3
mtp.layers.0.mlp.shared_expert.gate_proj.weight	BF16	640x2560	3.3
mtp.layers.0.mlp.shared_expert.up_proj.weight	BF16	640x2560	3.3
mtp.layers.0.mlp.shared_expert_gate.weight	BF16	1x2560	0.0
mtp.layers.0.mlp_hyper_connection.block_inject_weight.weight	BF16	4x10240	0.1
mtp.layers.0.mlp_hyper_connection.hc_norm.weight	BF16	10240	0.0
mtp.layers.0.mlp_hyper_connection.input_mix_weight_down.weight	BF16	320x10240	6.6
mtp.layers.0.mlp_hyper_connection.input_mix_weight_up.weight	BF16	10240x320	6.6
mtp.layers.0.self_attn.indexer.index_qk_proj.weight	BF16	640x2560	3.3
mtp.layers.0.self_attn.indexer.k_layernorm.weight	BF16	128	0.0
mtp.layers.0.self_attn.indexer.q_layernorm.weight	BF16	128	0.0
mtp.layers.0.self_attn.k_norm.weight	BF16	256	0.0
mtp.layers.0.self_attn.k_proj.weight	BF16	512x2560	2.6
mtp.layers.0.self_attn.o_proj.weight	BF16	2560x6144	31.5
mtp.layers.0.self_attn.q_norm.weight	BF16	256	0.0
mtp.layers.0.self_attn.q_proj.weight	BF16	12288x2560	62.9
mtp.layers.0.self_attn.v_proj.weight	BF16	512x2560	2.6
mtp.pre_fc_norm_embedding.weight	BF16	2560	0.0
mtp.pre_fc_norm_hidden.weight	BF16	10240	0.0

mtp.fc_embedding.weight 100%
mtp.fc_hidden.weight 100%
mtp.hyper_connection_mixer.hc_norm.weight 100%
mtp.hyper_connection_mixer.input_mix_weight_down.weight 100%
mtp.hyper_connection_mixer.input_mix_weight_up.weight 100%
mtp.layers.0.attn_hyper_connection.block_inject_weight.weight 100%
mtp.layers.0.attn_hyper_connection.hc_norm.weight 100%
mtp.layers.0.attn_hyper_connection.input_mix_weight_down.weight 100%
mtp.layers.0.attn_hyper_connection.input_mix_weight_up.weight 100%
mtp.layers.0.mlp.experts.down_proj 4%
mtp.layers.0.mlp.experts.down_proj 8%
mtp.layers.0.mlp.experts.down_proj 12%
mtp.layers.0.mlp.experts.down_proj 16%
mtp.layers.0.mlp.experts.down_proj 20%
mtp.layers.0.mlp.experts.down_proj 24%
mtp.layers.0.mlp.experts.down_proj 28%
mtp.layers.0.mlp.experts.down_proj 32%
mtp.layers.0.mlp.experts.down_proj 36%
mtp.layers.0.mlp.experts.down_proj 40%
mtp.layers.0.mlp.experts.down_proj 44%
mtp.layers.0.mlp.experts.down_proj 48%
mtp.layers.0.mlp.experts.down_proj 52%
mtp.layers.0.mlp.experts.down_proj 56%
mtp.layers.0.mlp.experts.down_proj 60%
mtp.layers.0.mlp.experts.down_proj 64%
mtp.layers.0.mlp.experts.down_proj 68%
mtp.layers.0.mlp.experts.down_proj 72%
mtp.layers.0.mlp.experts.down_proj 76%
mtp.layers.0.mlp.experts.down_proj 80%
mtp.layers.0.mlp.experts.down_proj 84%
mtp.layers.0.mlp.experts.down_proj 88%
mtp.layers.0.mlp.experts.down_proj 92%
mtp.layers.0.mlp.experts.down_proj 96%
mtp.layers.0.mlp.experts.down_proj 100%
mtp.layers.0.mlp.experts.gate_up_proj 2%
mtp.layers.0.mlp.experts.gate_up_proj 4%
mtp.layers.0.mlp.experts.gate_up_proj 6%
mtp.layers.0.mlp.experts.gate_up_proj 8%
mtp.layers.0.mlp.experts.gate_up_proj 10%
mtp.layers.0.mlp.experts.gate_up_proj 12%
mtp.layers.0.mlp.experts.gate_up_proj 14%
mtp.layers.0.mlp.experts.gate_up_proj 16%
mtp.layers.0.mlp.experts.gate_up_proj 18%
mtp.layers.0.mlp.experts.gate_up_proj 20%
mtp.layers.0.mlp.experts.gate_up_proj 22%
mtp.layers.0.mlp.experts.gate_up_proj 24%
mtp.layers.0.mlp.experts.gate_up_proj 26%
mtp.layers.0.mlp.experts.gate_up_proj 28%
mtp.layers.0.mlp.experts.gate_up_proj 30%
mtp.layers.0.mlp.experts.gate_up_proj 32%
mtp.layers.0.mlp.experts.gate_up_proj 34%
mtp.layers.0.mlp.experts.gate_up_proj 36%
mtp.layers.0.mlp.experts.gate_up_proj 38%
mtp.layers.0.mlp.experts.gate_up_proj 40%
mtp.layers.0.mlp.experts.gate_up_proj 42%
mtp.layers.0.mlp.experts.gate_up_proj 44%
mtp.layers.0.mlp.experts.gate_up_proj 46%
mtp.layers.0.mlp.experts.gate_up_proj 48%
mtp.layers.0.mlp.experts.gate_up_proj 50%
mtp.layers.0.mlp.experts.gate_up_proj 52%
mtp.layers.0.mlp.experts.gate_up_proj 54%
mtp.layers.0.mlp.experts.gate_up_proj 56%
mtp.layers.0.mlp.experts.gate_up_proj 58%
mtp.layers.0.mlp.experts.gate_up_proj 60%
mtp.layers.0.mlp.experts.gate_up_proj 62%
mtp.layers.0.mlp.experts.gate_up_proj 64%
mtp.layers.0.mlp.experts.gate_up_proj 66%
mtp.layers.0.mlp.experts.gate_up_proj 68%
mtp.layers.0.mlp.experts.gate_up_proj 70%
mtp.layers.0.mlp.experts.gate_up_proj 72%
mtp.layers.0.mlp.experts.gate_up_proj 74%
mtp.layers.0.mlp.experts.gate_up_proj 76%
mtp.layers.0.mlp.experts.gate_up_proj 78%
mtp.layers.0.mlp.experts.gate_up_proj 80%
mtp.layers.0.mlp.experts.gate_up_proj 82%
mtp.layers.0.mlp.experts.gate_up_proj 84%
mtp.layers.0.mlp.experts.gate_up_proj 86%
mtp.layers.0.mlp.experts.gate_up_proj 88%
mtp.layers.0.mlp.experts.gate_up_proj 90%
mtp.layers.0.mlp.experts.gate_up_proj 92%
mtp.layers.0.mlp.experts.gate_up_proj 94%
mtp.layers.0.mlp.experts.gate_up_proj 96%
mtp.layers.0.mlp.experts.gate_up_proj 98%
mtp.layers.0.mlp.experts.gate_up_proj 100%
mtp.layers.0.mlp.gate.weight 100%
mtp.layers.0.mlp.shared_expert.down_proj.weight 100%
mtp.layers.0.mlp.shared_expert.gate_proj.weight 100%
mtp.layers.0.mlp.shared_expert.up_proj.weight 100%
mtp.layers.0.mlp.shared_expert_gate.weight 100%
mtp.layers.0.mlp_hyper_connection.block_inject_weight.weight 100%
mtp.layers.0.mlp_hyper_connection.hc_norm.weight 100%
mtp.layers.0.mlp_hyper_connection.input_mix_weight_down.weight 100%
mtp.layers.0.mlp_hyper_connection.input_mix_weight_up.weight 100%
mtp.layers.0.self_attn.indexer.index_qk_proj.weight 100%
mtp.layers.0.self_attn.indexer.k_layernorm.weight 100%
mtp.layers.0.self_attn.indexer.q_layernorm.weight 100%
mtp.layers.0.self_attn.k_norm.weight 100%
mtp.layers.0.self_attn.k_proj.weight 100%
mtp.layers.0.self_attn.o_proj.weight 100%
mtp.layers.0.self_attn.q_norm.weight 100%
mtp.layers.0.self_attn.q_proj.weight 100%
mtp.layers.0.self_attn.v_proj.weight 100%
mtp.pre_fc_norm_embedding.weight 100%
mtp.pre_fc_norm_hidden.weight 100%

/opt/strata/.venv/bin/python /opt/strata/tools/mtp_pack.py --src /data/mtp --experts q2_0 --out /data/mtp/mtp-q2_0.gguf
mtp.fc_embedding.weight: BF16 13.1 MB
mtp.fc_hidden.weight: BF16 13.1 MB
mtp.hyper_connection_mixer.hc_norm.weight: F32 0.0 MB
mtp.hyper_connection_mixer.input_mix_weight_down.weight: BF16 6.6 MB
mtp.hyper_connection_mixer.input_mix_weight_up.weight: BF16 6.6 MB
mtp.layers.0.attn_hyper_connection.block_inject_weight.weight: BF16 0.1 MB
mtp.layers.0.attn_hyper_connection.hc_norm.weight: F32 0.0 MB
mtp.layers.0.attn_hyper_connection.input_mix_weight_down.weight: BF16 6.6 MB
mtp.layers.0.attn_hyper_connection.input_mix_weight_up.weight: BF16 6.6 MB
mtp.layers.0.mlp.experts.down_proj: q2_0 235.9 MB
mtp.layers.0.mlp.experts.gate_up_proj: q2_0 471.9 MB
mtp.layers.0.mlp.gate.weight: BF16 2.6 MB
mtp.layers.0.mlp.shared_expert.down_proj.weight: BF16 3.3 MB
mtp.layers.0.mlp.shared_expert.gate_proj.weight: BF16 3.3 MB
mtp.layers.0.mlp.shared_expert.up_proj.weight: BF16 3.3 MB
mtp.layers.0.mlp.shared_expert_gate.weight: BF16 0.0 MB
mtp.layers.0.mlp_hyper_connection.block_inject_weight.weight: BF16 0.1 MB
mtp.layers.0.mlp_hyper_connection.hc_norm.weight: F32 0.0 MB
mtp.layers.0.mlp_hyper_connection.input_mix_weight_down.weight: BF16 6.6 MB
mtp.layers.0.mlp_hyper_connection.input_mix_weight_up.weight: BF16 6.6 MB
mtp.layers.0.self_attn.indexer.index_qk_proj.weight: BF16 3.3 MB
mtp.layers.0.self_attn.indexer.k_layernorm.weight: F32 0.0 MB
mtp.layers.0.self_attn.indexer.q_layernorm.weight: F32 0.0 MB
mtp.layers.0.self_attn.k_norm.weight: F32 0.0 MB
mtp.layers.0.self_attn.k_proj.weight: BF16 2.6 MB
mtp.layers.0.self_attn.o_proj.weight: BF16 31.5 MB
mtp.layers.0.self_attn.q_norm.weight: F32 0.0 MB
mtp.layers.0.self_attn.q_proj.weight: BF16 62.9 MB
mtp.layers.0.self_attn.v_proj.weight: BF16 2.6 MB
mtp.pre_fc_norm_embedding.weight: F32 0.0 MB
mtp.pre_fc_norm_hidden.weight: F32 0.0 MB
wrote /data/mtp/mtp-q2_0.gguf: 0.889 GB of tensor data
/opt/strata/.venv/bin/python /opt/strata/tools/mtp_rt.py --gguf /data/mtp/mtp-q2_0.gguf --out /data/mtp/rt
experts.bin 707788800 B, dense.bin 116099072 B, 29 tensors -> /data/mtp/rt
fc_embedding.weight q8_0 2560 2560 0 6963200
fc_hidden.weight q8_0 2560 2560 6963200 6963200
hyper_connection_mixer.hc_norm.weight f32 1 10240 13926400 40960
hyper_connection_mixer.input_mix_weight_down.weight bf16 320 10240 13967360 6553600
hyper_connection_mixer.input_mix_weight_up.weight bf16 10240 320 20520960 6553600
attn_hyper_connection.block_inject_weight.weight bf16 4 10240 27074560 81920
attn_hyper_connection.hc_norm.weight f32 1 10240 27156480 40960
attn_hyper_connection.input_mix_weight_down.weight bf16 320 10240 27197440 6553600
attn_hyper_connection.input_mix_weight_up.weight bf16 10240 320 33751040 6553600
mlp.gate.weight bf16 512 2560 40304640 2621440
mlp.shared_expert.down_proj.weight q8_0 2560 640 42926080 1740800
mlp.shared_expert.gate_proj.weight q8_0 640 2560 44666880 1740800
mlp.shared_expert.up_proj.weight q8_0 640 2560 46407680 1740800
mlp.shared_expert_gate.weight bf16 1 2560 48148480 5120
mlp_hyper_connection.block_inject_weight.weight bf16 4 10240 48153600 81920
mlp_hyper_connection.hc_norm.weight f32 1 10240 48235520 40960
mlp_hyper_connection.input_mix_weight_down.weight bf16 320 10240 48276480 6553600
mlp_hyper_connection.input_mix_weight_up.weight bf16 10240 320 54830080 6553600
self_attn.indexer.index_qk_proj.weight q8_0 640 2560 61383680 1740800
self_attn.indexer.k_layernorm.weight f32 1 128 63124480 512
self_attn.indexer.q_layernorm.weight f32 1 128 63124992 512
self_attn.k_norm.weight f32 1 256 63125504 1024
self_attn.k_proj.weight q8_0 512 2560 63126528 1392640
self_attn.o_proj.weight q8_0 2560 6144 64519168 16711680
self_attn.q_norm.weight f32 1 256 81230848 1024
self_attn.q_proj.weight q8_0 12288 2560 81231872 33423360
self_attn.v_proj.weight q8_0 512 2560 114655232 1392640
pre_fc_norm_embedding.weight f32 1 2560 116047872 10240
pre_fc_norm_hidden.weight f32 1 10240 116058112 40960
[ok] MTP draft layer: /data/mtp/rt

=== Step 7: writing the start script ===
[ok] hipBLASLt tuning table: gfx1201-hipblaslt-100202.txt (faster prompts)
tip: for several agents or clients at once, --conversation-cache-mib 8192 in the config's args keeps each one's conversation; without it they were measured re-reading ~90% of their prompts (#882 #440; docs/DETAILS.md, Multiple conversations)
[ok] start script: run-iq2_xs.sh

All set.
API (OpenAI): http://127.0.0.1:8080/v1 (any API key; model name: anything)
API (Anthropic): http://127.0.0.1:8080/v1/messages
Other devices: the server window prints this PC's address (http://:8080/) - no API key set: anyone on your network can use it
Next time: just run ./setup.sh (or run-iq2_xs.sh) - it starts right away
Several requests at once (opt-in): --parallel 4 decodes up to 4 together instead of one after the other (each takes ~0.6 GB of VRAM from the expert cache; docs/BATCHING.md).
Strata - Qwen3.8-Flash-Next on a normal PC (a GPU + system RAM + CPU)
[ok] GPU: AMD Radeon RX 9070 series / AI PRO R9700 (gfx1201) (32 GB, AMD)

---

Starting qwen3.8-flash-next-iq2_xs: it loads about 39 GB into RAM and locks part of it for the GPU.
While it does, YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES (longer the first time after a
restart). That is normal: please wait and don't close this window - the browser opens when it is ready.
Later, closing this window stops the model.

Settings (strata-iq2_xs.json): --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5
--max-context 32768 --kv int8; server 0.0.0.0:8080, gpu 1
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] loading the experts into RAM (about 39 GB) and locking part of them for the GPU.
YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES NOW - this is normal.
Please wait and don't close this window; the browser opens when it is ready.
[strata] still starting (23 s) - please wait ...
[strata] experts loaded: 33.02 GiB at 3.81 GiB/s (41 s so far)
[strata] filling the GPU's expert cache (19470 experts, 26.16 GiB of VRAM) ...
[strata] almost ready ...
ready: http://127.0.0.1:8080/v1 (OpenAI: /v1/chat/completions, Anthropic: /v1/messages, context 32768 tokens)
open http://127.0.0.1:8080/ in a browser to chat; close this window to stop the model
from other devices: http://172.17.0.3:8080/ (API: http://172.17.0.3:8080/v1)
WARNING: no API key - anyone on your network can use this model. Add "api_key": "..." to the config (clients send it as their API key; the web page asks for it)
[strata] thinking: 30 of max 32707 tokens, 59.9 tok/s, 1 s
[strata] done: 59 tokens in 1 s (65.2 tok/s) (stop, cancel=False), expert cache 98.0% hit

[jintakhan]

@luckylinux I don't have a gfx1030 GPU handy for testing at the moment but as far as the Docker image goes, I was able to build targeting gfx1030 without any hiccups.

[luckylinux]

@jintakhan: Uhm OK

I fixed my Dockerfile so it finally builds, though it finishes with

[273/275] Building CXX object llama/tools/mtmd/CMakeFiles/mtmd.dir/models/parakeet.cpp.o
[274/275] Linking CXX static library llama/tools/mtmd/libmtmd.a
[275/275] Linking CXX executable bin/strata-vision
Traceback (most recent call last):
  File "<stdin>", line 88, in <module>
  File "/usr/lib/python3.13/pathlib/_local.py", line 505, in __init__
    super().__init__(*args)
    ~~~~~~~~~~~~~~~~^^^^^^^
  File "/usr/lib/python3.13/pathlib/_local.py", line 132, in __init__
    raise TypeError(
    ...<2 lines>...
        f"not {type(path).__name__!r}")
TypeError: argument should be a str or an os.PathLike object where __fspath__ returns a str, not 'NoneType'
Error: building at STEP "RUN .venv/bin/python - <<'PYEOF'": while running runtime: exit status 1

So I guess one of the copy Instructions after the build fails 😕.

[jintakhan]

@luckylinux You have more than double the compile steps compared to this Dockerfile build. If you are building against your own Dockerfile I cannot help because I don't know how you structured yours. Please try the build instructions in this PR's Dockerfile instead. Thanks

[jintakhan]

Also good news: ROCm 10.1.0 build verified. Keeps the same rocBLASLt version as 10.0.0, so the base image is a drop-in replacement.

[luckylinux]

Well I did what I could based on the Informations that were available Yesterday. Then I opened my Issue (this PR or Issue was not present yet).

gfx1030 wasn't documented in setup.py and the Old GPUs Page directly referenced running cmake Commands.

Double the lines is a bit misleading:

• I had to try several Things with back and Forths, I didn't want to have 100 backup Files all over the Place
• You are doing a single Stage build, I am doing a multi Stage build (same as I do with llama.cpp)

I don't get why a multi stage Build (build Image -> transfer compiled Binaries into the target Image) is not the default.

[jintakhan]

@luckylinux This PR's scope is strictly the commits I am trying to get merged. If you want feedback/help on your multi-stage build Dockerfile, please open a competing PR, because this is not the right place for it.

Regarding structure, I deliberately kept the Dockerfile as analogous as possible to the preexisting CUDA one to respect the original design and standardize it for easier maintenance across different backends because I reckon we'll eventually end up with one Dockerfile for each backend.

[luckylinux]

Fair enough.

As I said, I already opened an Issue about it.

I won't be opening a PR until and unless we figure out a Way to make it work.

[dambaev]

Hi, guys, check this one #1486 you may be interested in (tested on gfx1201 and gfx1031)

[luckylinux]

Oh no, yet another PR that seems to do the same and doesn't really reply to the Question 🙄.

I think it's more important to understand the Method that is used, since that is clearly needed (stock ROCM does NOT work) for the AMD Radeon Instinct Mi50 which I also own (which I only managed to get working with llama.cpp using https://github.com/mixa3607/ML-gfx906/).

[dambaev]

Oh no, yet another PR that seems to do the same and doesn't really reply to the Question 🙄.

I think it's more important to understand the Method that is used, since that is clearly needed (stock ROCM does NOT work) for the AMD Radeon Instinct Mi50 which I also own (which I only managed to get working with llama.cpp using https://github.com/mixa3607/ML-gfx906/).

okay, let's assume, that there is yet another the same PR....

Other than that:

1. I am not sure what the Question is, but the PR says Add Dockerfile for building ROCm-based Strata Docker images;
2. accordingly to https://github.com/Niko1221/Strata/pull/1307/changes#diff-21cfa228c69f2aa456d76adb243bd3a9aa8e35d05cca8a18c744b354cd73c817R69 your default arguments for build will fail, because it will require to build for 2 different families of drivers;
3. accordingly to Add Dockerfile for building ROCm-based Strata Docker images #768 (comment) , the previous PR had been closed with the requirement to use setup.py's public interface. (I read it as 'call setup.py with command line-arguments'). My PR violates that requirement as well. At the same time I see no options to run setup.py only to build engine without hardware passthrough (as docker build is not accepting --device= and co options). And I see such change cuda-targeted Dockerfile.

[luckylinux]

1. I'm not sure what the point of doing 3+ Times (almost) the same Thing is, but besides that small Detail ...
2. Not sure, as I said, I was trying to build for gfx1030 then I opened the Issue because it clearly wasn't working. Yesterday I managed to finish both the llama.cpp and strata Build, but I still need to see how to copy the Artifacts from the Build to the Runtime Stage (in a similar Way as I did for llama.cpp)
3. Besides the fact that there were 631 Commits. If you look at my Issue, in the last Comment, the Containerfile builds successfully (both the llama.cpp and strata Build) so if you don't want a Multi-Stage Build, then it should work. There is some NoneType Error when it tries to save the Metadata JSON File, but apart from that and the 2nd Stage File Copy it works. I have the GPU on that System when I do the build using podman build, but I didn't pass the GPU through so not quite sure what that Problem / Requirement is. I am not passing the GPU to the podman build Command either and it builds successfully !

[jintakhan]

@dambaev Our approaches are similar. I do think it's better to use the ROCm base image instead of rely on setup.py to install the libraries, since that gives the user the choice of which base image to use, otherwise it will be pinned to whatever version setup.py downloads.
I also build tune_hipblaslt to make sure the tooling for supporting additional architectures/rocBLASLt versions is present in the image, in case there is no precompiled tuning table for a user's card/version combo.

[dambaev]

I do think it's better to use the ROCm base image instead of rely on setup.py to install the libraries,

is there any particular reason why you choosed 7.2 instead of, for example 7.14.1?

as far as I understand, setup.py is choosing rocm version based on architecture, which may be more flexible.

It seems, that there is a use case for both options: one can prefer to build per-arch, other can prefer to base of rocm/ (but I there are more recent versions available to stick to, imho).

[jintakhan]

is there any particular reason why you choosed 7.2 instead of, for example 7.14.1?

At the time I started working on this, Strata had more tuning tables for 7.2.4 than any other ROCm version so that’s what I pinned it to. As far as the build process goes, base images are drop-in replacements up to 10.1.0, you just need to make your own tuning table if the repo lacks a prebuilt one.

[luckylinux]

Just add an ARG at the Top of the File for like BASE_IMAGE with a sane default Value and then use FROM ${BASE_IMAGE} as whatever instead of the hardcoded one.

The one can also use https://github.com/mixa3607/ML-gfx906 Image for AMD Radeon Mi50 most likely as a sane base (AMD Radeon Instinct Mi50 doesn't work with stock ROCM).

[jintakhan]

Just add an ARG at the Top of the File for like BASE_IMAGE with a sane default Value and then use FROM ${BASE_IMAGE} as whatever instead of the hardcoded one.

Implemented, easy add, thanks. I want to keep this PR focused on ROCm/HIP, but we should make this change in the CUDA dockerfile too.

[luckylinux]

Just add an ARG at the Top of the File for like BASE_IMAGE with a sane default Value and then use FROM ${BASE_IMAGE} as whatever instead of the hardcoded one.

Implemented, easy add, thanks. I want to keep this PR focused on ROCm/HIP, but we should make this change in the CUDA dockerfile too.

I appreciate that, thanks 👍.

Sorry if I sounded rude, I just had been battling with this almost the entire Day yesterday and seeing like 3 (or potentially more soon ?) people reinventing the Wheel (including me, you can argue of course) within 24h is just ... kind of a Waste of Time 😕.

If you (as you just did with BASE_IMAGE, can of course be a different Argument name) can make it in a Way that everybody can get what they need in a single Solution (so it could compile against https://github.com/mixa3607/ML-gfx906/ for instance for the AMD Radeon Instinct Mi50 or People can choose between Ubuntu 24.04, Debian Trixie, Official ROCM Images or whatever as a Base Image), I'm all in favor of merging Efforts 😉.

I don't have a big Stake in the Game anyway. I didn't read or contribute to all the low-level Features of the Code Base, I just wanted to try to get it working on my System as a Docker Image with AMD Radeon Pro V620 and (on another System) AMD Radeon Instinct Mi50. I think the Documentation needs to be improved as well (speaking from a User perspective).

As a Sidenote and to better troubleshoot IMHO (because that is the issue I was facing with writing the Metadata JSON File which was failing for me AFTER the entire Compilation completed successfully), IMHO the Python Script should be fed as a separate File and do a COPY of it inside the build Stage. That also applies to upstream / CUDA Variant of course. I wasn't getting a very clear stack Trace or Debug Messages at all with this inline Python Code (even with PYTHONVERBOSE=1 set with RUN export PYTHONVERBOSE=1) ...

But OK, I realize that last Point might be going a bit off a Tangent 😆.

[jintakhan]

No worries. The Dockerfile already has the build arg HIP_ARCHITECTURES for targeting different architectures. As for MI50/gfx906, official TheRock documentation says it still has official ROCm support and should work with stock ROCm. There's no prebuilt image with gfx906 support, but AMD still builds nightly packages for it in their ROCm nightly repo.

As there is no prebuilt image, you need to make your own. This is pretty easy. Just pull the base Ubuntu image, install the ROCm toolkit inside it (make sure to use the unsigned nightly repo instead of the signed stable repo) and build it as an image. Then that image can be used as the base image for this Dockerfile via BASE_IMAGE. This is the method I use when targeting gfx90c.

[luckylinux]

I don't think that's the Case for the Mi50. For llama.cpp, after wasting 1 entire Day on it trying to build my own Image, I just gave up and use https://github.com/mixa3607/ML-gfx906/ which worked at the first try. I was using Debian Trixie as a base though, not Ubuntu. Maybe that was the Issue or I was missing some Build/Runtime Library, I cannot remember the exact Issue now 😕. Or as you say I would have needed to use the nightly Packages instead of the stable ones.

[jintakhan]

Hope #1594 gets addressed in time for this PR.

[jintakhan]

Hi @Niko1221 sorry to ping again, just want to make sure this stays on your radar. Thanks!

[BBQigniter]

thank you for those files - tested yesterday on my homelab with an AMD 7800 XT (gfx1101) and it worked flawlessly.

[jintakhan]

This PR is still current as of v0.1.44.1.

[luckylinux]

Yeah, my Issue just got closed saying that 0.1.44 introduced many changes (except those that I wanted and needed and it didn't answer the Question 🙄).

[qiuzan2001]

The docker-entrypoint.sh change does more than add BACKEND: compared with main (git diff upstream/main...pr/1307 -- docker-entrypoint.sh) it deletes the three lines that pass --gguf-dir "$GGUF_DIR", --resident-budget-gib "$RESIDENT_BUDGET_GIB" and --kv-streaming "$KV_STREAMING" to setup.py --setup, while the variables are still read. So those settings would be ignored on first setup for the existing CUDA image too. It looks like a merge slip; keeping those lines and adding only the --backend one would fix it. tools/test_docker_entrypoint.py passes either way (9 tests), so a test for the forwarding would catch it. #1486 adds another ROCm Dockerfile for the same purpose.

[jintakhan]

@qiuzan2001 thanks for the catch. It is indeed a repo sync slip; those flags were most likely added after the original PR was made several weeks ago. I will make the fix when I’m back at a computer.

[jintakhan]

Thanks again @qiuzan2001 docker_entrypoint.sh merge slip has been addressed.

[julianxhokaxhiu]

@Niko1221 is there any chance you can have a look at this one? Having a common image that everyone on the AMD world can use right away without the need to compile Strata everytime under Linux would be a game changer.

[Niko1221]

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.