Fix asym_w4a8_int8 loading on MPS

[scriptease]

Problem

No asym_w4a8_int8 model loads on Apple Silicon. Loading one (e.g. Kijai/MiniMax-H3-experimental/minimax_h3_fl2va_pruned_w4a8_mixed.safetensors) fails as soon as a weight is dequantized:

File ".../comfy_kitchen/backends/eager/w4a8_int8.py", line 414, in _dequant_int4_grouped_to_int8
    values = values.view(n, groups, group_size) * s_rel.float().unsqueeze(-1)
RuntimeError: Undefined type Float8_e4m3fn

The cause is in torch, not here — the MPS backend implements no fp8 casts at all:

>>> t = torch.tensor([1.0, 2.0]).to(torch.float8_e4m3fn)
>>> t.float()                     # cpu
tensor([1., 2.])
>>> t.to('mps').float()           # mps
RuntimeError: Undefined type Float8_e4m3fn

Since weight_s_rel is stored as float8_e4m3fn, the eager W4A8 kernels cannot touch it once it is on an MPS device.

Change

Convert the group scale to fp32 at load time when the compute device is MPS. float32 is one of the two dtypes the eager backend declares for s_rel:

"s_rel": ParamConstraint(dtypes=frozenset({torch.float8_e4m3fn, torch.float32}), ...)

so the registry dispatches normally afterwards. This is a one-time conversion at load, not a per-forward cast.

Verification

On an M-series Mac (torch 2.14.0.dev, MPS), with the patch applied, minimax_h3_fl2va_pruned_w4a8_mixed.safetensors loads and samples end to end and produces correct video output.

Isolated check through the registry-dispatched entry points, fp32 vs fp8 scales carrying the same values:

• w4a8_int8_linear on MPS with fp32 s_rel matches the CPU fp8 reference exactly (maxdiff 0.0)
• dequantize_w4a8_int8_weight on MPS with fp32 s_rel succeeds; with fp8 it raises the error above

Why not fp16

fp16 represents every float8_e4m3fn value exactly, so it would halve the memory cost of this widening (the group scales are one value per 16 weights, so on a ~22B-param model fp32 costs roughly 4 GB more than the native fp8). I tried it and the numerics are bit-identical.

It does not work, though: the eager backend's ParamConstraint for s_rel admits only {float8_e4m3fn, float32}, so a fp16 scale fails dispatch with

NoCapableBackendError: No backend can handle 'dequantize_w4a8_int8_weight': eager: s_rel: dtype torch.float16 not in {torch.float8_e4m3fn, torch.float32}

Adding fp16 to that constraint in comfy-kitchen would be the cheaper fix if the extra memory matters; this PR stays within what the current backend accepts.

[github-actions]

✅ All contributors have signed the CLA. Thank you! This PR is ready to be merged.
_{Posted by the CLA Assistant Lite bot.}

[coderabbitai]

📝 Walkthrough

Walkthrough

The asym_w4a8_int8 quantization path converts weight_s_rel to float32 on MPS when its storage dtype is FP8-compatible. Other platforms keep the existing scale-loading behavior.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely identifies the MPS loading fix for asym_w4a8_int8 quantization.
Description check	✅ Passed	The description directly explains the MPS FP8 failure, the float32 load-time conversion, and the verification results.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@comfy/ops.py`:
- Around line 1206-1210: Update the MPS conditional in the surrounding module
loading logic to check the module’s target device via the existing device
variable (device.type), rather than
comfy.model_management.get_torch_device().type. Preserve the scale.float()
conversion for MPS and leave non-MPS behavior unchanged.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 563ad9c4-1ba9-496f-99a4-17a4449129c9

📥 Commits

Reviewing files that changed from the base of the PR and between bd34f33 and 2346dad.

📒 Files selected for processing (1)

• comfy/ops.py

📜 Review details

⚠️ CI failures not shown inline (2)

GitHub Actions: CLA Assistant / cla-assistant: Fix asym_w4a8_int8 loading on MPS

Conclusion: failure

View job details

##[group]Run contributor-assistant/github-action@ca4a40a7d1004f18d9960b404b97e5f30a505a08
 with:
   path-to-document: https://github.com/Comfy-Org/comfy-cla/blob/main/comfyui_icla.md
   remote-organization-name: comfy-org
   remote-repository-name: comfy-cla
   path-to-signatures: signatures/cla.json
   branch: main
   allowlist: action@github.com,actions-user,ampagent,claude,comfy-pr-bot,GitHub Action,github-actions,github-actions[bot],Glary Bot,Glary-Bot,*[bot]
   custom-notsigned-prcomment: 🎉 Thank you for your contribution, we really appreciate it! 🎉
Like many open source projects, we require contributors to sign our [Contributor License Agreement (CLA)](https://github.com/Comfy-Org/comfy-cla/blob/main/comfyui_icla.md). A CLA makes the ownership of contributions explicit, so contributors and the project share a clear understanding of how the code can be used. By signing, you:
- Confirm that you own your contribution.
- Keep the right to reuse your own code.
- Grant us a copyright license to include and share it within our projects.
CLAs are standard practice across major open source projects including those under the Apache Software Foundation and the Linux Foundation. Ours is based on the Apache Software Foundation's CLA. Most importantly, it would enable us to relicense the project under a more permissive license in the future, giving the project and its community greater flexibility.
✍ **To sign, please post a new comment on this PR with exactly the following text:** ✍
   custom-pr-sign-comment: I have read and agree to the Contributor License Agreement
   custom-allsigned-prcomment: ✅ All contributors have signed the CLA. Thank you! This PR is ready to be merged.
   use-dco-flag: false
   lock-pullrequest-aftermerge: true
   suggest-recheck: true
 env:
   GITHUB_***REDACTED_SECRET_ASSIGNMENT***
   PERSONAL_ACCESS_***REDACTED_SECRET_ASSIGNMENT***
 ##[endgroup]
 CLA Assistant GitHub Action bot has started the process
 (node:2145) [DEP0040] DeprecationWarn...

---

GitHub Actions: CLA Assistant / 0_cla-assistant.txt: Fix asym_w4a8_int8 loading on MPS

Conclusion: failure

View job details

##[group]Run contributor-assistant/github-action@ca4a40a7d1004f18d9960b404b97e5f30a505a08
 with:
   path-to-document: https://github.com/Comfy-Org/comfy-cla/blob/main/comfyui_icla.md
   remote-organization-name: comfy-org
   remote-repository-name: comfy-cla
   path-to-signatures: signatures/cla.json
   branch: main
   allowlist: action@github.com,actions-user,ampagent,claude,comfy-pr-bot,GitHub Action,github-actions,github-actions[bot],Glary Bot,Glary-Bot,*[bot]
   custom-notsigned-prcomment: 🎉 Thank you for your contribution, we really appreciate it! 🎉
Like many open source projects, we require contributors to sign our [Contributor License Agreement (CLA)](https://github.com/Comfy-Org/comfy-cla/blob/main/comfyui_icla.md). A CLA makes the ownership of contributions explicit, so contributors and the project share a clear understanding of how the code can be used. By signing, you:
- Confirm that you own your contribution.
- Keep the right to reuse your own code.
- Grant us a copyright license to include and share it within our projects.
CLAs are standard practice across major open source projects including those under the Apache Software Foundation and the Linux Foundation. Ours is based on the Apache Software Foundation's CLA. Most importantly, it would enable us to relicense the project under a more permissive license in the future, giving the project and its community greater flexibility.
✍ **To sign, please post a new comment on this PR with exactly the following text:** ✍
   custom-pr-sign-comment: I have read and agree to the Contributor License Agreement
   custom-allsigned-prcomment: ✅ All contributors have signed the CLA. Thank you! This PR is ready to be merged.
   use-dco-flag: false
   lock-pullrequest-aftermerge: true
   suggest-recheck: true
 env:
   GITHUB_***REDACTED_SECRET_ASSIGNMENT***
   PERSONAL_ACCESS_***REDACTED_SECRET_ASSIGNMENT***
 ##[endgroup]
 CLA Assistant GitHub Action bot has started the process
 (node:2145) [DEP0040] DeprecationWarn...

🧰 Additional context used

📓 Path-based instructions (6)

**/*

📄 CodeRabbit inference engine (AGENTS.md)

**/*: Keep changes small, direct, and limited to the narrowest necessary code path and smallest number of files.
Prefer practical fixes, minimal dependencies, and existing repository patterns; remove obsolete, dead, unreachable, or unused code.
Preserve existing APIs, node names, model-loading behavior, file layout, and workflow compatibility unless replacement is explicitly intended.
Core ComfyUI must not add outbound internet requests, telemetry, tracking, reporting, remote configuration, or background network activity. User-authorized model downloads are limited to the requested artifact and must exclude telemetry and unrelated metadata.

Files:

• comfy/ops.py

**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

**/*.py: Keep state and capability flags on the object that owns the behavior. Prefer explicit parent-owned attributes over probing child objects with getattr; use child checks only when the child owns the delegated behavior.
Preserve shared method signatures, argument order, return shapes, side effects, and error behavior unless every affected caller and interface is intentionally updated.
Do not add unused compatibility parameters, flags, attributes, constructor options, or model-specific options to shared helpers; keep one-off behavior at the integration boundary.
Normalize third-party return conventions at integration boundaries so core code receives the expected type and shape; avoid undocumented caller-side unwrapping.
Do not add torch.no_grad, torch.inference_mode, or inference-mode wrappers. Do not add model freeze/unfreeze toggles; only disable globally enabled inference mode when a training path requires gradients.
Remove inference-only training behavior such as dropout while preserving checkpoint and state-dict compatibility; use nn.Identity when deleting a module would alter keys or ordering.
Keep imports at module scope except established optional-backend probes or imports required to avoid cycles; avoid unnecessary try/except blocks and use specific exceptions with useful fallbacks.
Do not add workarounds for unsupported library versions, especially PyTorch exception-and-float-cast retries, unless a comment names the exact versions still requiring them.
Let unsupported model formats, invalid quantization metadata, and bad states fail with clear errors instead of silently degrading output.
Match local style, keep comments sparse and useful, and remove comments that merely restate obvious code.
Treat dtype, device placement, VRAM use, and offloading as correctness concerns across CPU, CUDA, ROCm, MPS, DirectML, XPU, NPU, and low-VRAM environments.
Prefer existing ComfyUI and Comfy Kitchen operations, quantization helpers, cast/offload helpe...

Files:

• comfy/ops.py

**/*.{py,json}

📄 CodeRabbit inference engine (AGENTS.md)

Treat legacy combo, io.Combo, and io.DynamicCombo values affecting filesystem access as untrusted; revalidate them at load/save boundaries with folder_paths, containment checks, or fixed allowlists.

Files:

• comfy/ops.py

**/*.{py,md,txt,json}

📄 CodeRabbit inference engine (AGENTS.md)

Keep warning and info messages short and actionable, remove noisy or misleading logging, and make documentation edits concise, factual, and tied to changed behavior.

Files:

• comfy/ops.py

**

⚙️ CodeRabbit configuration file

**: IMPORTANT: Only comment on issues directly introduced by this PR's code changes.
Treat AGENTS.md as mandatory repository policy, not optional style guidance.
Flag PR changes that violate AGENTS.md even when the code is otherwise functional.
In particular, enforce architecture boundaries, dtype/device/memory rules,
interface contracts, import style, no unnecessary try/except blocks, no inline
imports, no outbound internet paths in core ComfyUI, and narrow scoped fixes.
Prefer direct findings over suggestions when a rule is violated. Only ignore
AGENTS.md when it clearly conflicts with a newer explicit maintainer instruction
in the PR.
Do NOT flag pre-existing issues in code that was merely moved, re-indented,
de-indented, or reformatted without logic changes. If code appears in the diff
only due to whitespace or structural reformatting (e.g., removing a with: block),
treat it as unchanged. Contributors should not feel obligated to address
pre-existing issues outside the scope of their contribution.

Files:

• comfy/ops.py

comfy/**

⚙️ CodeRabbit configuration file

comfy/**: Core ML/diffusion engine. Focus on:

• Backward compatibility (breaking changes affect all custom nodes)
• Memory management and GPU resource handling
• Performance implications in hot paths
• Thread safety for concurrent execution

Files:

• comfy/ops.py

🧠 Learnings (3)

📚 Learning: 2026-02-21T14:01:41.482Z

Learnt from: pythongosssss
Repo: Comfy-Org/ComfyUI PR: 12555
File: comfy_extras/nodes_glsl.py:719-724
Timestamp: 2026-02-21T14:01:41.482Z
Learning: In PyOpenGL, bare Python scalars can be accepted for 1-element array parameters by NumberHandler. This means you can pass an int/float directly to OpenGL texture deletion (e.g., glDeleteTextures(tex)) without wrapping in a list. Verify function-specific expectations and ensure types match what the OpenGL call expects; use explicit lists only when the API requires an array.

Applied to files:

• comfy/ops.py

📚 Learning: 2026-05-13T12:31:45.069Z

Learnt from: rattus128
Repo: Comfy-Org/ComfyUI PR: 13802
File: comfy/pinned_memory.py:19-30
Timestamp: 2026-05-13T12:31:45.069Z
Learning: When reviewing code that uses comfy/pinned_memory.py’s `HostBuffer.extend(size=..., reallocate=...)`: by default (`reallocate` is not True / False), `extend(size=...)` is a *relative increment* that grows the buffer by `size` bytes—so slicing like `[offset:offset+size]` after `hostbuf.extend(size=size)` is correct and the argument should not be rewritten to `offset + size`. Only in the single-segment reallocation mode (`reallocate=True`, e.g., as used by `resize_pin_buffer()` in `comfy/model_management.py`) should `size` be treated as an *absolute target* and the call/arguments should be checked accordingly.

Applied to files:

• comfy/ops.py

📚 Learning: 2026-08-06T22:18:59.719Z

Learnt from: kijai
Repo: Comfy-Org/ComfyUI PR: 15362
File: comfy/ldm/wan/model_animate2.py:186-223
Timestamp: 2026-08-06T22:18:59.719Z
Learning: When reviewing ComfyUI quantization code, treat `comfy.quant_ops.TensorWiseINT8Layout` and `comfy.quant_ops.TensorCoreConvRotW4A4Layout` as re-exports from `comfy_kitchen`. Validate their behavior against the re-exported `comfy_kitchen` implementations rather than assuming they are local fallback classes.

Applied to files:

• comfy/ops.py

[scriptease]

I have read and agree to the Contributor License Agreement

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

comfy/ops.py (1)

1206-1210: 🩺 Stability & Availability | 🔴 Critical | ⚡ Quick win

Decode weight_s_rel on CPU before moving it to MPS.

pop_scale("weight_s_rel") transfers the scale to device before the MPS branch. Native FP8 scales therefore reach MPS before conversion, and scale.float() can fail because MPS does not support FP8 tensors or casts.

Pop the raw scale without transferring it. On MPS, decode uint8 storage and convert the scale to float32 on CPU, then move it to device. Preserve the existing path for non-MPS devices.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@comfy/ops.py` around lines 1206 - 1210, Update the weight_s_rel scale
handling around pop_scale so the raw scale is retrieved without transferring it
to device. In the MPS branch, decode uint8 storage and convert it to float32 on
CPU before moving the scale to MPS; preserve the existing transfer and
conversion behavior for non-MPS devices.

Sources: Coding guidelines, Path instructions, MCP tools

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@comfy/ops.py`:
- Around line 1206-1210: Update the weight_s_rel scale handling around pop_scale
so the raw scale is retrieved without transferring it to device. In the MPS
branch, decode uint8 storage and convert it to float32 on CPU before moving the
scale to MPS; preserve the existing transfer and conversion behavior for non-MPS
devices.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3760e7b0-dfd4-4e84-aae6-da5e1e5eb4c9

📥 Commits

Reviewing files that changed from the base of the PR and between 2346dad and b830297.

📒 Files selected for processing (1)

• comfy/ops.py

📜 Review details

🧰 Additional context used

📓 Path-based instructions (6)

**/*

📄 CodeRabbit inference engine (AGENTS.md)

**/*: Keep changes small, direct, and limited to the narrowest necessary code path and smallest number of files.
Prefer practical fixes, minimal dependencies, and existing repository patterns; remove obsolete, dead, unreachable, or unused code.
Preserve existing APIs, node names, model-loading behavior, file layout, and workflow compatibility unless replacement is explicitly intended.
Core ComfyUI must not add outbound internet requests, telemetry, tracking, reporting, remote configuration, or background network activity. User-authorized model downloads are limited to the requested artifact and must exclude telemetry and unrelated metadata.

Files:

• comfy/ops.py

**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

**/*.py: Keep state and capability flags on the object that owns the behavior. Prefer explicit parent-owned attributes over probing child objects with getattr; use child checks only when the child owns the delegated behavior.
Preserve shared method signatures, argument order, return shapes, side effects, and error behavior unless every affected caller and interface is intentionally updated.
Do not add unused compatibility parameters, flags, attributes, constructor options, or model-specific options to shared helpers; keep one-off behavior at the integration boundary.
Normalize third-party return conventions at integration boundaries so core code receives the expected type and shape; avoid undocumented caller-side unwrapping.
Do not add torch.no_grad, torch.inference_mode, or inference-mode wrappers. Do not add model freeze/unfreeze toggles; only disable globally enabled inference mode when a training path requires gradients.
Remove inference-only training behavior such as dropout while preserving checkpoint and state-dict compatibility; use nn.Identity when deleting a module would alter keys or ordering.
Keep imports at module scope except established optional-backend probes or imports required to avoid cycles; avoid unnecessary try/except blocks and use specific exceptions with useful fallbacks.
Do not add workarounds for unsupported library versions, especially PyTorch exception-and-float-cast retries, unless a comment names the exact versions still requiring them.
Let unsupported model formats, invalid quantization metadata, and bad states fail with clear errors instead of silently degrading output.
Match local style, keep comments sparse and useful, and remove comments that merely restate obvious code.
Treat dtype, device placement, VRAM use, and offloading as correctness concerns across CPU, CUDA, ROCm, MPS, DirectML, XPU, NPU, and low-VRAM environments.
Prefer existing ComfyUI and Comfy Kitchen operations, quantization helpers, cast/offload helpe...

Files:

• comfy/ops.py

**/*.{py,json}

📄 CodeRabbit inference engine (AGENTS.md)

Treat legacy combo, io.Combo, and io.DynamicCombo values affecting filesystem access as untrusted; revalidate them at load/save boundaries with folder_paths, containment checks, or fixed allowlists.

Files:

• comfy/ops.py

**/*.{py,md,txt,json}

📄 CodeRabbit inference engine (AGENTS.md)

Keep warning and info messages short and actionable, remove noisy or misleading logging, and make documentation edits concise, factual, and tied to changed behavior.

Files:

• comfy/ops.py

**

⚙️ CodeRabbit configuration file

**: IMPORTANT: Only comment on issues directly introduced by this PR's code changes.
Treat AGENTS.md as mandatory repository policy, not optional style guidance.
Flag PR changes that violate AGENTS.md even when the code is otherwise functional.
In particular, enforce architecture boundaries, dtype/device/memory rules,
interface contracts, import style, no unnecessary try/except blocks, no inline
imports, no outbound internet paths in core ComfyUI, and narrow scoped fixes.
Prefer direct findings over suggestions when a rule is violated. Only ignore
AGENTS.md when it clearly conflicts with a newer explicit maintainer instruction
in the PR.
Do NOT flag pre-existing issues in code that was merely moved, re-indented,
de-indented, or reformatted without logic changes. If code appears in the diff
only due to whitespace or structural reformatting (e.g., removing a with: block),
treat it as unchanged. Contributors should not feel obligated to address
pre-existing issues outside the scope of their contribution.

Files:

• comfy/ops.py

comfy/**

⚙️ CodeRabbit configuration file

comfy/**: Core ML/diffusion engine. Focus on:

• Backward compatibility (breaking changes affect all custom nodes)
• Memory management and GPU resource handling
• Performance implications in hot paths
• Thread safety for concurrent execution

Files:

• comfy/ops.py

🧠 Learnings (3)

📚 Learning: 2026-02-21T14:01:41.482Z

Learnt from: pythongosssss
Repo: Comfy-Org/ComfyUI PR: 12555
File: comfy_extras/nodes_glsl.py:719-724
Timestamp: 2026-02-21T14:01:41.482Z
Learning: In PyOpenGL, bare Python scalars can be accepted for 1-element array parameters by NumberHandler. This means you can pass an int/float directly to OpenGL texture deletion (e.g., glDeleteTextures(tex)) without wrapping in a list. Verify function-specific expectations and ensure types match what the OpenGL call expects; use explicit lists only when the API requires an array.

Applied to files:

• comfy/ops.py

📚 Learning: 2026-05-13T12:31:45.069Z

Learnt from: rattus128
Repo: Comfy-Org/ComfyUI PR: 13802
File: comfy/pinned_memory.py:19-30
Timestamp: 2026-05-13T12:31:45.069Z
Learning: When reviewing code that uses comfy/pinned_memory.py’s `HostBuffer.extend(size=..., reallocate=...)`: by default (`reallocate` is not True / False), `extend(size=...)` is a *relative increment* that grows the buffer by `size` bytes—so slicing like `[offset:offset+size]` after `hostbuf.extend(size=size)` is correct and the argument should not be rewritten to `offset + size`. Only in the single-segment reallocation mode (`reallocate=True`, e.g., as used by `resize_pin_buffer()` in `comfy/model_management.py`) should `size` be treated as an *absolute target* and the call/arguments should be checked accordingly.

Applied to files:

• comfy/ops.py

📚 Learning: 2026-08-06T22:18:59.719Z

Learnt from: kijai
Repo: Comfy-Org/ComfyUI PR: 15362
File: comfy/ldm/wan/model_animate2.py:186-223
Timestamp: 2026-08-06T22:18:59.719Z
Learning: When reviewing ComfyUI quantization code, treat `comfy.quant_ops.TensorWiseINT8Layout` and `comfy.quant_ops.TensorCoreConvRotW4A4Layout` as re-exports from `comfy_kitchen`. Validate their behavior against the re-exported `comfy_kitchen` implementations rather than assuming they are local fallback classes.

Applied to files:

• comfy/ops.py