Skip to content

LoRA fallback follow-ups from #32 (TAE on Metal, layer-split probe, test gaps) #33

Open

Description

@leehack

Non-blocking follow-ups from the re-audit of #32 (CLEAR at 6c4d699; evidence llamadart-evidence/sd-native-pr-32/audit-r2/).

  • TAE LoRA silently not applied on Metal. CPU auto merges a tae. LoRA; Metal falls back to at_runtime, which upstream never applies to the TAE. Previously it aborted. Warn (or refuse) when a LoRA has tensors no module consumes. Reachable only via tae.-prefixed files or SDXS VAE LoRAs.
  • Layer split probe: the fallback probe uses runtime_backend(), the first device of a & split (ggml_extend_backend.cpp:934); CPU&MTL0 isn't probed on Metal. Use runtime_backends(module).
  • Unprobed modules: CLIP_VISION, CONTROL_NET, PHOTOMAKER, AUDIO_ENCODER.
  • Tests: BLAS "keeps merging" untested (host-buffer shortcut mutant survives); stale docstring tests/test_fp8_runtime.py:15 (runners list MTL0 but can't generate); CI never exercises the Metal fallback (local only).
  • Warning wording: "F16 weights on MTL0" appears for F32 models (accurate: conv weights load as F16) — consider naming the cause.
  • Behavior note: linear-only LoRAs on F32/BF16 bases on Metal now apply at runtime instead of merging (same image, per-step cost).

Activity

  1. leehack commented on Oct 28, 2026

    @leehack
    OwnerAuthor

    Triage: P3. Non-blocking LoRA-fallback follow-ups from #32 (TAE on Metal, layer-split probe, test gaps); not needed for llamadart 0.17.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority:P3Useful cleanup or longer-term work

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions