x86-64 SIMD: SSE intrinsics disabled due to LLVM crash on soft-float target

[johnjezl]

Description

The inference engine's SIMD operators (relu, add, matmul, softmax) have AArch64 NEON implementations but fall back to scalar on x86-64. SSE2 intrinsic versions were implemented but had to be removed because they trigger an LLVM code generation crash:

rustc-LLVM ERROR: Do not know how to split the result of this operator!

This occurs on the `x86_64-unknown-none` target, which defaults to soft-float (no SSE). Even with `-C target-feature=+sse,+sse2` in .cargo/config.toml, LLVM crashes during LTO when the `libm` crate's arch-specific code interacts with SSE-enabled FP operations.

Root Cause

The `x86_64-unknown-none` Rust target spec disables SSE by default for bare-metal. Enabling SSE via target features causes incompatibilities between:

1. The soft-float ABI assumed by the target spec
2. SSE register usage in optimized FP code
3. `libm` crate's arch-specific optimizations (`cfg(arch_enabled)`)

Impact

• x86-64 inference uses scalar loops (~2-3x slower than SSE for matmul)
• MNIST inference still works correctly, just slower
• ARM64 (primary target) is unaffected — full NEON SIMD

Possible Fixes

1. Custom target spec with SSE enabled (requires nightly `-Z build-std`)
2. `#[target_feature(enable = "sse2")]` on individual functions (may work on stable)
3. Disable `libm` arch feature for x86-64 only (`cfg`-gated dependency)
4. Wait for Rust x86-64 bare-metal improvements (target spec evolution)

Documented In

• `runtime/src/inference/ops.rs` line 188-189 (code comment)
• `docs/api/runtime.md` SIMD Optimization section

Priority

Medium — affects x86-64 performance but not correctness. ARM64 is the primary deployment target.

[johnjezl]

Investigation — 2026-04-13 (stable Rust 1.92, bare-metal)

Attempted the three paths listed in the issue on stable Rust 1.92. None of them land cleanly on the current x86_64-unknown-none target. Summary of what each failure looks like, so the next attempt doesn't have to re-derive them:

Per-function #[target_feature(enable = "sse2")] alone

Adding #[target_feature(enable = "sse2")] to an inner unsafe fn containing _mm_loadu_ps / _mm_max_ps / _mm_storeu_ps is the lightest-touch approach and the one the issue recommends. Under the default cargo config (no global +sse,+sse2) it still crashes at codegen time:

rustc-LLVM ERROR: Do not know how to split the result of this operator!
error: could not compile `slm-runtime`

This happens on debug, release, and release + LTO. The target spec pins +soft-float in its features list, and __m128 return values from intrinsics can't be legalized against a soft-float ABI, even when only a single function opts into sse2.

Path B — global +sse,+sse2, libm's arch feature disabled

Removing libm's default features (it enables arch-specific FP optimizations that go through SSE on x86) avoids the "soften this operator's operand" crash inside libm itself. But the slm-runtime crate still hits the "split the result" crash as soon as it uses any core::arch::x86_64 intrinsic — same root cause as above, the +soft-float ABI in the target spec.

Path C — global +sse,+sse2,-soft-float (override the target spec)

This is the only combination that compiles (cargo build --target x86_64-unknown-none --release finishes). rustc warns:

warning: target feature `soft-float` must be enabled to ensure that the ABI
         of the current target can be implemented correctly
         (will become a hard error in a future release)

At runtime it crashes inside unrelated code — LLVM starts emitting 16-byte-aligned movaps for routine struct zero-init, but Rust keeps laying structs out for the soft-float ABI (8-byte alignment), so the kernel takes #GP inside component_register on the first SSE-backed zero store:

*** EXCEPTION: #GP General Protection (vector 0xd)
  RIP: 0x14406b  → movaps %xmm0,0x40(%r13)    ; r13 is 8-aligned

What would actually work (out of scope for this PR)

The only genuine fix on this target is a custom target JSON that drops +soft-float from features and sets the right stack / struct alignment in data-layout. That requires cargo build -Z build-std (nightly-only) and a committed decision to use nightly in CI for the x86-64 build. Relevant upstream work:

• .gitignore files for nix rust-lang/rust#131176 — tracking issue for freestanding SSE/vector support on bare-metal x86
• rustc --target x86_64-unknown-none -Z unstable-options --print target-spec-json — starting point for the custom spec

What I verified and reverted

• Per-function #[target_feature(enable = "sse2")]: reverted.
• libm = { default-features = false }: reverted (it's a valid change but pointless without usable SSE).
• -C target-feature=+sse,+sse2,-soft-float: reverted (would silently #GP in production).
• CR0.MP / CR4.OSFXSR / CR4.OSXMMEXCPT setup in trampoline32.S and ap_trampoline.S: reverted. These would be needed alongside a working Rust toolchain path; on their own they're dead code.

Recommendation

Mark this blocked on upstream Rust (custom target spec + -Z build-std once stabilized, or a #[repr(align(16))] + asm-based alternative — both significantly larger than the scope of this ticket). x86-64 inference stays on the scalar path, matching the doc comment in runtime/src/inference/ops.rs.