Repository navigation
Conversation
Collaborator
|
r? @Amanieu rustbot has assigned @Amanieu. Use Why was this reviewer chosen?The reviewer was selected based on:
|
This was referenced Oct 4, 2026
JonathanBrouwer
added a commit
to JonathanBrouwer/rust
that referenced
this pull request
Oct 4, 2026
…=folkertdev
Add the `movdir64b` and `movdiri` x86 target features
Adds the unstable `movdir64b` and `movdiri` x86 target features, gated behind `#![feature(movdir64b_target_feature)]` and `#![feature(movdiri_target_feature)]`, and runtime detection with `is_x86_feature_detected!("movdir64b")` / `is_x86_feature_detected!("movdiri")` (CPUID.(EAX=7,ECX=0):ECX[28] and ECX[27]).
`MOVDIR64B` moves 64 bytes from memory to a 64-byte aligned destination as a single direct store; `MOVDIRI` stores a 32- or 64-bit integer as a direct store. Both are on Intel Tiger Lake / Alder Lake and later, Sapphire Rapids and later Xeons, and AMD Zen 5. They're standalone CPUID features that aren't part of any psABI microarchitecture level, so they need their own feature flags.
**This is the compiler half of a two-PR feature, like `clflushopt` in rust-lang#157098.** The `_movdir64b`, `_directstoreu_u32` and `_directstoreu_u64` intrinsics (rust-lang/stdarch#2239) can't compile until these target features exist, so the stdarch PR is blocked on this one merging and syncing into stdarch's pinned toolchain.
- Tracking issue: rust-lang#163741
- Unblocks intrinsics: rust-lang/stdarch#2239
Today the only way to emit these instructions from Rust is `asm!`, which also hides them from LLVM: a copy loop through `llvm.x86.movdir64b` gets unrolled and the source address becomes a displacement, while the `asm!` version stays one instruction per iteration with both addresses in registers.
`movdir64b` and `movdiri` are LLVM's feature names, so they map 1:1 through `to_llvm_features` with no remap entry. I ran the two new feature-gate tests, `check-cfg/target_feature` and `target-feature/invalid-attribute`, the std_detect tests (which report both features as `true` on a Xeon 6975P-C), and tidy locally on x86_64-pc-windows-msvc.
r? @folkertdev
JonathanBrouwer
added a commit
to JonathanBrouwer/rust
that referenced
this pull request
Oct 4, 2026
…=folkertdev
Add the `movdir64b` and `movdiri` x86 target features
Adds the unstable `movdir64b` and `movdiri` x86 target features, gated behind `#![feature(movdir64b_target_feature)]` and `#![feature(movdiri_target_feature)]`, and runtime detection with `is_x86_feature_detected!("movdir64b")` / `is_x86_feature_detected!("movdiri")` (CPUID.(EAX=7,ECX=0):ECX[28] and ECX[27]).
`MOVDIR64B` moves 64 bytes from memory to a 64-byte aligned destination as a single direct store; `MOVDIRI` stores a 32- or 64-bit integer as a direct store. Both are on Intel Tiger Lake / Alder Lake and later, Sapphire Rapids and later Xeons, and AMD Zen 5. They're standalone CPUID features that aren't part of any psABI microarchitecture level, so they need their own feature flags.
**This is the compiler half of a two-PR feature, like `clflushopt` in rust-lang#157098.** The `_movdir64b`, `_directstoreu_u32` and `_directstoreu_u64` intrinsics (rust-lang/stdarch#2239) can't compile until these target features exist, so the stdarch PR is blocked on this one merging and syncing into stdarch's pinned toolchain.
- Tracking issue: rust-lang#163741
- Unblocks intrinsics: rust-lang/stdarch#2239
Today the only way to emit these instructions from Rust is `asm!`, which also hides them from LLVM: a copy loop through `llvm.x86.movdir64b` gets unrolled and the source address becomes a displacement, while the `asm!` version stays one instruction per iteration with both addresses in registers.
`movdir64b` and `movdiri` are LLVM's feature names, so they map 1:1 through `to_llvm_features` with no remap entry. I ran the two new feature-gate tests, `check-cfg/target_feature` and `target-feature/invalid-attribute`, the std_detect tests (which report both features as `true` on a Xeon 6975P-C), and tidy locally on x86_64-pc-windows-msvc.
r? @folkertdev
rust-bors Bot
pushed a commit
to rust-lang/rust
that referenced
this pull request
Oct 4, 2026
Rollup merge of #163742 - jlpetz:movdir64b-target-feature, r=folkertdev Add the `movdir64b` and `movdiri` x86 target features Adds the unstable `movdir64b` and `movdiri` x86 target features, gated behind `#![feature(movdir64b_target_feature)]` and `#![feature(movdiri_target_feature)]`, and runtime detection with `is_x86_feature_detected!("movdir64b")` / `is_x86_feature_detected!("movdiri")` (CPUID.(EAX=7,ECX=0):ECX[28] and ECX[27]). `MOVDIR64B` moves 64 bytes from memory to a 64-byte aligned destination as a single direct store; `MOVDIRI` stores a 32- or 64-bit integer as a direct store. Both are on Intel Tiger Lake / Alder Lake and later, Sapphire Rapids and later Xeons, and AMD Zen 5. They're standalone CPUID features that aren't part of any psABI microarchitecture level, so they need their own feature flags. **This is the compiler half of a two-PR feature, like `clflushopt` in #157098.** The `_movdir64b`, `_directstoreu_u32` and `_directstoreu_u64` intrinsics (rust-lang/stdarch#2239) can't compile until these target features exist, so the stdarch PR is blocked on this one merging and syncing into stdarch's pinned toolchain. - Tracking issue: #163741 - Unblocks intrinsics: rust-lang/stdarch#2239 Today the only way to emit these instructions from Rust is `asm!`, which also hides them from LLVM: a copy loop through `llvm.x86.movdir64b` gets unrolled and the source address becomes a displacement, while the `asm!` version stays one instruction per iteration with both addresses in registers. `movdir64b` and `movdiri` are LLVM's feature names, so they map 1:1 through `to_llvm_features` with no remap entry. I ran the two new feature-gate tests, `check-cfg/target_feature` and `target-feature/invalid-attribute`, the std_detect tests (which report both features as `true` on a Xeon 6975P-C), and tidy locally on x86_64-pc-windows-msvc. r? @folkertdev
jlpetz
force-pushed
the
movdir64b-intrinsic
branch
from
October 5, 2026 01:46
adf1ee4 to
fe6658b
Compare
Add the intrinsics for the `MOVDIR64B` (64-byte direct store) and `MOVDIRI` (32/64-bit integer direct store) instructions, gated on the `movdir64b` and `movdiri` target features. They link `llvm.x86.movdir64b` and `llvm.x86.directstore32`/`directstore64`, and have `assert_instr` codegen tests and `simd_test` runtime tests. `_directstoreu_u64` lives in `x86_64` since `MOVDIRI` with a 64-bit operand needs 64-bit mode. stdarch-verify maps `MOVDIR64B`'s `void*` operands to `*u8` and `MOVDIRI`'s to the integer type it stores, and missing-x86.md is regenerated.
jlpetz
force-pushed
the
movdir64b-intrinsic
branch
from
October 7, 2026 12:35
fe6658b to
ac8ea51
Compare
jlpetz
added a commit
to jlpetz/plumb
that referenced
this pull request
Oct 8, 2026
- Each crate carries LICENSE-APACHE and LICENSE-MIT, so `cargo package` includes them. - plumb_tiles' example is now `tile_kernels`; both crates had an example called `asm_kernels`. - README gains an Upstream section, and the docs link the PRs that would retire `asm!`: llvm/llvm-project#229240 for NT stores, rust-lang/rust#163742 and rust-lang/stdarch#2239 for MOVDIR64B, and llvm/llvm-project#229025 for the AMX intrinsics' memory effects. - TODO 1 closes (first CI runs green); TODO 5 records #228875's close and the fixes offered on #229240. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
|
So "direct store" and "non-temporal store" mean the same thing on x86? Or is this a different kind of cursed semantics? If so, it's not obvious to me that the approach we took for non-temporal stores will work here. (Note that under our LLM policy, PR descriptions must be fully written by humans, and AI-written code requires a pre-agreed reviewer before opening the PR. And in particular for tricky questions such as these, it would be completely irresponsible to rely on AI output. We need human experts to answer these questions.) |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the
_movdir64bintrinsic forMOVDIR64B(a 64-byte direct store) and the_directstoreu_u32/_directstoreu_u64intrinsics forMOVDIRI(a 32/64-bit integer direct store). Until now stdarch had no way to emit either instruction from an intrinsic.Gated on the
movdir64bandmovdiritarget features added in rust-lang/rust#163742 (tracking issue rust-lang/rust#163741). CI will be red until that compiler change merges and syncs into the pinned toolchain here:#[target_feature(enable = "movdir64b")]andenable = "movdiri"won't be recognized before then.Modeled on
_mm_clflushopt: each links its LLVM intrinsic (llvm.x86.movdir64b,llvm.x86.directstore32,llvm.x86.directstore64) and has anassert_instrcodegen test and asimd_testruntime test that stores and reads the value back._directstoreu_u64is inx86_64, sinceMOVDIRIwith a 64-bit operand needs 64-bit mode.Types: Intel declares the destinations as
void*._movdir64btakes*mut u8/*const u8(MOVDIR64Bjoins the*u8CPUID list in stdarch-verify), and_directstoreu_u32/_directstoreu_u64take*mut u32/*mut u64, the type they store (a new MOVDIRI rule in stdarch-verify).missing-x86.mdis regenerated withGENERATE_MISSING_X86_MD.Docs:
# Safetycovers the pointers:_movdir64b'sdstmust be 64-byte aligned (the instruction faults otherwise), andMOVDIRI's destination needs no alignment.# Safety of direct storessays the store is weakly ordered like a non-temporal store, so it carries the same_mm_sfencerequirement, in the same words as_mm_stream_si128. Unlikestore !nontemporal(rust-lang/rust#114582), these are target intrinsics with unknown memory effects, so LLVM doesn't optimize them as ordinary stores.I tested it locally against a stage1 toolchain built from rust-lang/rust#163742: the six
core_archtests (runtime tests run on a Xeon 6975P-C, which has both features),stdarch-verify's x86 test, andcargo fmt --check.