Intel architecture improvements for .NET 9

[BruceForstall]

This issue describes planned improvements to Intel architecture (x86, x64) ISA support for .NET 9.

In .NET 8, AVX-512 ISA support was added (see #77034). In .NET 9, this support will be further improved and leveraged for improved performance, especially with expanded libraries utilization of the recently implemented AVX-512 support. Investigations and implementation will start to support the newly announced AVX10.

Libraries work

• (Q4'23) Light up BitArray with Vector512
  • Light up BitArray APIs with Vector512 code path #91903
• (Q4'23) Light up String with Vector512
  • Light up String.Manipulation APIs with Vector512 codepath #93043
• (Q4'23) Light up Base64 encode/decode with Vector512
  • Adding AVX512 path to Base64 encoding/Decoding #92241
• Consider SIMD/AVX optimization for Tensor ([API Proposal]: Future of Numerics and AI - Provide a downlevel System.Numerics.Tensors.TensorPrimitives class #89639)

AVX10

AVX10 is a new set of vector ISA extensions, described here. We expect to begin preliminary work to support AVX10 in .NET 9, at least the parts that most directly map to the already supported AVX-512. An arch-avx10 GitHub label is defined to be added to all related PRs and issues: https://github.com/dotnet/runtime/labels/arch-avx10

• Add VM/JIT AVX10 awareness: CPUID enumeration and detection Adding Avx10v1 to the runtime #99784
• Propose a new AVX10 API: [API Proposal]: Expose AVX10 converged vector ISA #98069
• (Q2'24) Do JIT codegen implementation of the API
• (Q2'24) Add AVX10 APIs AVX10.1 API introduction in JIT #101938 Cleanup some handling around Avx10v1 #103241
• Enhance Vector256 codegen with AVX10 instructions (related to what has already been done for AVX512VL) AVX10.1 API introduction in JIT #101938 Cleanup some handling around Avx10v1 #103241
• (Q2'24) Allow additional 16 YMM registers for AVX10 AVX10.1 API introduction in JIT #101938 Cleanup some handling around Avx10v1 #103241
• Allow AVX-512 optimizations for YMM (e.g., scalar conversion, vpternlog) AVX10.1 API introduction in JIT #101938 Cleanup some handling around Avx10v1 #103241
  -- The current avx512 optimizations is working for avx10 targets
• Identify test plan for .NET 9 sign-off
  -- @tannergooding has identified a set of tests and @khushal1996 has successfully completed all stress tests getting expected results.

RyuJIT feature work

• (Q4'23) Enable EVEX embedded rounding support in xarch emitter. Enable EVEX embedded rounding support in xarch emitter #93154
  • Enable EVEX feature: Embedded Rounding for Avx512F.Add() #94684
• (Q4'23) Add optimization for scalar/vector conversion of uint32/uint64 to/from packed float/double. Add optimization for scalar/vector conversion of uint32/uint64 to/from packed float/double #80829
• (Q4'23) Finish AVX-512 specific light-up for Vector128/256/512<T> Finish Avx512 specific lightup for Vector128/256/512<T> #85207
  • Accelerating Vector512.Sum() #87851
  • Updating Sum() implementation for Vector128 and Vector256 + adding lowering for Vector512 #95568
    - All done except for Vector512.Dot, which will be pushed out to future item.
• Add EVEX encoding opmask (k) register masking for per-instruction opmask to xarch emitter. Add EVEX encoding opmask (k) register masking to xarch emitter #80821

RyuJIT optimization work

• AVX512: Fold some bitwise operations to vpternlogq AVX512: Fold some bitwise operations to vpternlogq #84534 (@tannergooding to open a future work item for further optimizations)
• Add optimization for scalar conversion of float/double to ulong Add optimization for scalar conversion of float/double to ulong. #89279

API design work

• Expose System.Runtime.Intrinsics.X86.Avx512F #73604
• (Reconsider implementing?) Expose VectorMask<T> to support generic masking for Vector<T> #74613
• Expose AVX512BW, AVX512CD, and AVX512DQ #76579

Future Work

Some of the planned work for .NET 9 have been pushed out to future work.

Libraries work

• Light up IndexOfAnyAsciiSearcher for AVX512. Light up IndexOfAnyAsciiSearcher for AVX512 #93222 (@MihaZupan)
• Consider SIMD JSON acceleration (Owner: Intel)
• Consider XML API acceleration (Owner: Intel)
• (Help Wanted) Light up Utf8/Utf16 code with Vector512. Light up Utf8Utility.*.cs and Utf16Utility.*.cs with Vector512 code paths.  #86119
• (Help Wanted) Light up Ascii.Utility methods with Vector512 code paths. Light up Ascii.Utility methods with Vector512 code paths. #89280
  • Switch to iSimdVector and Align WidenAsciiToUtf16 #99982
  • Light up core ASCII.Utility methods with Vector256/Vector512 code paths. #88532
  • Use StoreAligned not Store in WidenAsciiToUtf16 #89892

AVX10

• Allow embedded rounding for YMM/ZMM (related: Enable EVEX embedded rounding support in xarch emitter #93154) (Owner: Intel, Starting AVX10.2)
• (Help Wanted) Convert remaining AVX2 implementations to Vector256 Switching to Vectors from target dependent instrinsics #101251

RyuJIT feature work

• Vector512.Dot: AVX-512 specific light-up for Vector512.Dot Finish Avx512 specific lightup for Vector128/256/512<T> #85207

Vector<T>

• Consider Vector<T> expanding to Vector512<T>, either automatically or opt-in. (@tannergooding plans to get back to it as a best effort.)

JCC erratum

• RyuJIT: Implement mitigation for Intel JCC erratum #93243 (@TIHan)

Debugging / diagnostics work (@BruceForstall)

• AVX-512 debugger support: view registers #87854
• Ensure ELT (enter/leave/tailcall hooks, for profiling) works.

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This issue describes planned improvements to Intel architecture (x86, x64) ISA support for .NET 9.

In .NET 8, AVX-512 ISA support was added (see #77034). In .NET 9, this support will be further improved and leveraged for improved performance, especially with expanded libraries utilization of the recently implemented AVX-512 support. Investigations and implementation will start to support the newly announced AVX10.

Libraries work

• Light up Utf8/Utf16 code with Vector512. Light up Utf8Utility.*.cs and Utf16Utility.*.cs with Vector512 code paths.  #86119
• Light up Ascii.Utility methods with Vector512 code paths. Light up Ascii.Utility methods with Vector512 code paths. #89280
• Light up BitArray with Vector512
• Light up String with Vector512
• Light up Base64 encode/decode with Vector512
• Consider SIMD JSON acceleration?
• Consider XML API acceleration?
• Consider SIMD/AVX optimization for Tensor ([API Proposal]: Future of Numerics and AI - Provide a downlevel System.Numerics.Tensors.TensorPrimitives class #89639)

RyuJIT feature work

• Consider Vector expanding to Vector512, either automatically or opt-in.

RyuJIT feature work

• Add EVEX encoding opmask (k) register masking for per-instruction opmask to xarch emitter. Add EVEX encoding opmask (k) register masking to xarch emitter #80821
• Enable EVEX embedded rounding support in xarch emitter. Enable EVEX embedded rounding support in xarch emitter #93154
• Add optimization for scalar/vector conversion of uint32/uint64 to/from packed float/double. Add optimization for scalar/vector conversion of uint32/uint64 to/from packed float/double #80829
• Finish Avx512 specific lightup for Vector128/256/512<T> #85207

RyuJIT optimization work

• AVX512: Fold some bitwise operations to vpternlogq AVX512: Fold some bitwise operations to vpternlogq #84534
• Add optimization for scalar conversion of float/double to ulong Add optimization for scalar conversion of float/double to ulong. #89279

AVX10

AVX10 is a new set of vector ISA extensions, described here. We expect to begin preliminary work to support AVX10 in .NET 9, at least the parts that most directly map to the already supported AVX-512.

• Add VM/JIT AVX10 awareness: CPUID enumeration and detection
• Propose a new AVX10 API
• Do JIT codegen implementation of the API
• Enhance Vector256 codegen with AVX10 instructions (related to what has already been done for AVX512VL)
• Allow additional 16 YMM registers for AVX10
• Allow embedded rounding for YMM/ZMM (related: Enable EVEX embedded rounding support in xarch emitter #93154)
• Convert remaining AVX2 implementations to Vector256
• Allow AVX-512 optimizations for YMM (e.g., scalar conversion, vpternlog)

CI/testing work

Debugging / diagnostics work

• AVX-512 debugger support: view registers #87854
• Ensure ELT (enter/leave/tailcall hooks, for profiling) works.

API design work

• Expose System.Runtime.Intrinsics.X86.Avx512F #73604
• (Reconsider implementing?) Expose VectorMask<T> to support generic masking for Vector<T> #74613
• Expose AVX512BW, AVX512CD, and AVX512DQ #76579

Author:	BruceForstall
Assignees:	-
Labels:	area-CodeGen-coreclr, User Story
Milestone:	9.0.0

[MichalPetryka]

Is there maybe any interest in adding the workaround for the JCC erratum (#35730) in .Net 9? I've seen minor codegen improvements be reported as huge regressions because the code started to hit this issue.

[BruceForstall]

Is there maybe any interest in adding the workaround for the JCC erratum (#35730) in .Net 9? I've seen minor codegen improvements be reported as huge regressions because the code started to hit this issue.

@AndyAyersMS has expressed a desire to at least have a mode that could be used for performance testing to avoid the JCC erratum. Whether we could enable this always would depend on how uniform the improvements would be. It is expected there would be some code size regressions -- possibly significant -- due to the need to insert NOPs.

[MichalPetryka]

It is expected there would be some code size regressions -- possibly significant -- due to the need to insert NOPs.

Didn't we already accept that tradeoff with loop alignment?

[BruceForstall]

Didn't we already accept that tradeoff with loop alignment?

Yes, but this could be a very different magnitude of regression that will need to be measured.

[BruceForstall]

I went ahead and created #93243 related to adding a JIT mode to avoid the JCC erratum, and linked it here.

[Spacefish]

I added Vector512 support for Min/Max of simple numeric datatypes in this PR: #93369

[huoyaoyuan]

What about the upcoming APX extension? It looks like a major change of x86-64. I can see discussions around ABI for APX in GCC mail thread: https://gcc.gnu.org/pipermail/gcc/2023-July/242154.html https://gcc.gnu.org/pipermail/gcc-help/2023-August/142801.html

Maybe it's too early for .NET to adopt APX, but I'd like to see the estimated timeline. Should we wait for MSVC to define the calling convention?

[tannergooding]

We want to have hardware available on which it can run.

While Intel hasn't given an official timeline as of yet, such hardware is most likely not in the .NET 9 lifetime which ships in November 2024 and will be out of support around May 2026.

I expect this work will be done for .NET 10 which will likely ship around November 2025 (assuming we don't change our current pacing of releases) and be out of support November 2028.

[MichalPetryka]

We want to have hardware available on which it can run.

Would using Intel SDE not be enough for testing the support for it? It seems to already have support for emulating AVX10 and APX.

[tannergooding]

There's no point in scheduling work to be done for hardware that doesn't exist yet, particularly if that hardware is unlikely to exist within the lifetime of a release.

That is, we know that AVX10 is going to exist for Granite Rapids, as per the official announcement: https://www.intel.com/content/www/us/en/content-details/784267/intel-advanced-vector-extensions-10-intel-avx10-architecture-specification.html. The AVX10.1 work is correspondingly happening in .NET 9

While no official release date has been announced for APX, it is unlikely to happen in a timeframe that makes .NET 9 a good choice to target.