[release/11.0] Backport Tensor correctness and shape consistency fixes

src/libraries/System.Numerics.Tensors/README.md

@@ -1,3 +1,65 @@
 # System.Numerics.Tensors
 
 Provides APIs for performing primitive operations over tensors represented by spans of memory.
+
+Some shape and storage behavior intentionally differs from NumPy:
+
+- Creating a tensor from an empty shape (`[]`) produces shape `[0]`, with no elements. In NumPy,
+  shape `()` has rank zero but contains one scalar element; `np.empty(())` leaves that element
+  uninitialized ("empty" refers to initialization, not the number of elements).
+- `Tensor<T>.Empty` and default tensor spans retain rank-zero metadata but contain no
+  elements, unlike NumPy's scalar shape `()`. Tensor computations treat them as empty
+  vectors with effective shape `[0]`, corresponding to NumPy's `(0,)`. This includes
+  equality, broadcasting, reshaping, stacking, concatenation, splitting, slicing, and
+  dimension operations. Explicitly ranked empty shapes retain their axes.
+- Squeezing shape `[1, 1]` produces shape `[1]`, rather than NumPy's rank-zero shape `()`.
+  Both contain one element and can broadcast as a scalar; code that inspects rank, selects
+  axes, or indexes the result must account for the retained dimension.
+- Directional broadcasting can discard excess leading singleton dimensions. For example,
+  a source with shape `[1, 1, 3]` can broadcast into destination shape `[3]`, whereas
+  NumPy's `broadcast_to` rejects a target with fewer dimensions than the source.
+  This also applies to tensor copies and elementwise operations with supplied destinations.
+  Only leading singleton dimensions are redundant: `[2, 1]` and `[1, 2]` are distinct,
+  and zero-length dimensions cannot be discarded. Source metadata and explicitly requested
+  destination shapes are preserved.
+- Shape equality also ignores leading singleton padding, without broadcasting other
+  dimensions. Stacking requires equivalent input shapes; concatenation requires matching
+  aligned dimensions except along its selected axis. These operations use the first
+  input's effective shape to interpret the axis and determine the result's rank. Other
+  inputs and supplied destinations can add or omit leading singleton padding, but cannot
+  omit significant axes. For example, stacking `[2]` with `[1, 2]` at axis `0` produces
+  `[2, 2]`, while stacking `[1, 2]` with `[2]` at axis `0` produces `[2, 1, 2]`: inserting
+  the new axis makes the first input's retained singleton axis non-leading. Existing
+  axes are not renumbered or removed from the first input.
+- Strides must be nonnegative. NumPy can represent a reversed view with a negative stride
+  (for example, `array[::-1]`); use `Tensor.ReverseDimension` with dimension `0` to
+  reverse the first axis instead. This produces a new tensor, not a reversed view.
+- When growing a tensor, `Tensor.Resize` and `Tensor.ResizeTo` fill new elements with
+  `default(T)`. Resizing `[1, 2]` to five `int` elements yields `[1, 2, 0, 0, 0]`,
+  whereas `np.resize` repeats the input and yields `[1, 2, 1, 2, 1]`.
+- `Tensor.ResizeTo` and concatenation into an existing destination reject a zero-stride
+  dimension with more than one logical element: multiple output indexes would refer to the
+  same storage and could not hold distinct values. Zero strides in singleton dimensions
+  and empty destinations do not have this conflict.
+
+When strides are omitted, a shape containing a zero-length dimension has zero strides
+in every dimension. Its element count and storage requirement are zero regardless of
+the other dimension lengths. Negative lengths remain invalid, and explicitly supplied
+strides must still satisfy the normal layout validation.
+Empty slices and empty dimension views retain their source's storage origin: endpoint
+ranges do not move an empty view beyond its backing storage.
+
+Native-backed tensor spans can have more than `int.MaxValue` logical elements. Operations
+retain native-sized lengths and offsets, using bounded spans or indexed iteration rather
+than narrowing the total element count. Dense copies preserve overlap-safe ordering.
+Index-of-min/max reductions use `TensorPrimitives` over dense spans and dense suffixes.
+Other layouts retain the flattened primitive path for span-sized inputs and use bounded
+gathered blocks for native-width inputs. Chunk results retain logical native-sized indexes,
+including first-NaN and tie handling.
+
+Overlapping sources and destinations are supported for equal-length dense copies, which
+use the same overlap-safe behavior as `Span<T>.CopyTo`, and for elementwise operations on identical
+non-broadcast views. An in-place reversal of a dense tensor also needs no temporary
+storage. Other overlapping tensor layouts throw `ArgumentException` before writing:
+copying them correctly could require buffering an amount of data proportional to the
+tensor's size. Nonoverlapping strided copies do not create such a buffer.

src/libraries/System.Numerics.Tensors/src/Resources/Strings.resx

@@ -129,6 +129,9 @@
   <data name="Argument_InputAndDestinationSpanMustNotOverlap" xml:space="preserve">
     <value>The destination span may only overlap with an input span if the two spans start at the same memory location.</value>
   </data>
+  <data name="Argument_OverlappingTensorLayoutsNotSupported" xml:space="preserve">
+    <value>The input and destination tensor spans overlap in an unsupported layout.</value>
+  </data>
   <data name="Argument_DestinationSpansMustNotOverlap" xml:space="preserve">
     <value>Destination spans must not overlap with each other.</value>
   </data>
@@ -138,6 +141,9 @@
   <data name="Argument_InvalidEnumValue" xml:space="preserve">
     <value>The value '{0}' is not valid for this usage of the type {1}.</value>
   </data>
+  <data name="InvalidOperation_EnumerationNotPositioned" xml:space="preserve">
+    <value>Enumeration has not started or has already finished.</value>
+  </data>
   <data name="Argument_TypeContainsReferences" xml:space="preserve">
     <value>The type '{0}' is not supported because it contains references.</value>
   </data>
@@ -168,6 +174,9 @@
   <data name="ThrowArgument_DimensionsNotSame" xml:space="preserve">
     <value>Number of dimensions to slice does not equal the number of dimensions in the span</value>
   </data>
+  <data name="ThrowArgument_DestinationHasOverlappingElements" xml:space="preserve">
+    <value>The destination contains overlapping elements and cannot represent distinct output values.</value>
+  </data>
   <data name="ThrowArgument_FilterTensorMustEqualTensorLength" xml:space="preserve">
     <value>The total length of the filter tensor must equal the length of the tensor to be filtered.</value>
   </data>

src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/Common/TensorPrimitives.IIndexOfMinMaxOperator.cs

@@ -9,7 +9,7 @@ namespace System.Numerics.Tensors
 {
     public static unsafe partial class TensorPrimitives
     {
-        private interface IIndexOfMinMaxOperator<T>
+        internal interface IIndexOfMinMaxOperator<T>
         {
             static abstract T Aggregate(Vector128<T> value);
             static abstract T Aggregate(Vector256<T> value);
@@ -20,7 +20,7 @@ private interface IIndexOfMinMaxOperator<T>
             static abstract Vector512<T> Compare(Vector512<T> x, Vector512<T> y);
         }
 
-        private static int IndexOfMinMaxCore<T, TOperator>(ReadOnlySpan<T> x)
+        internal static int IndexOfMinMaxCore<T, TOperator>(ReadOnlySpan<T> x)
             where T : INumber<T> where TOperator : struct, IIndexOfMinMaxOperator<T>
         {
             if (x.IsEmpty)

src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/IReadOnlyTensor_1.cs

@@ -49,14 +49,14 @@ public interface IReadOnlyTensor<TSelf, T> : IReadOnlyTensor
 
         /// <summary>Copies the contents of the tensor into a destination tensor span.</summary>
         /// <param name="destination">The destination tensor span.</param>
-        /// <exception cref="ArgumentException"><paramref name="destination" /> is shorter than the source tensor.</exception>
-        /// <remarks>This method copies all of the source tensor to <paramref name="destination" /> even if they overlap.</remarks>
+        /// <exception cref="ArgumentException"><paramref name="destination" /> is shorter than the source tensor, or the source and destination overlap in an unsupported layout.</exception>
+        /// <remarks>Overlapping dense tensors with equal element counts and identical views are supported. Other overlapping layouts are rejected before copying.</remarks>
         void CopyTo(scoped in TensorSpan<T> destination);
 
         /// <summary>Flattens the contents of the tensor into a destination span.</summary>
         /// <param name="destination">The destination span.</param>
-        /// <exception cref="ArgumentException"><paramref name="destination" /> is shorter than the source tensor.</exception>
-        /// <remarks>This method copies all of the source tensor to <paramref name="destination" /> even if they overlap.</remarks>
+        /// <exception cref="ArgumentException"><paramref name="destination" /> is shorter than the source tensor, or the source and destination overlap in an unsupported layout.</exception>
+        /// <remarks>Overlapping dense tensors are supported. Other overlapping layouts are rejected before copying.</remarks>
         void FlattenTo(scoped Span<T> destination);
 
         /// <summary>Returns a span that can be used to access the flattened elements for a given dimension.</summary>
@@ -110,17 +110,19 @@ public interface IReadOnlyTensor<TSelf, T> : IReadOnlyTensor
         /// <summary>Attempts to copy the contents of this tensor into a destination tensor span and returns a value to indicate whether or not the operation succeeded.</summary>
         /// <param name="destination">The target of the copy operation.</param>
         /// <returns><see langword="true"/> if the copy operation succeeded; otherwise, <c>false</c>.</returns>
+        /// <exception cref="ArgumentException">The source and <paramref name="destination" /> overlap in an unsupported layout.</exception>
         /// <remarks>
-        ///   <para>If the source and <paramref name="destination" /> overlap, the entirety of the source is handled as if it was copied to a temporary location before it is copied to <paramref name="destination" />.</para>
+        ///   <para>Overlapping dense tensors with equal element counts and identical views are supported. Other overlapping layouts throw <see cref="ArgumentException" /> before copying.</para>
         ///   <para>If the <paramref name="destination" /> length is shorter than the source, no items are copied and the method returns <c>false</c>.</para>
         /// </remarks>
         bool TryCopyTo(scoped in TensorSpan<T> destination);
 
         /// <summary>Attempts to flatten the contents of this tensor into a destination span and returns a value to indicate whether or not the operation succeeded.</summary>
         /// <param name="destination">The target of the copy operation.</param>
         /// <returns><see langword="true"/> if the copy operation succeeded; otherwise, <c>false</c>.</returns>
+        /// <exception cref="ArgumentException">The source and <paramref name="destination" /> overlap in an unsupported layout.</exception>
         /// <remarks>
-        ///   <para>If the source and <paramref name="destination" /> overlap, the entirety of the source is handled as if it was flattened to a temporary location before it is copied to <paramref name="destination" />.</para>
+        ///   <para>Overlapping dense tensors are supported. Other overlapping layouts throw <see cref="ArgumentException" /> before copying.</para>
         ///   <para>If the <paramref name="destination" /> length is shorter than the source, no items are copied and the method returns <c>false</c>.</para>
         /// </remarks>
         bool TryFlattenTo(scoped Span<T> destination);

src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorDimensionSpan_1.cs

@@ -18,16 +18,24 @@ public readonly ref struct ReadOnlyTensorDimensionSpan<T>
 
         internal ReadOnlyTensorDimensionSpan(ReadOnlyTensorSpan<T> tensor, int dimension)
         {
-            if ((uint)dimension >= tensor.Rank)
+            int rank = tensor.Rank;
+            if (rank == 0)
+            {
+                tensor = new ReadOnlyTensorSpan<T>(in tensor._reference, TensorShape.Normalize(tensor._shape));
+                rank = tensor.Rank;
+            }
+
+            if ((uint)dimension >= rank)
             {
                 ThrowHelper.ThrowArgumentOutOfRangeException();
             }
             dimension += 1;
 
+            ReadOnlySpan<nint> lengths = tensor.Lengths;
             _tensor = tensor;
-            _length = TensorPrimitives.Product(tensor.Lengths[..dimension]);
+            _length = TensorShape.GetProduct(lengths[..dimension]);
             _dimension = dimension;
-            _sliceShape = TensorShape.Create((dimension != tensor.Rank) ? tensor.Lengths[dimension..] : [1], tensor.Strides[dimension..], tensor.IsPinned);
+            _sliceShape = TensorShape.Create((dimension != rank) ? lengths[dimension..] : [1], tensor.Strides[dimension..], tensor.IsPinned);
         }
 
         /// <summary>Gets <c>true</c> if the slices that exist within the tracked dimension are dense; otherwise, <c>false</c>.</summary>

src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan_1.cs

@@ -148,8 +148,11 @@ public ReadOnlyTensorSpan(ReadOnlySpan<T> span, scoped ReadOnlySpan<nint> length
         ///   <para>Returns default when <paramref name="array"/> is null.</para>
         ///   <para>The created tensor span has a single dimension that is the same length as <paramref name="array" />.</para>
         /// </remarks>
+        /// <exception cref="ArrayTypeMismatchException">The type of <paramref name="array"/> is not compatible with an array of <typeparamref name="T"/>.</exception>
         public ReadOnlyTensorSpan(Array? array)
         {
+            ThrowHelper.ThrowIfArrayTypeMismatch<T>(array, isReadOnly: true);
+
             _shape = TensorShape.Create(array);
             _reference = ref (array is not null)
                        ? ref Unsafe.As<byte, T>(ref MemoryMarshal.GetArrayDataReference(array))
@@ -175,8 +178,11 @@ public ReadOnlyTensorSpan(Array? array)
         ///   * <paramref name="strides" /> is not empty and contains an element that is negative.
         ///   * <paramref name="strides" /> is not empty and contains an element that is zero in a non leading position.
         /// </exception>
+        /// <exception cref="ArrayTypeMismatchException">The type of <paramref name="array"/> is not compatible with an array of <typeparamref name="T"/>.</exception>
         public ReadOnlyTensorSpan(Array? array, scoped ReadOnlySpan<int> start, scoped ReadOnlySpan<nint> lengths, scoped ReadOnlySpan<nint> strides)
         {
+            ThrowHelper.ThrowIfArrayTypeMismatch<T>(array, isReadOnly: true);
+
             _shape = TensorShape.Create(array, start, lengths, strides, out nint linearOffset);
             _reference = ref (array is not null)
                        ? ref Unsafe.Add(ref Unsafe.As<byte, T>(ref MemoryMarshal.GetArrayDataReference(array)), linearOffset)
@@ -237,7 +243,7 @@ public unsafe ReadOnlyTensorSpan(T* data, nint dataLength, scoped ReadOnlySpan<n
 
         internal ReadOnlyTensorSpan(ref readonly T data, nint dataLength, scoped ReadOnlySpan<nint> lengths, scoped ReadOnlySpan<nint> strides, bool pinned)
         {
-            _shape = TensorShape.Create(in data, dataLength, lengths, strides, pinned);
+            _shape = TensorShape.CreateForView(dataLength, lengths, strides, pinned);
             _reference = ref Unsafe.AsRef(in data);
         }
 
@@ -402,7 +408,7 @@ public ReadOnlySpan<T> GetSpan(scoped ReadOnlySpan<NIndex> startIndexes, int len
         /// <inheritdoc cref="IReadOnlyTensor{TSelf, T}.Slice(ReadOnlySpan{nint})" />
         public ReadOnlyTensorSpan<T> Slice(params scoped ReadOnlySpan<nint> startIndexes)
         {
-            TensorShape shape = _shape.Slice<TensorShape.GetOffsetAndLengthForNInt, nint>(startIndexes, out nint linearOffset);
+            TensorShape shape = _shape.Slice<TensorShape.GetOffsetAndLengthForSlice, nint>(startIndexes, out nint linearOffset);
             return new ReadOnlyTensorSpan<T>(
                 ref Unsafe.Add(ref _reference, linearOffset),
                 shape
@@ -447,7 +453,8 @@ ref Unsafe.Add(ref _reference, linearOffset),
         /// <inheritdoc cref="IReadOnlyTensor{TSelf, T}.TryCopyTo(in TensorSpan{T})" />
         public bool TryCopyTo(scoped in TensorSpan<T> destination)
         {
-            if (TensorShape.AreCompatible(destination._shape, _shape, false))
+            if ((_shape.FlattenedLength <= destination.FlattenedLength) &&
+                TensorShape.AreCompatible(destination._shape, _shape, false))
             {
                 TensorOperation.Invoke<TensorOperation.CopyTo<T>, T, T>(this, destination);
                 return true;
@@ -536,46 +543,55 @@ ReadOnlyTensorSpan<T> IReadOnlyTensor<ReadOnlyTensorSpan<T>, T>.ToDenseTensor()
         public ref struct Enumerator : IEnumerator<T>
         {
             private readonly ReadOnlyTensorSpan<T> _span;
-            private readonly nint[] _indexes;
             private nint _linearOffset;
             private nint _itemsEnumerated;
+            private bool _hasCurrent;
 
             internal Enumerator(ReadOnlyTensorSpan<T> span)
             {
                 _span = span;
-                _indexes = new nint[span.Rank];
-
-                _indexes[^1] = -1;
-
-                _linearOffset = 0 - (!span.IsEmpty ? span.Strides[^1] : 0);
+                _linearOffset = 0;
                 _itemsEnumerated = 0;
+                _hasCurrent = false;
             }
 
             /// <summary>Gets the element at the current position of the enumerator.</summary>
-            public readonly ref readonly T Current => ref Unsafe.Add(ref _span._reference, _linearOffset);
+            public readonly ref readonly T Current
+            {
+                get
+                {
+                    if (!_hasCurrent)
+                    {
+                        ThrowHelper.ThrowInvalidOperation_EnumerationNotPositioned();
+                    }
+                    return ref Unsafe.Add(ref _span._reference, _linearOffset);
+                }
+            }
 
             /// <summary>Advances the enumerator to the next element of the tensor span.</summary>
             public bool MoveNext()
             {
                 if (_itemsEnumerated == _span._shape.FlattenedLength)
                 {
+                    _hasCurrent = false;
                     return false;
                 }
 
-                _linearOffset = _span._shape.AdjustToNextIndex(_span._shape, _linearOffset, _indexes);
+                _linearOffset = _span.IsDense
+                    ? _itemsEnumerated
+                    : _span._shape.GetLinearOffsetForDimension(_itemsEnumerated, _span.Rank);
 
                 _itemsEnumerated++;
+                _hasCurrent = true;
                 return true;
             }
 
             /// <summary>Sets the enumerator to its initial position, which is before the first element in the tensor span.</summary>
             public void Reset()
             {
-                Array.Clear(_indexes);
-                _indexes[^1] = -1;
-
-                _linearOffset = 0 - (!_span.IsEmpty ? _span.Strides[^1] : 0);
+                _linearOffset = 0;
                 _itemsEnumerated = 0;
+                _hasCurrent = false;
             }
 
             //