.NET 11 regression: ANSI char[] marshalling rejects multibyte characters #134573

Description

Built-in LPArray marshalling of char[] with CharSet.Ansi rejects a character that is representable in the process ANSI code page but needs more than one encoded byte. With a UTF-8 process manifest, U+00E9 should marshal to the two bytes C3 A9. The same program succeeds on .NET 10 and throws ArgumentException before the native callee runs on .NET 11.

Reproduction steps

Create an empty directory outside the runtime repository and save the following as Repro.csproj and Program.cs. No external native library or NuGet package is required; the native-facing entry point is provided through UnmanagedCallersOnly or Windows built-in COM interop.

Repro.csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <Nullable>enable</Nullable>
    <ApplicationManifest>app.manifest</ApplicationManifest>
  </PropertyGroup>
</Project>

Program.cs

// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Text;

internal static unsafe class Program
{
    [UnmanagedFunctionPointer(CallingConvention.Cdecl, CharSet = CharSet.Ansi, BestFitMapping = false)]
    private delegate int ReadArray([MarshalAs(UnmanagedType.LPArray)] char[] values, int byteCount);

    [DllImport("kernel32.dll")]
    private static extern uint GetACP();

    private static int Main()
    {
        Console.WriteLine(RuntimeInformation.FrameworkDescription);
        uint codePage = OperatingSystem.IsWindows() ? GetACP() : 65001;
        Console.WriteLine($"ANSI code page: {codePage}");
        Encoding.RegisterProvider(CodePagesEncodingProvider.Instance);
        Encoding encoding = Encoding.GetEncoding((int)codePage);
        char value = codePage == 65001 ? '\u00E9' : '\u3042';
        byte[] expected = encoding.GetBytes([value]);
        if (expected.Length <= 1)
        {
            Console.WriteLine("Requires UTF-8 or a multibyte Windows ANSI code page, such as CP932.");
            return 2;
        }

        var read = Marshal.GetDelegateForFunctionPointer<ReadArray>(
            (nint)(delegate* unmanaged[Cdecl]<byte*, int, int>)&SumBytes);
        try
        {
            int actual = read([value], expected.Length);
            int expectedSum = 0;
            foreach (byte b in expected)
            {
                expectedSum += b;
            }
            Console.WriteLine($"Expected encoded byte sum: {expectedSum}; actual: {actual}");
            Console.WriteLine(actual == expectedSum ? "PASS" : "BUG: incorrect ANSI array encoding.");
            return actual == expectedSum ? 0 : 1;
        }
        catch (ArgumentException ex)
        {
            Console.WriteLine($"BUG: one character requiring {expected.Length} bytes was rejected: {ex}");
            return 1;
        }
    }

    [UnmanagedCallersOnly(CallConvs = [typeof(CallConvCdecl)])]
    private static int SumBytes(byte* values, int count)
    {
        int sum = 0;
        for (int i = 0; i < count; i++)
        {
            sum += values[i];
        }
        return sum;
    }
}

app.manifest

<?xml version="1.0" encoding="utf-8"?>
<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1">
  <assemblyIdentity version="1.0.0.0" name="AnsiCharArray" />
  <application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings>
      <activeCodePage xmlns="http://schemas.microsoft.com/SMI/2019/WindowsSettings">UTF-8</activeCodePage>
    </windowsSettings>
  </application>
</assembly>

Build once with dotnet build Repro.csproj. Run the generated apphost against the two runtimes:

$env:DOTNET_ROLL_FORWARD_TO_PRERELEASE = '1'
$env:DOTNET_ROLL_FORWARD = 'LatestPatch'
& .\bin\Debug\net10.0\Repro.exe

$env:DOTNET_ROLL_FORWARD = 'LatestMajor'
& .\bin\Debug\net10.0\Repro.exe

These commands assume the apphost resolves to a .NET installation containing the tested .NET 10 and .NET 11 runtimes, with no newer major version. If needed, set DOTNET_ROOT to that installation. Check the framework version printed by the program rather than assuming which runtime was selected. Remove the temporary roll-forward environment variables after the comparison.

Expected behavior

The native callee receives the encoded bytes. For U+00E9 under UTF-8, the sum is 364 and the program prints PASS.

.NET 10 output:

.NET 10.0.5
ANSI code page: 65001
Expected encoded byte sum: 364; actual: 364
PASS

Actual behavior

.NET 11 output from the same compiled program:

.NET 11.0.0-rc.1.26420.103
ANSI code page: 65001
BUG: one character requiring 2 bytes was rejected: System.ArgumentException: Cannot marshal: Encountered unmappable character.
   at System.StubHelpers.AnsiCharArrayMarshaler`2.ConvertContentsToUnmanaged(Array managedArray, Byte* unmanaged, Int32 length)
   at System.StubHelpers.StubHelpers.ConvertArrayContentsToUnmanaged[T,TMarshaler](Array managed, Byte* pNative, Int32 numElements)
   at Program.Main() in Program.cs:line 36

Regression

Confirmed by running the same net10.0-targeted program on .NET 10.0.5 and .NET 11.0.0-rc.1.26420.103.

The exact introducing commit has not been established by a runtime build-and-bisect. The implementation observations below are based on source inspection.

Configuration

• Windows x64, CoreCLR, built-in runtime marshalling.
• Passing runtime: .NET 10.0.5.
• Failing runtime: .NET 11.0.0-rc.1.26420.103.
• Compiled with SDK 11.0.100-rc.1.26420.103, targeting net10.0.
• The repository runtime was not rebuilt for this comparison. Linux, ARM64, x86, Mono, NativeAOT, and source-generated marshalling were not tested.

Implementation observations

The new AnsiCharArrayMarshaler allocates managedArray.Length bytes and passes the character count as the destination byte capacity. The old converter allocated for multibyte expansion using GetMaxDBCSCharByteSize(). See allocation and conversion.

A separate control also reproduced rejection of U+00E9 with BestFitMapping=true, while ASCII succeeded with either best-fit setting. This is not an unmappable-character example: U+00E9 is representable in UTF-8.

Run the generated executable, not dotnet Repro.dll, so its UTF-8 application manifest is applied. Both recorded runs printed ANSI code page 65001.