Fix UTF-8 surrogate pair encoding

[janrysavy]

Summary

This is a small reference PR for completeness from the same review pass as #67 and #68.

I understand that you have been preparing your own upstream patches for the reported issues, so this PR is intended mainly as a concrete reference: it shows the reproduced behavior, the minimal affected code location, and the validation result.

This PR fixes the UTF-16 to UTF-8 conversion path for non-BMP diagnostic text.

Problem

ConvertUTF16toUTF8 correctly recognizes UTF-16 surrogate pairs and computes the Unicode code point:

LCode := ((LCode and $3ff) shl 10) + ((LCode shr 16) and $3ff) + $10000;

However, the leading byte of the four-byte UTF-8 sequence used $e0:

Result[0] := Byte((LCode shr 18) and $07) or $e0;

Four-byte UTF-8 sequences must start with 11110xxx, so the leading mask should be $f0. With the current code, non-BMP text such as U+1F600 is written as invalid UTF-8 bytes:

E0 9F 98 80

The valid UTF-8 sequence is:

F0 9F 98 80

This affects diagnostic text written through UTF-8 text-file output, for example FastMM_LogStateToFile additional details or other diagnostic messages containing non-BMP characters.

Fix

Use $f0 for the leading byte of a four-byte UTF-8 sequence:

Result[0] := Byte((LCode shr 18) and $07) or $f0;

The change is intentionally minimal and only affects UTF-8 encoding of UTF-16 surrogate pairs.

Reproducer

I kept the patch minimal and did not add a new test directory because the repository does not currently appear to have one. I can add this reproducer wherever you prefer.

Standalone reproducer:

program Utf8NonBmpDiagnosticsRegression;

{$APPTYPE CONSOLE}

uses
  FastMM5,
  System.Classes,
  System.SysUtils;

function ContainsByteSequence(const ABytes: TBytes; const ASequence: array of Byte): Boolean;
var
  LIndex: Integer;
  LSequenceIndex: Integer;
  LMatches: Boolean;
begin
  Result := False;

  {The regression check works on raw bytes. We do not decode the log file through
  Delphi string routines, because the original bug writes malformed UTF-8 and a
  decoder could hide the exact byte sequence that we need to verify.}
  if (Length(ASequence) = 0) or (Length(ABytes) < Length(ASequence)) then
    Exit;

  for LIndex := 0 to Length(ABytes) - Length(ASequence) do
  begin
    LMatches := True;
    for LSequenceIndex := 0 to Length(ASequence) - 1 do
    begin
      if ABytes[LIndex + LSequenceIndex] <> ASequence[LSequenceIndex] then
      begin
        LMatches := False;
        Break;
      end;
    end;

    if LMatches then
      Exit(True);
  end;
end;

function ReadFileBytes(const AFileName: string): TBytes;
var
  LFileStream: TFileStream;
  LSize: Integer;
begin
  LFileStream := TFileStream.Create(AFileName, fmOpenRead or fmShareDenyNone);
  try
    LSize := LFileStream.Size;
    SetLength(Result, LSize);
    if LSize > 0 then
      LFileStream.ReadBuffer(Result[0], LSize);
  finally
    LFileStream.Free;
  end;
end;

procedure CheckNonBmpTextIsWrittenAsValidUtf8;
var
  LLogFileName: string;
  LAdditionalDetails: string;
  LLogBytes: TBytes;
begin
  {Write the test log next to the executable so both Win32 and Win64 builds can
  be run from their artifact directories without sharing one output file.}
  LLogFileName := IncludeTrailingPathDelimiter(ExtractFilePath(ParamStr(0)))
    + 'utf8-non-bmp-diagnostics.log';

  {Step 1: Start from a clean log file. The test inspects raw file bytes, so old
  content would make a false pass or false failure possible.}
  if FileExists(LLogFileName) and not DeleteFile(LLogFileName) then
    raise Exception.CreateFmt('Could not delete old log file: %s', [LLogFileName]);

  {Step 2: Use a non-BMP code point written as an explicit UTF-16 surrogate pair.
  The selected code point is U+1F600, whose valid UTF-8 encoding is:

    F0 9F 98 80

  With the bug present, FastMM's surrogate-pair branch emits:

    E0 9F 98 80}
  LAdditionalDetails := 'Non-BMP diagnostic marker: ' + WideChar($D83D) + WideChar($DE00) + sLineBreak;

  {Step 3: Force the public text-file logging path to use FastMM's UTF-8 encoder.
  This exercises ConvertUTF16toUTF8 indirectly instead of calling private
  implementation details from the reproducer.}
  FastMM_TextFileEncoding := teUTF8;

  if not FastMM_LogStateToFile(PWideChar(LLogFileName), PWideChar(LAdditionalDetails), True) then
    raise Exception.Create('FastMM_LogStateToFile failed');

  {Step 4: Read the generated log as bytes and verify the exact UTF-8 sequence
  written for the non-BMP marker.}
  LLogBytes := ReadFileBytes(LLogFileName);

  if ContainsByteSequence(LLogBytes, [$F0, $9F, $98, $80]) then
    Writeln('OK: non-BMP diagnostic text was written as valid UTF-8.')
  else if ContainsByteSequence(LLogBytes, [$E0, $9F, $98, $80]) then
    raise Exception.Create('BUG: non-BMP diagnostic text used E0 as the first byte of a four-byte UTF-8 sequence.')
  else
    raise Exception.Create('The non-BMP marker was not found in the generated UTF-8 log file.');
end;

begin
  try
    CheckNonBmpTextIsWrittenAsValidUtf8;
  except
    on E: Exception do
    begin
      Writeln(E.ClassName, ': ', E.Message);
      Halt(1);
    end;
  end;
end.

Validation

Tested locally against upstream 1dad84b with RAD Studio 37.0:

• Win64 dcc64: reproduced the invalid E0 9F 98 80 sequence before the fix; build passed and reproducer passed after the fix
• Win32 DCC32: build passed and reproducer passed after the fix

Expected successful output:

OK: non-BMP diagnostic text was written as valid UTF-8.

Risk

Low. The change only affects UTF-8 output for UTF-16 surrogate pairs. BMP characters, ASCII text, UTF-16LE output, and allocator behavior are unchanged.

[pleriche]

Thanks Jan, I actually just pushed a fix for this in response to the ChatGPT report you linked in your previous report. I am currently busy adding locking to the event log handling.