Skip to content

Remove unsafe code from CborHelpers, Crc32, and BlobUtilities#126269

Closed
EgorBo wants to merge 7 commits into
mainfrom
unsafe-code-removal
Closed

Remove unsafe code from CborHelpers, Crc32, and BlobUtilities#126269
EgorBo wants to merge 7 commits into
mainfrom
unsafe-code-removal

Conversation

@EgorBo

@EgorBo EgorBo commented Mar 29, 2026

Copy link
Copy Markdown
Member

Note

This PR was generated with the assistance of GitHub Copilot.

Summary

Replace Unsafe.* and MemoryMarshal.* API usage with safe BinaryPrimitives equivalents in three libraries:

System.Formats.Cbor — CborHelpers.netstandard.cs

  • ReadHalfBigEndian: MemoryMarshal.Read + manual endian swap → BinaryPrimitives.ReadUInt16BigEndian
  • WriteHalfBigEndian: MemoryMarshal.Write + manual endian swap → BinaryPrimitives.WriteUInt16BigEndian
  • ReadDoubleBigEndian: MemoryMarshal.Read + ReverseEndianness → BinaryPrimitives.ReadInt64BigEndian
  • WriteDoubleBigEndian: MemoryMarshal.Write + ReverseEndianness → BinaryPrimitives.WriteInt64BigEndian

System.IO.Hashing — Crc32ParameterSet.WellKnown.cs

  • UpdateScalarArm64, UpdateScalarArm: Unsafe.ReadUnaligned → BinaryPrimitives.ReadUInt64/32LittleEndian
  • UpdateIntrinsic (SSE path): MemoryMarshal.Cast<byte, ulong/uint> → explicit for-loops with BinaryPrimitives
  • UpdateIntrinsic (ARM path): Unsafe.ReadUnaligned → BinaryPrimitives.ReadUInt64/32LittleEndian
  • Removed MemoryMarshal.GetReference + Unsafe.Add patterns
  • Simplified remainder-byte loops

System.Reflection.Metadata — BlobUtilities.cs

  • WriteUInt16/32/64 and BE variants: Unsafe.WriteUnaligned → BinaryPrimitives.Write*LittleEndian/BigEndian
  • WriteDouble (#else branch): pointer cast *(ulong*)&valueBitConverter.DoubleToInt64Bits

All changes are behavioral equivalents — no public API changes. All existing tests pass.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-reflection-metadata
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR replaces several Unsafe.* / MemoryMarshal.*-based unaligned read/write patterns with System.Buffers.Binary.BinaryPrimitives (and BitConverter bit-conversion helpers) to keep the implementations safe while preserving behavior.

Changes:

  • System.Reflection.Metadata: replace unaligned integer writes with BinaryPrimitives.Write*Endian and remove unsafe double bit reinterpretation in WriteDouble (non-NET path).
  • System.IO.Hashing: replace Unsafe.ReadUnaligned / MemoryMarshal.Cast patterns in CRC32(C) intrinsic/scalar paths with BinaryPrimitives.Read*LittleEndian.
  • System.Formats.Cbor: simplify big-endian half/double read/write helpers using BinaryPrimitives.Read/Write*BigEndian.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/libraries/System.Reflection.Metadata/src/System/Reflection/Internal/Utilities/BlobUtilities.cs Switches unaligned integer writes to BinaryPrimitives and uses BitConverter.DoubleToInt64Bits for safe double bit extraction in non-NET builds.
src/libraries/System.IO.Hashing/src/System/IO/Hashing/Crc32ParameterSet.WellKnown.cs Removes Unsafe/MemoryMarshal usage in CRC32(C) update loops, using BinaryPrimitives reads instead.
src/libraries/System.Formats.Cbor/src/System/Formats/Cbor/CborHelpers.netstandard.cs Replaces endian-swap + MemoryMarshal patterns for half/double big-endian helpers with BinaryPrimitives calls.

@github-actions

This comment has been minimized.

@EgorBo EgorBo marked this pull request as draft March 29, 2026 00:21
- Revert Crc32ParameterSet.WellKnown.cs changes (bounds check concern)
- BlobUtilities: use BitConverter.DoubleToInt64Bits on all TFMs
- CborHelpers: remove ReadDoubleBigEndian/WriteDoubleBigEndian wrappers,
  inline BinaryPrimitives calls at call sites with #if NET polyfill.
  Remove WriteHalfBigEndian wrapper, call BinaryPrimitives directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 29, 2026 13:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.Formats.Cbor/src/System/Formats/Cbor/CborHelpers.netstandard.cs:104

  • PR description indicates CborHelpers replaces Unsafe.* and MemoryMarshal.* usage with BinaryPrimitives equivalents, but CborHelpers.netstandard.cs still uses MemoryMarshal.Read/Write (e.g., ReadSingleBigEndian/WriteSingleBigEndian) and contains unsafe pointer casts for float bit conversions. If the PR goal is to fully remove these APIs/unsafe code from CborHelpers, consider updating the remaining helpers (where possible for the target TFMs) or clarifying the scope in the PR description.
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static ushort ReadHalfBigEndian(ReadOnlySpan<byte> source)
        {
            return BinaryPrimitives.ReadUInt16BigEndian(source);
        }

        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        public static float ReadSingleBigEndian(ReadOnlySpan<byte> source)
        {
            return BitConverter.IsLittleEndian ?
                Int32BitsToSingle(BinaryPrimitives.ReverseEndianness(MemoryMarshal.Read<int>(source))) :
                MemoryMarshal.Read<float>(source);
        }

@github-actions

This comment has been minimized.

@@ -135,28 +117,6 @@ public static void WriteSingleBigEndian(Span<byte> destination, float value)
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
extension(BinaryPrimitives)
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static double ReadDoubleBigEndian(ReadOnlySpan<byte> source)
{
return BitConverter.Int64BitsToDouble(BinaryPrimitives.ReadInt64BigEndian(source);
}
}

Can we convert all polyfills in this type to extension methods like this, so that the actual source can use all modern APIs directly without unnecessary wrappers?

EnsureWriteCapacity(1 + sizeof(double));
WriteInitialByte(new CborInitialByte(CborMajorType.Simple, CborAdditionalInfo.Additional64BitData));
CborHelpers.WriteDoubleBigEndian(_buffer.AsSpan(_offset), value);
BinaryPrimitives.WriteInt64BigEndian(_buffer.AsSpan(_offset), BitConverter.DoubleToInt64Bits(value));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BinaryPrimitives.WriteInt64BigEndian(_buffer.AsSpan(_offset), BitConverter.DoubleToInt64Bits(value));
BinaryPrimitives.WriteDoubleBigEndian(_buffer.AsSpan(_offset), value);

Address jkotas feedback: convert CborHelpers Single/Double wrappers
to extension(BinaryPrimitives) polyfills so callers can use
BinaryPrimitives.Read/WriteSingle/DoubleBigEndian directly on all TFMs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@@ -27,21 +27,5 @@ public static string BuildStringFromIndefiniteLengthTextString<TState>(int lengt
[MethodImpl(MethodImplOptions.AggressiveInlining)]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we go further and delete this file completely?

Comment on lines +130 to +132
return BitConverter.IsLittleEndian ?
CborHelpers.Int32BitsToSingle(BinaryPrimitives.ReverseEndianness(MemoryMarshal.Read<int>(source))) :
MemoryMarshal.Read<float>(source);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return BitConverter.IsLittleEndian ?
CborHelpers.Int32BitsToSingle(BinaryPrimitives.ReverseEndianness(MemoryMarshal.Read<int>(source))) :
MemoryMarshal.Read<float>(source);
return CborHelpers.Int32BitsToSingle(BinaryPrimitives.ReadInt32BigEndian(source));

public static void WriteSingleBigEndian(Span<byte> destination, float value)
{
MemoryMarshal.Write(destination, ref value);
if (BitConverter.IsLittleEndian)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

@jkotas jkotas Mar 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
BinaryPrimitives.WriteSingleLittleEndian(buffer.AsSpan(start), value);

MemoryMarshal.Write(destination, ref value);
}
}
internal static uint SingleToUInt32Bits(float value)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inline the unsafe casts into the BinaryPrimitives extensions directly?

@github-actions

This comment has been minimized.

…ves directly

- CborHelpers: simplify ReadSingleBigEndian/WriteSingleBigEndian extensions
  to use ReadInt32BigEndian/WriteInt32BigEndian with inlined unsafe casts,
  removing MemoryMarshal.Read/Write and ReverseEndianness
- BlobUtilities: use BinaryPrimitives.WriteSingle/DoubleLittleEndian on NET,
  keep existing fallback on netstandard2.0/net462

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 29, 2026 15:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

@github-actions

This comment has been minimized.

Move all TFM-specific CborHelpers methods into one file with #if NET
guards. Delete the netcoreapp-only partial class file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
namespace System.Formats.Cbor
{
internal static partial class CborHelpers
internal static class CborHelpers

@jkotas jkotas Mar 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not mean to add a bunch of ifdefs to the common file.

I meant to:

  • Delete CborHelpers.netcoreapp.cs
  • Use the netcoreapp APIs directly throughout the code
  • Add extensions to CborHelpers.netstandard.cs to polyfill the netcoreapp APIs

@EgorBo EgorBo Mar 29, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I did have that in mind, in my workflow I just ask AI to address the feedback and then change it. It sometimes proactively pushes it before I change it, but I assumed it's fine since the PR is in draft 🙂 Still, I appreciate the early feedback!

@github-actions

This comment has been minimized.

Copilot AI review requested due to automatic review settings March 29, 2026 16:28
…n APIs on NET

- Add extension(BitConverter) polyfills for SingleToInt32Bits,
  Int32BitsToSingle, SingleToUInt32Bits, UInt32BitsToSingle on netstandard
- Update HalfHelpers and CborWriter.Simple to use BitConverter.* directly
- Keep CborHelpers wrappers for UnixEpoch, BigInteger, decimal.GetBits,
  string.Create (different signatures per TFM, not pure polyfills)
- Guard netstandard-only polyfills with #if !NET, add NET versions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

src/libraries/System.Formats.Cbor/src/System/Formats/Cbor/CborHelpers.cs:11

  • using System.Buffers; and using System.Runtime.InteropServices; are not used anywhere in this file, which will produce unused-using warnings (treated as errors in this repo). Please remove the unused using directives.
    src/libraries/System.Formats.Cbor/src/System/Formats/Cbor/CborHelpers.cs:179
  • The BinaryPrimitivesPolyfills / BitConverterPolyfills blocks introduce new unsafe code and rely on extension(Type) syntax. This undermines the PR goal of removing unsafe code, and the extension-member syntax is not used elsewhere in the repo (making it risky/unclear for multi-target builds). Since BinaryPrimitives.Read/WriteSingle/Double*Endian already exist in System.Memory (used in netstandard builds), please remove these polyfills and call the existing BinaryPrimitives APIs directly; for bit conversions, keep the previous internal helper methods instead of extending BitConverter.

Comment on lines 39 to 41
result = HalfHelpers.FloatToHalf(value);
return float.IsNaN(value) || CborHelpers.SingleToInt32Bits(HalfHelpers.HalfToFloat(result)) == CborHelpers.SingleToInt32Bits(value);
return float.IsNaN(value) || BitConverter.SingleToInt32Bits(HalfHelpers.HalfToFloat(result)) == BitConverter.SingleToInt32Bits(value);
}

Copilot AI Mar 29, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On non-.NETCoreApp targets (netstandard2.0 / .NET Framework), BitConverter.SingleToInt32Bits is not available (the test helper for netstandard uses an unsafe bitcast for this reason). This change will break compilation unless you provide a valid polyfill; consider restoring the previous internal helper (e.g., CborHelpers.SingleToInt32Bits) or comparing bit patterns via BinaryPrimitives-based conversions instead.

Copilot uses AI. Check for mistakes.
Comment on lines 56 to 66
@@ -62,7 +62,7 @@ public static float HalfToFloat(ushort value)
return CreateSingle(sign, (byte)(exp + 0x70), sig << 13);

static float CreateSingle(bool sign, byte exp, uint sig)
=> CborHelpers.Int32BitsToSingle((int)(((sign ? 1U : 0U) << FloatSignShift) + ((uint)exp << FloatExponentShift) + sig));
=> BitConverter.Int32BitsToSingle((int)(((sign ? 1U : 0U) << FloatSignShift) + ((uint)exp << FloatExponentShift) + sig));
}

Copilot AI Mar 29, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BitConverter.UInt32BitsToSingle, BitConverter.Int32BitsToSingle, and BitConverter.SingleToUInt32Bits are not available on the netstandard2.0 / .NET Framework TFMs that this file targets, so these replacements will fail to compile without a robust polyfill. Please keep using the existing internal conversion helpers (previously on CborHelpers) or implement local bit-conversion helpers for these TFMs.

Copilot uses AI. Check for mistakes.
@github-actions

Copy link
Copy Markdown
Contributor

🤖 Copilot Code Review — PR #126269

Note

This review was generated by GitHub Copilot and has not been manually verified.

Holistic Assessment

Motivation: Well-justified. Replacing manual Unsafe.WriteUnaligned / MemoryMarshal.Read/Write + endian-swap patterns with BinaryPrimitives calls is the right direction. The reduce-unsafe label is appropriate. The Crc32 revert after stephentoub/jkotas feedback about bounds-check overhead was a good call.

Approach: Clean. The C# 14 extension(BinaryPrimitives) / extension(BitConverter) pattern for netstandard polyfills is exactly what jkotas requested — callers use the modern APIs directly (BinaryPrimitives.ReadDoubleBigEndian, BitConverter.SingleToInt32Bits) and the polyfills transparently fill the gap for older TFMs. Merging the two partial class files into a single CborHelpers.cs with #if NET / #else blocks is a net simplification.

Summary: ✅ LGTM. The changes are semantically equivalent, correctly constrained to internal helpers, and address all maintainer review feedback. No blocking issues found. Two minor observations below.


Detailed Findings

✅ Correctness — All BinaryPrimitives replacements are semantically equivalent

Verified each transformation:

  • CborHelpers: MemoryMarshal.Read<ushort> + ReverseEndiannessBinaryPrimitives.ReadUInt16BigEndian — equivalent. All four Read/Write Half/Single/Double helpers are correct.
  • BlobUtilities: Unsafe.WriteUnaligned(ref buffer[start], endian-swapped-value)BinaryPrimitives.Write*LittleEndian/BigEndian(buffer.AsSpan(start), value) — equivalent and strictly safer (the old Unsafe.WriteUnaligned could silently write past array bounds if start was near the end).
  • BlobUtilities WriteDouble #else: *(ulong*)&valueunchecked((ulong)BitConverter.DoubleToInt64Bits(value)) — bit-preserving. DoubleToInt64Bits is available on netstandard2.0.
  • HalfHelpers.netstandard.cs: CborHelpers.UInt32BitsToSingle(...)BitConverter.UInt32BitsToSingle(...) — resolves to the polyfill extension; correct.
  • CborWriter.Simple.netstandard.cs: CborHelpers.SingleToInt32Bits(...)BitConverter.SingleToInt32Bits(...) — resolves to the polyfill extension; correct.

✅ Extension polyfill design — Correct and consistent with maintainer guidance

The BinaryPrimitivesPolyfills and BitConverterPolyfills classes using extension(BinaryPrimitives) / extension(BitConverter) are properly guarded by #if !NET, so they only compile for netstandard2.0 / .NET Framework where the real APIs are missing. On modern .NET, the actual BCL methods are used directly with no wrapper overhead.

The remaining unsafe pointer casts in the polyfills (*(float*)&intValue, *(int*)&value) are necessary for netstandard2.0 where BitConverter.SingleToInt32Bits / Int32BitsToSingle don't exist. This is the right tradeoff — unsafe code is isolated to polyfills that only compile on older TFMs.

✅ File consolidation — CborHelpers merge is correct

Deleting CborHelpers.netcoreapp.cs and merging into CborHelpers.cs is clean. The .csproj correctly moves the <Compile> include from the TFM-conditional ItemGroup to the unconditional one. The class is no longer partial since there's only one file, which is correct.

✅ No public API surface changes

All changed types are internal. No ref/ assembly changes. No new public members.

💡 Unused using directives — Minor cleanup

using System.Runtime.InteropServices; appears unused in both CborHelpers.cs (line 11) and BlobUtilities.cs (line 8) now that all MemoryMarshal / Unsafe.WriteUnaligned calls have been removed. CI/analyzers should catch this, but noting it in case they don't.

Generated by Code Review for issue #126269 ·

@EgorBo

EgorBo commented Mar 29, 2026

Copy link
Copy Markdown
Member Author

I'll split this PR into smaller ones since it now does more than I initially wanted (to remove unsafe) also it is getting harder to review. Also, I should probably rely less on AI on addressing feedback 😥

@EgorBo EgorBo closed this Mar 29, 2026
EgorBo added a commit that referenced this pull request Mar 30, 2026
> [!NOTE]
> This PR description was AI/Copilot-generated.

Extracted from #126269

I wonder if we should move all Polyfils for NS/NETF to just one common
place for all libraries. Currently, libs just define their own in-place.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 29, 2026
@jkotas jkotas deleted the unsafe-code-removal branch May 22, 2026 17:01
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants