Skip to content

Document that BpeOptions.ByteLevel requires a PreTokenizer - #7716

Merged
rosebyte merged 4 commits into
dotnet:mainfrom
lunosia:patch-1
Oct 13, 2026
Merged

Document that BpeOptions.ByteLevel requires a PreTokenizer#7716
rosebyte merged 4 commits into
dotnet:mainfrom
lunosia:patch-1

Commits

  1. Commits on Sep 14, 2026

  2. Commits on Sep 18, 2026

  3. Commits on Oct 12, 2026

  4. Commits on Oct 13, 2026