Skip to content

Document that BpeOptions.ByteLevel requires a PreTokenizer - #7716

Merged
rosebyte merged 4 commits into
dotnet:mainfrom
lunosia:patch-1
Oct 13, 2026
Merged

Document that BpeOptions.ByteLevel requires a PreTokenizer#7716
rosebyte merged 4 commits into
dotnet:mainfrom
lunosia:patch-1

Merge branch 'main' into patch-1

bc05887
Select commit
Loading
Failed to load commit list.
Build Analysis / Build Analysis succeeded Oct 13, 2026 in 0s

.NET Result Analysis

Details

All checks completed

✅ [MachineLearning-CodeCoverage]

✅ [MachineLearning-CI]

Was this helpful? Yes No

View more details on Build Analysis