[nico-martin]

Summary

• load model-specific response_template definitions from tokenizer configuration
• parse completed single or batched responses from strings, token IDs, or tensors with tokenizer.parse_response()
• stream Python-compatible region_open, region_chunk, and region_close events with tokenizer.get_response_parser()
• support prompt-prefilled regions, typed and repeated fields, transforms, required fields, tool-schema coercion, and zero-width delimiters
• translate supported Python regex syntax to JavaScript once when a template loads and reject unsupported constructs clearly
• automatically parse chat outputs in the text-generation pipeline when the tokenizer has a response template
• alias reasoning_content, thinking, and reasoning inputs when rendering chat templates for backward compatibility
• keep TextStreamer as a raw text streamer so applications decide how to consume streaming parser events
• version browser and filesystem cache keys for tokenizer_config.json with the transformersjs_metadata_v2 epoch without invalidating tokenizer files or model weights

API

The streaming API follows Python Transformers directly:

• parser.feed(chunk) returns response events
• parser.initial_events contains events produced by prompt-prefilled regions
• parser.finalize() returns [message, finalEvents]
• tools can be passed to one-shot and streaming parsing to coerce tool arguments from their JSON schemas
• processor.parse_response() forwards one-shot parsing, while streaming uses processor.tokenizer.get_response_parser()
• chat pipelines parse final assistant messages automatically when tokenizer.response_template is set

Pipeline usage

For chat input, the text-generation pipeline returns a parsed assistant message automatically when the tokenizer has a response template:

const output = await generator(messages, {
  max_new_tokens: 512,
  do_sample: false,
});

const message = output[0].generated_text.at(-1);
// { role: "assistant", reasoning_content: "...", content: "..." }

Plain string pipeline inputs keep returning strings. TextStreamer also remains unparsed.

One-shot usage

Manual parsing is useful when calling model.generate() directly or when passing tools for schema-based argument coercion:

const message = tokenizer.parse_response(generatedText, {
  prefix: prompt,
  tools,
});

generatedText may also be one token-ID sequence or a 1D tensor. Batches of strings, token-ID sequences, or a 2D tensor return an array of messages. A single prefix is broadcast across a batch, or one prefix can be supplied per response.

Streaming usage

Streaming consumers handle events from the prompt, each newly decoded chunk, and finalization:

const parser = tokenizer.get_response_parser({
  prefix: prompt,
  tools,
});

function handleEvent(event) {
  if (event.type === "region_chunk" && !event.dirty) {
    renderText(event.field, event.text);
  } else if (event.type === "region_close") {
    renderValue(event.field, event.value);
  }
}

for (const event of parser.initial_events) {
  handleEvent(event);
}

const streamer = new TextStreamer(tokenizer, {
  skip_prompt: true,
  skip_special_tokens: false,
  callback_function(chunk) {
    for (const event of parser.feed(chunk)) {
      handleEvent(event);
    }
  },
});

await model.generate({ ...inputs, streamer, max_new_tokens: 512 });

const [message, finalEvents] = parser.finalize();
for (const event of finalEvents) {
  handleEvent(event);
}

dirty: false chunks are directly renderable text-like content. Structured regions such as JSON emit dirty: true raw chunks and provide their authoritative parsed value in region_close.

Notes

• The text-generation pipeline passes tools to parse_response() for tool-argument coercion. Python's pipeline doesn't; parse_response() itself accepts tools in both.
• With a response_template, the pipeline decodes with skip_special_tokens: false, including for plain-string inputs, same as Python (text_generation.py:443-445). Since many onnx-community repos now ship templates, plain-string outputs from those models will include special tokens. Worth a release-notes line.
• get_response_parser() matches upstream (tokenization_utils_base.py:3517).

ONNX response-template PRs

The corresponding onnx-community tokenizer configuration updates are tracked in:

• DeepSeek-R1-Distill-Qwen-1.5B-ONNX
• Josiefied-Qwen3.5-0.8B-gabliterated-v1-ONNX
• Qwen1.5-0.5B-Chat-ONNX
• Qwen1.5-0.5B-ONNX
• Qwen2-0.5B-Instruct-ONNX
• Qwen2-0.5B-ONNX
• Qwen2-VL-2B-Instruct
• Qwen2.5-0.5B
• Qwen2.5-0.5B-Instruct
• Qwen2.5-0.5B-Instruct-Gensyn-Swarm-mammalian_lithe_barracuda-ONNX
• Qwen2.5-0.5B-Instruct-ONNX
• Qwen2.5-0.5B-Instruct-ONNX-GQA
• Qwen2.5-0.5B-Instruct-ONNX-MHA
• Qwen2.5-0.5B-Instruct-abliterated-ONNX
• Qwen2.5-0.5B-Instruct-abliterated-v3-ONNX
• Qwen2.5-0.5B-ONNX
• Qwen2.5-1.5B
• Qwen2.5-1.5B-Instruct
• Qwen2.5-Coder-0.5B-Instruct
• Qwen2.5-Coder-0.5B-ONNX
• Qwen2.5-Coder-1.5B
• Qwen2.5-Coder-1.5B-Instruct
• Qwen2.5-Coder-3B-Instruct
• Qwen2.5-Math-1.5B
• Qwen2.5-Math-1.5B-Instruct
• Qwen2.5-VL-3B-Instruct-ONNX
• Qwen3-0.6B-DQ-ONNX
• Qwen3-0.6B-ONNX
• Qwen3-1.7B-ONNX
• Qwen3-4B-Instruct-2507-ONNX
• Qwen3-4B-ONNX
• Qwen3-4B-Thinking-2507-ONNX
• Qwen3-VL-2B-Instruct-ONNX
• Qwen3.5-0.8B-ONNX
• Qwen3.5-0.8B-ONNX-OPT
• Qwen3.5-0.8B-Text-ONNX
• Qwen3.5-2B-ONNX
• Qwen3.5-2B-ONNX-OPT
• Qwen3.5-4B-ONNX
• Qwen3.5-4B-ONNX-OPT
• gemma-4-E2B-it-ONNX
• gemma-4-E2B-it-qat-mobile-ONNX
• gemma-4-E4B-it-ONNX
• gemma-4-E4B-it-qat-mobile-ONNX
• gpt-oss-20b-ONNX
• Granite-3.3-8B-Instruct-Onnx
• granite-3.0-2b-instruct
• granite-4.0-1b-ONNX-web
• granite-4.0-350m-ONNX-web
• granite-4.0-h-1b-ONNX
• granite-4.0-h-350m-ONNX
• granite-4.0-h-micro-ONNX
• granite-4.0-micro-ONNX-web
• LFM2-1.2B-Extract-ONNX
• LFM2-1.2B-ONNX
• LFM2-1.2B-RAG-ONNX
• LFM2-1.2B-Tool-ONNX
• LFM2-2.6B-Exp-ONNX
• LFM2-2.6B-ONNX
• LFM2-24B-A2B-ONNX
• LFM2-350M-Extract-ONNX
• LFM2-350M-Math-ONNX
• LFM2-350M-ONNX
• LFM2-700M-ONNX
• LFM2-8B-A1B-ONNX
• LFM2-VL-1.6B-ONNX
• LFM2-VL-3B-ONNX
• LFM2-VL-450M-ONNX
• LFM2.5-350M-ONNX
• LFM2.5-VL-450M-ONNX

Cache invalidation

Cache keys for tokenizer_config.json use the transformersjs_metadata_v2 cache epoch: browser caches append ?transformersjs_metadata_v=2, while filesystem caches add a transformersjs_metadata_v2/ path prefix. This changes only the proposed cache key, not the Hub request URL or cache keys for tokenizer.json, model weights, and other resources.

This ensures browsers that cached tokenizer metadata before response_template was added can retrieve the updated configuration without invalidating large model assets.

Python Transformers instead relies on the huggingface_hub content-addressed cache. Mutable revisions such as main are resolved or revalidated against a repository commit, and files are cached under that immutable commit snapshot, so an updated tokenizer_config.json naturally produces a new cached snapshot.

Transformers.js uses an explicit metadata cache epoch because it commonly runs across many browser and client installations, where resolving revisions adds network requests and cache complexity. The epoch provides a one-time, targeted invalidation with no ongoing revalidation cost; it can be bumped again if a future release depends on newly added tokenizer metadata.

Python implementation

This implementation follows the declarative response_template format introduced in Python Transformers by #45847, Streamable chat parsing, merged on June 24, 2026. It replaced the regex-heavy response_schema prototype originally introduced by #40894 on October 21, 2025; the legacy format was subsequently removed by #47320.

Python stores the template on tokenizer.response_template in tokenizer_config.json. Applications can parse a completed generation with tokenizer.parse_response(output, prefix=prompt), while streaming applications create a stateful parser with tokenizer.get_response_parser(prefix=prompt). The parser emits region_open, region_chunk, and region_close events before finalize() returns the complete message and final events. Python pipelines also use the template automatically when one is available.

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
generated = outputs[0, inputs.input_ids.shape[-1]:]

message = tokenizer.parse_response(generated, prefix=inputs.input_ids[0])

For streaming, feed only each newly decoded chunk to one parser instance:

parser = tokenizer.get_response_parser(prefix=inputs.input_ids[0])

for event in parser.initial_events:
    handle_event(event)
for chunk in text_stream:
    for event in parser.feed(chunk):
        handle_event(event)

message, final_events = parser.finalize()
for event in final_events:
    handle_event(event)

Transformers.js follows the same model-side template format, required prompt-prefix semantics, event stream, finalization tuple, tool argument coercion, and automatic chat-pipeline parsing. JavaScript does not support callable tool definitions, and unsupported Python regex constructs are rejected during template loading rather than interpreted differently.

References:

• Python response parsing guide
• Python serving behavior
• Python parser implementation
• Python tokenizer integration

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[xenova]

in the process of reviewing this... but while we work on integration, I think we should work on updating tokenizer_config.json files upstream in https://huggingface.co/onnx-community
As we've done in the past before, maybe we can write a script to scan the HF hub for transformers.js-compatible models that support response_template, and we can open PRs to add the HF transformers/upstream template. This will also make integration easier when we eventually merge + put out a new release, since there will be more integrated models with this.

[xenova]

initial review done with help from opus 5.5 👍

See list of changes made

Changes (under packages/transformers/):

• New regex.js converts the templates' Python regex patterns to JavaScript once, when the template loads.
  • It keeps Python's behaviour: Unicode \w/\d/\b, \A/\Z, Python's $, (?P<…>, (?P=…), comments and inline flags.
  • Syntax JavaScript can't support (atomic groups, possessive quantifiers, verbose mode) raises a clear error instead of misbehaving.
  • Each pattern also gets a variant that matches "the start of a match that runs into the end of the text", which is how Python's partial=True works. Literal delimiters go through the same path.
• response_parser.js, response_templates.js, content_parsers.js are line-by-line ports with Python's names.
  • feed() returns events, initial_events and the dirty flag exist, and finalize() returns [message, events].
  • tools coercion and zero-width opening delimiters work as in Python.
  • Removed: the partial option, snapshots, freezing and the literal-prefix guessing.
• tokenization_utils.js: parse_response and get_response_parser take { prefix, response_template, tools }. response and prefix can be strings, token ids or Tensors, single or batched, with Python's broadcasting rules.
• text-generation.js: when the tokenizer has a response_template, output is decoded with skip_special_tokens: false and chat outputs are parsed using the prompt as prefix, as in Python. Without a template, behaviour is unchanged.
• Tests:
  • Python's test_chat_parsing.py is ported (66 tests). It includes the fixed and random chunking checks and a Qwen-template streaming test; the callable-tools test is skipped because JS has no callables.
  • Python's tokenizer-level tests and its pipeline <think>-prefix test are ported as well.
  • The tests that locked in the old behaviour are gone.
• docs-generate updated the tracked .ai/skills/transformers-js/references/TASKS.md, which needs to be committed with this change.

Verification:

• Against Python: on 31 cases (Python's 8 streaming fixtures plus edge cases), JS gives the same output and holds back the same text at every streaming step. The only differences are error class names (Error vs ValueError/KeyError).
• Partial matching: it agrees with Python's regex on 5996 of 6000 random cases. In the other 4, JS holds text back a little longer, which is the safe direction.
• Speed: streaming 80k characters takes 22–27 ms in JS vs 57–76 ms in Python. The inkling case that took 272 s now takes 4.8 ms.
• Test suite: passes.
• tsc, the type tests, prettier and the docs validator are all clean.

The biggest issue was a performance issue with a known inkling template/regex.

The whole stream is roughly cubic. Quadratic work per feed, times one feed per chunk, is why each doubling of the text costs about 7× more. At 20k characters it took 272 s. It now takes 4.8 ms.

== original PR (594b3df0) ==
2023 chars: 327 ms; feed() at char 1000 returned {"role":"assistant"}
4011 chars: 2726 ms; feed() at char 1000 returned {"role":"assistant"}
8015 chars: 18190 ms; feed() at char 1000 returned {"role":"assistant"}
== current ==
2023 chars: 2 ms; feed() at char 1000 returned [{"type":"region_chunk","field":"thinking","text":"lore","dirty":false}]
4011 chars: 1 ms; …
8015 chars: 2 ms; …

See slow snippet

import { ResponseParser } from './src/utils/chat_parsing/index.js';

// Inkling (TMLv0) response template, from transformers' tests/utils/test_chat_parsing.py.
// Both open patterns are regexes with no literal prefix: (?:<|message_model|>)?[^<]*<|content_KIND|>
const inkling = {
    defaults: { role: 'assistant' },
    start_anchor: '<|message_model|>',
    fields: {
        thinking: {
            open_pattern: String.raw`(?:<\|message_model\|>)?[^<]*<\|content_thinking\|>`,
            close: '<|end_message|>',
            repeats: true,
            join: '',
            content_args: { strip: false },
        },
        content: {
            open_pattern: String.raw`(?:<\|message_model\|>)?[^<]*<\|content_text\|>`,
            close: '<|end_message|>',
            repeats: true,
            join: '',
            content_args: { strip: false },
        },
    },
};

// A model that thinks for a while, streamed 4 characters at a time.
for (const length of [2000, 4000, 8000]) {
    const thinking = 'lorem ipsum dolor sit amet, '.repeat(length / 28);
    const text = `<|content_thinking|>${thinking}<|end_message|>`;
    const parser = new ResponseParser(inkling, { prefix: '' });
    const start = performance.now();
    let halfway;
    for (let i = 0; i < text.length; i += 4) {
        const result = parser.feed(text.slice(i, i + 4));
        if (i === 1000) halfway = result;
    }
    parser.finalize();
    console.log(`${text.length} chars: ${(performance.now() - start).toFixed(0)} ms; feed() at char 1000 returned`, JSON.stringify(halfway));
}

also:

• I removed get_response_parser since upstream doesn't have it
• fixed upstream bug: a list default like tool_calls: [] keeps growing across parse_response calls and mutates the caller's template. cc @Rocketknight1

---

follow-ups:

• maybe we could consider creating a new package @huggingface/regex (or similar) which can consolidate tokenizers.js regex parsing and also potentially this PR's functionality. Basically a new library to handle PY -> JS regex handling. could be useful for others too
• TODO: we should test our implementation with all response templates on the HF hub / transformers tests.

[Rocketknight1]

@xenova if you found an upstream bug in the transformers implementation, can you open a PR to transformers for it and ping me? We definitely don't want to keep appending to a shared tool calls dict like that!

[nico-martin]

I implemented response parsing for the protocol grammars below. Parser grammar support is not a capability declaration for every checkpoint in a model family. Model-side response_template metadata must match the exact checkpoint’s documented output format.

Checkpoint-specific audit — October 7, 2026

Audited all 70 linked Hub PRs against checkpoint model cards and chat/output protocols:

• 29 templates corrected, removing unverified/unsupported regions and correcting protocol delimiters.
• 21 templates retained, with checkpoint-specific descriptions.
• 20 PRs withdrawn, covering ordinary content-only protocols, pretrained base checkpoints, and fine-tunes whose proposed structured regions still need checkpoint-specific validation.

Important corrections:

• Qwen2.5 Instruct/Coder, Qwen3-4B-Instruct-2507, Qwen3-VL Instruct, and Granite 4: retain content and tool calls; remove the separate thinking parser.
• DeepSeek-R1 distill: thinking and content only; use its actual <｜Assistant｜> and <｜end▁of▁sentence｜> delimiters instead of Qwen ChatML anchors.
• Documented LFM tool-use checkpoints: content and Python-style tool calls. LFM2-2.6B retains hybrid thinking, as explicitly documented in its model card.
• LFM2-350M-Math: thinking and content only, using <|cot_start|> / <|cot_end|> rather than <think> tags.
• The Gensyn fine-tune, Qwen2.5-Math Instruct, and LFM Extract/RAG additions are withdrawn pending validation of their exact structured output protocols. This does not assert that they cannot reason in plain text or be prompted to use tools. Inherited template code alone is insufficient evidence.

Fields remain optional for checkpoints that support a region but do not emit it in every response. Ordinary chat without a response template continues to use the existing { role: "assistant", content: ... } fallback.

Verification: all 50 retained templates pass template validation; 505 total validation and synthetic one-shot/streaming checks cover the corrected templates, split delimiters, assistant prefills, and preservation of unsupported-marker text. All 70 remote PR statuses and all 50 retained remote configs were re-read and verified. No model weights were regenerated, and these checks do not establish tool-use quality.

The earlier protocol fixtures and generation tests remain useful for parser behavior, but they must not be read as end-to-end capability validation of every linked checkpoint.

Parser protocol coverage

The table describes protocol grammars. In particular, the generic Qwen3 row includes thinking-capable Qwen3 checkpoints; the Instruct-2507 and VL-Instruct checkpoints use tool/content-only templates.

Family / protocol	Parsed regions	Content parsers
Cohere Command	content, thinking, tool calls	text, json
GPT-OSS Harmony	content, thinking, tool calls	text, json
SmolLM	content, thinking, tool calls	text, json
Qwen2.5 Instruct (selected checkpoints)	content, tool calls	text, json
Qwen3	content, thinking, JSON or XML-style tool calls	text, json, xml-inline
Qwen3.5	content, thinking, tool calls	text, json
DeepSeek-R1 Qwen distill	content, thinking	text
Gemma 4	content, thinking, tool calls	text, lax json
Inkling / TMLv0	content, thinking, tool calls	text, json
Granite 3.0	content, bare JSON tool calls	text, json
Granite 3.3	content, thinking, JSON-list tool calls	text, json
Granite 4	content, repeated JSON tool calls	text, json
LFM2 / LFM2.5 (documented tool-use checkpoints)	content, Python-style tool calls	text, python-call-list
LFM2-2.6B	content, thinking, Python-style tool calls	text, python-call-list
LFM2-350M-Math	content, thinking (<|cot_start|> / <|cot_end|>)	text
ERNIE structured protocol	content, thinking, tool calls	text, json

The python-call-list parser is a non-evaluating parser added for LFM's [function_name(arg=value)] output grammar. It is currently a Transformers.js extension and is not yet supported by Python Transformers.

onnx-community PRs

Corrected — 29

• DeepSeek-R1-Distill-Qwen-1.5B-ONNX
• LFM2-1.2B-ONNX
• LFM2-1.2B-Tool-ONNX
• LFM2-2.6B-Exp-ONNX
• LFM2-24B-A2B-ONNX
• LFM2-350M-Math-ONNX
• LFM2-350M-ONNX
• LFM2-700M-ONNX
• LFM2-8B-A1B-ONNX
• LFM2.5-350M-ONNX
• LFM2.5-VL-450M-ONNX
• Qwen2.5-0.5B-Instruct-ONNX-GQA
• Qwen2.5-0.5B-Instruct-ONNX-MHA
• Qwen2.5-0.5B-Instruct-ONNX
• Qwen2.5-0.5B-Instruct-abliterated-ONNX
• Qwen2.5-0.5B-Instruct-abliterated-v3-ONNX
• Qwen2.5-0.5B-Instruct
• Qwen2.5-1.5B-Instruct
• Qwen2.5-Coder-0.5B-Instruct
• Qwen2.5-Coder-1.5B-Instruct
• Qwen2.5-Coder-3B-Instruct
• Qwen3-4B-Instruct-2507-ONNX
• Qwen3-VL-2B-Instruct-ONNX
• granite-4.0-1b-ONNX-web
• granite-4.0-350m-ONNX-web
• granite-4.0-h-1b-ONNX
• granite-4.0-h-350m-ONNX
• granite-4.0-h-micro-ONNX
• granite-4.0-micro-ONNX-web

Retained — 21

• Granite-3.3-8B-Instruct-Onnx
• Josiefied-Qwen3.5-0.8B-gabliterated-v1-ONNX
• LFM2-2.6B-ONNX
• Qwen3-0.6B-DQ-ONNX
• Qwen3-0.6B-ONNX
• Qwen3-1.7B-ONNX
• Qwen3-4B-ONNX
• Qwen3-4B-Thinking-2507-ONNX
• Qwen3.5-0.8B-ONNX-OPT
• Qwen3.5-0.8B-ONNX
• Qwen3.5-0.8B-Text-ONNX
• Qwen3.5-2B-ONNX-OPT
• Qwen3.5-2B-ONNX
• Qwen3.5-4B-ONNX-OPT
• Qwen3.5-4B-ONNX
• gemma-4-E2B-it-ONNX
• gemma-4-E2B-it-qat-mobile-ONNX
• gemma-4-E4B-it-ONNX
• gemma-4-E4B-it-qat-mobile-ONNX
• gpt-oss-20b-ONNX
• granite-3.0-2b-instruct

Withdrawn — 20

• LFM2-1.2B-Extract-ONNX
• LFM2-1.2B-RAG-ONNX
• LFM2-350M-Extract-ONNX
• LFM2-VL-1.6B-ONNX
• LFM2-VL-3B-ONNX
• LFM2-VL-450M-ONNX
• Qwen1.5-0.5B-Chat-ONNX
• Qwen1.5-0.5B-ONNX
• Qwen2-0.5B-Instruct-ONNX
• Qwen2-0.5B-ONNX
• Qwen2-VL-2B-Instruct
• Qwen2.5-0.5B-Instruct-Gensyn-Swarm-mammalian_lithe_barracuda-ONNX
• Qwen2.5-0.5B-ONNX
• Qwen2.5-0.5B
• Qwen2.5-1.5B
• Qwen2.5-Coder-0.5B-ONNX
• Qwen2.5-Coder-1.5B
• Qwen2.5-Math-1.5B-Instruct
• Qwen2.5-Math-1.5B
• Qwen2.5-VL-3B-Instruct-ONNX

[Rocketknight1]

Made a PR to transformers to fix this at huggingface/transformers#49370!

[xenova]

hmm... this feels really hacky 😅

If we do have to support this, something we could do is add a cache breadcrumb which essentially refreshes already-cached tokenizer_config.json files exactly once.
e.g., I'd be happier to avoid changing the query parameter, and rather either setting the value in the config itself under some key before we cache (meaning we can detect if it's an older version, or is missing the response template parameter)

[xenova]

maybe a better solution is a rework of the caching mechanism which essentially bundles the pre-flight checks we do with an API check for files with their hashes, which will help us keep things up to date.

Another, weaker proxy, could be file size differences (only would work for JSON cases like this where we materially change the size of the config file)

[nico-martin]

I see your concerns, but imo versioned cache keys aren't really a hack. They are a well-established pattern to ensure users get the latest version of an asset without changing the filename itself.

Adding a breadcrumb to the config feels more complex since we'd be modifying the cached content and adding special detection/refetch logic. We also can't rely on a missing response_template, because many valid configs won't have one. File size seems too fragile as well.

The hash-based preflight idea could be a good general improvement, but it’s a much bigger caching rework and would add extra requests across clients. For this case, I think a targeted cache version is the simplest solution, and we can bump it again if we ever need to invalidate tokenizer metadata in the future.

[nico-martin]

We could even take this further and establish cache versions for different file types, or potentially any cached asset. That would give us a consistent way to invalidate a specific class of files when needed, without clearing unrelated model files or changing their actual filenames.

[xenova]

Sure, a more general solution instead of an if (name == 'tokenizer_config.json') statement is better.
we could have a registry, but then we'll eventually accumulate a long list of old tokenizer configs/models/... in the cache that are never purged (unless we add some mechanism to purge all previous versions... but that would need to be run each request).

[nico-martin]

That’s fair. A registry might be the cleaner general solution, but it also feels like a broader cache architecture change for a PR focused on response_template. For this PR, I’d prefer keeping the targeted cache-key versioning and opening a follow-up to design a general versioning and cleanup mechanism properly, rather than introducing a partial registry now.

[nico-martin]

Python Transformers is standardizing parsed reasoning messages on reasoning_content instead of thinking. To keep Transformers.js and the ONNX model metadata aligned with that convention, I've updated our examples and model configs to use reasoning_content consistently too.

The parser itself is still generic and returns whatever fields a model defines in its response_template. We're only standardizing the templates and examples here, not adding any hardcoded alias. The actual output formats, such as <think>...</think>, Gemma's thought channel, and GPT-OSS's analysis channel, stay exactly the same.

I also made a few related changes in the Transformers.js branch:

• uses reasoning_content in public JSDoc and response-parsing fixtures
• accepts parsed assistant messages without content, including reasoning-only and tool-call-only history entries
• passes tools through automatic pipeline parsing for schema-based argument coercion
• includes one-shot, streaming, pipeline, and parse-to-chat-template round-trip coverage

I opened follow-up PRs for all 22 merged ONNX configs that were returning thinking:

DeepSeek

• DeepSeek-R1-Distill-Qwen-1.5B-ONNX #15

Qwen

• Josiefied-Qwen3.5-0.8B-gabliterated-v1-ONNX #2
• Qwen3-0.6B-DQ-ONNX #2
• Qwen3-0.6B-ONNX #5
• Qwen3-1.7B-ONNX #5
• Qwen3-4B-ONNX #5
• Qwen3-4B-Thinking-2507-ONNX #3
• Qwen3.5-0.8B-ONNX #8
• Qwen3.5-0.8B-ONNX-OPT #2
• Qwen3.5-0.8B-Text-ONNX #4
• Qwen3.5-2B-ONNX #2
• Qwen3.5-2B-ONNX-OPT #2
• Qwen3.5-4B-ONNX #2
• Qwen3.5-4B-ONNX-OPT #2

Gemma 4

• gemma-4-E2B-it-ONNX #6
• gemma-4-E2B-it-qat-mobile-ONNX #4
• gemma-4-E4B-it-ONNX #5
• gemma-4-E4B-it-qat-mobile-ONNX #2

These also remove the old response_schema. The Gemma chat templates already understand reasoning_content during an active tool-call flow, so this fixes the mismatch there.

Other protocols

• gpt-oss-20b-ONNX #3
• Granite-3.3-8B-Instruct-Onnx #2
• LFM2-2.6B-ONNX #2
• LFM2-350M-Math-ONNX #3

GPT-OSS now prefers reasoning_content when rendering assistant history, but still accepts thinking as a fallback for existing callers. I left models that intentionally drop reasoning from previous turns alone.

[nico-martin]

this PR adds two things that Python chat_parsing doesn't have yet: a python-call-list content parser (LFM2 tool calls) and strip_prefix/strip_suffix text args (Granite 3.3's <response> wrapper). I've proposed both upstream in huggingface/transformers#49918.

I don't want to block this PR on it. If upstream ends up doing it differently, I'll align the JS side and the Hub templates.