Repository navigation
FEAT: keep reasoning text from Chat Completions responses - #4267
Open
CHERIF ANES ABD EL MALEK (anascherif) wants to merge 3 commits into
Open
CHERIF ANES ABD EL MALEK (anascherif) wants to merge 3 commits into
CHERIF ANES ABD EL MALEK (anascherif) wants to merge 3 commits into
Conversation
Reasoning models served over Chat Completions return their chain of thought in a message field next to the answer: reasoning_content (DeepSeek, LiteLLM, xAI, Qwen) or reasoning (OpenRouter, Ollama, vLLM). The shared parser read neither, so OpenAIChatTarget and LiteLLMChatTarget dropped it. Store it in the response's prompt_metadata["reasoning_content"], captured with token usage and finish_reason so the success, truncation and content-filter paths record it alike. The key is cleared first so a value inherited from request metadata cannot survive. The text is kept for review and scoring and is never replayed as history.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Richard Lundeen (richlundeen)
approved these changes
Nov 14, 2026
Richard Lundeen (richlundeen)
left a comment
Richard Lundeen (richlundeen)
left a comment
Contributor
There was a problem hiding this comment.
Automated review completed. No blocking concerns remain. This review records source coverage; merging is handled separately.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Keeps the reasoning text that reasoning models return over Chat Completions, which
OpenAIChatTargetandLiteLLMChatTargetdropped until now. Closes #4266.extract_reasoning_contentreads the message field providers use for it:reasoning_content(DeepSeek, LiteLLM, xAI, Qwen, older vLLM) orreasoning(OpenRouter, Ollama, newer vLLM). Only non-empty strings count.capture_reasoning_contentstores it in the first response piece'sprompt_metadata["reasoning_content"]. It is called fromcapture_usage_and_finish_reason, so both targets record it on the success, token-limit truncation and content-filter paths without target-specific code.finish_reason, the key is cleared from every piece first, so a value inherited from request metadata can't survive on a response that has no reasoning.partial_contentdoes), so it is never sent back to the model as history and no history builder or capability changes.Before / after for a DeepSeek-style response cut off at the token limit while still reasoning (
content=None,finish_reason="length",reasoning_content="Let me think about this step by step"):finish_reason: lengthand token counts; reasoning text lostprompt_metadata["reasoning_content"] = "Let me think about this step by step"The alternative of storing a
reasoningpiece likeOpenAIResponseTargetis discussed in the issue; I'm happy to switch if you prefer it.