Skip to content

fix(agents): keep native tool messages on result_as_answer steps - #8054

Open
BlueX888 wants to merge 1 commit into
crewAIInc:mainfrom
BlueX888:fix/prep-step-executor-result-as-answer-dangling-tool-calls
Open

BlueX888 wants to merge 1 commit into
crewAIInc:mainfrom
BlueX888:fix/prep-step-executor-result-as-answer-dangling-tool-calls

Conversation

@BlueX888

@BlueX888 BlueX888 commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Related issue

None — found by code inspection of the plan-and-execute step executor; no existing issue covers this path. (Returning contributor — #7487 merged — so the first-time-contributor issue requirement does not apply.)

Summary

StepExecutor._execute_native_tool_calls returned as soon as a tool marked result_as_answer ran — before appending the tool message for that call and before executing the rest of the batch. The assistant message carrying tool_calls was therefore left without any tool reply (dangling tool_call_ids), and the _execute_native multi-turn loop kept going: the next LLM call saw a malformed history that OpenAI/Anthropic reject ("An assistant message with 'tool_calls' must be followed by tool messages responding to each tool_call_id"), and the tool's result was discarded in favor of the LLM's next text answer.

The default executor handles the same situation by appending the tool message first and finishing with the tool's result (AgentFinish on result_as_answer, lib/crewai/src/crewai/agents/crew_agent_executor.py:1096-1104), matching the documented meaning of the flag: "Flag to check if the tool should be the final agent answer." (lib/crewai/src/crewai/tools/base_tool.py:180-183).

Changes

  • _execute_native_tool_calls now executes the whole batch, answers every declared tool_call_id with a tool message, and returns (results, final_answer), where final_answer is the first result_as_answer result in the batch (if any).
  • _execute_native returns final_answer immediately, ending the multi-turn loop, so the step's result is the tool's result — mirroring CrewAgentExecutor's AgentFinish behavior.
  • One existing test that patches _execute_native_tool_calls was updated for the new return shape.

Verification

  • Tests added or updated for the changed behavior
  • Relevant tests and quality checks pass locally

Both new regression tests fail on main and pass with this change. Commands run (uv workspace, macOS arm64, Python 3.13.15):

  • uv run pytest lib/crewai/tests/agents/test_agent_executor.py -q -k "native_result_as_answer"
    • before the fix: 2 failed — no tool messages appended (assert set() == {'call_answer', 'call_side'}) and the step result was the LLM's next answer (assert 'done' == 'answer_tool-result')
    • after the fix: 2 passed
  • uv run pytest lib/crewai/tests/agents/test_agent_executor.py -q → 100 passed, 2 failed; the two failures (TestVisionImageFormatContract::test_anthropic_provider_has_image_block_converter, TestResponseFormatWithKickoff::test_kickoff_response_format_with_planning_and_tools) fail identically on unpatched main — pre-existing, unrelated to this change.
  • uv run pytest lib/crewai/tests/tools/test_tool_failure.py -q → 97 passed
  • uv run pytest lib/crewai/tests/hooks/test_deny_reaches_the_caller.py -q → 15 passed
  • uv run pytest lib/crewai/tests/agents/ -q -k "result_as_answer" → 5 passed
  • uv run ruff check lib/ → All checks passed!
  • uv run ruff format --check lib/ → 942 files already formatted
  • uv run mypy --config-file pyproject.toml lib/crewai/src/crewai/agents/step_executor.py → Success: no issues found in 1 source file

Additional context

This pull request was prepared with AI coding assistance. Per .github/CONTRIBUTING.md it requires the llm-generated label; external contributors cannot apply labels, so maintainers please apply it during triage (same as #7336).


Note

Medium Risk
Changes plan-and-execute native tool batching and step termination; behavior is localized to StepExecutor but affects any planning step using result_as_answer with native function calling.

Overview
Fixes native plan-step execution when a tool is marked result_as_answer.

Previously StepExecutor._execute_native_tool_calls exited on the first such tool before running the rest of the batch or appending tool messages, leaving assistant tool_calls without replies and letting _execute_native call the LLM again (often failing provider validation and returning the wrong step result). The batch now runs to completion, every tool_call_id gets a reply, and the method returns (concatenated_results, final_answer); _execute_native stops the loop immediately when final_answer is set so the step answer is the tool output, aligned with the default agent executor’s result_as_answer behavior.

Regression tests cover multi-tool batches (stub LLM, message history, and execute() end-to-end).

Reviewed by Cursor Bugbot for commit 00a0807. Bugbot is set up for automated code reviews on this repo. Configure here.

@strix-security

strix-security Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 78 commits after the last Strix review (adf5756). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

No security issues found.

Review summary

Reviewed PR #8054, which changes StepExecutor._execute_native_tool_calls to execute the full batch of native tool calls, append a tool message for every declared tool_call_id, and return the first result_as_answer tool result as the step's final answer. I read both changed files in full and traced the modified data flow (_execute_native → _execute_native_tool_calls → execute_single_native_tool_call). The change mirrors the existing CrewAgentExecutor behavior (tool message first, then AgentFinish on result_as_answer) and introduces no new injection sinks, authorization gaps, secret handling, or other security-relevant surfaces. The tool calls executed are those the LLM declared against the agent's already-authorized tool set, and the surfaced result_as_answer result matches the flag's documented semantics. The test additions are test-only and reveal no production security patterns. semgrep (151 rules) reported 0 findings and bandit reported no issues on the production file. No security issues found.

Updated for adf5756.


Reviewed by Strix
Re-run review · Configure security review settings

@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Important

Review skipped

We couldn't safely recover the incremental review. No full review was started, and the last reviewed checkpoint was preserved. Retry later, or explicitly request a full review by commenting @coderabbitai full review.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 47a22e1e-0b23-49fa-8817-ab229641887f
📥 Commits

Reviewing files that changed from the base of the PR and between 6b93fa0 and adf5756.

📒 Files selected for processing (2)
  • lib/crewai/src/crewai/agents/step_executor.py
  • lib/crewai/tests/agents/test_agent_executor.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

Native tool-call batches now execute every declared call, even when one tool is marked result_as_answer. The executor returns the first such result after the batch. Tests cover answer-based step completion and tool replies for each declared call ID.

Changes: Native tool batch execution

Layer / File(s) Summary
Batch execution and answer return
lib/crewai/src/crewai/agents/step_executor.py, lib/crewai/tests/agents/test_agent_executor.py
The native tool-call method returns accumulated tool results and the first result_as_answer result. The executor returns that answer after processing the batch. Tests check that all declared calls receive replies and that the answer completes the step without another LLM call.

Suggested reviewers: joaomdmoura

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to adf57

No actionable merge-blocking issue is identified; proceed with normal checks.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 14 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: preserving native tool messages when result_as_answer ends a step.
Description check Passed The description is complete and directly addresses the template. It includes issue context, a detailed summary, implementation changes, verification results, test status, and additional context. The a…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

StepExecutor._execute_native_tool_calls returned before appending the
tool message when a result_as_answer tool ran, so an assistant message
with tool_calls was left with dangling tool_call_ids and the outer loop
called the LLM again on that malformed history. The result_as_answer
result was discarded in favor of the next LLM answer and the remaining
calls in the batch were never executed.

Answer every declared tool call with a tool message, execute the whole
batch, and end the step with the result_as_answer tool's result.
@BlueX888
BlueX888 force-pushed the fix/prep-step-executor-result-as-answer-dangling-tool-calls branch from adf5756 to 00a0807 Compare October 17, 2026 03:36

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant