fix(agents): keep null in the task output schema embedded in the prompt

[monkscode]

build_task_prompt_with_schema embeds the task output schema into the prompt using
generate_model_description, whose strip_null_types defaults to True. Combined with
ensure_all_properties_required, an Optional[str] = None field is presented to the
model as a required, non-nullable string — leaving it no way to express "not
applicable", and contradicting the provider-side response schema generated from the
same model.

That sanitizer targets OpenAI strict function-calling schemas. This call site produces
prompt prose, where those constraints do not apply.

This passes strip_null_types=False, matching the existing call for tool schemas at
utilities/agent_utils.py:268. required still lists every property, which is the
standard strict-mode idiom once the type is nullable.

Before / after

For class Step(BaseModel): name: str; note: Optional[str] = None:

before   note: {"default": null, "type": "string"}                              <- cannot be null
after    note: {"anyOf": [{"type": "string"}, {"type": "null"}], "default": null}

required is ['name', 'note'] in both cases — unchanged by this PR.

Tests

Adds lib/crewai/tests/agents/test_agent_utils.py. No existing test referenced
build_task_prompt_with_schema, so the file is new; maintainers may prefer it
elsewhere. It fails on main with KeyError: 'anyOf' and passes with this change.

Existing cassette-based tests are unaffected: vcr_config matches on
["method", "scheme", "host", "port", "path"], not on the request body.

Fixes #6774

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The task prompt schema now preserves null types for JSON and Pydantic outputs. Tests verify that optional Pydantic fields produce schemas accepting both strings and null values.

Changes

Nullable task output schemas

Layer / File(s)	Summary
Schema generation and nullable-field validation lib/crewai/src/crewai/agent/utils.py, lib/crewai/tests/agents/test_agent_utils.py	build_task_prompt_with_schema passes strip_null_types=False when generating JSON and Pydantic schemas. Tests verify that optional fields retain an anyOf schema for string and null values.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the agent fix that preserves null values in task output schemas embedded in prompts.
Description check	✅ Passed	The description explains the nullable-schema bug, the implementation, the test coverage, and the expected behavior.
Linked Issues check	✅ Passed	The changes satisfy [#6774] by preserving nullable types in both prompt schema branches and adding a regression test.
Out of Scope Changes check	✅ Passed	The code and test changes remain focused on preserving null types in task output schemas and contain no unrelated changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (2)

lib/crewai/tests/agents/test_agent_utils.py (2)

25-27: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Cover the output_json branch.

This test constructs Task with output_pydantic only. It never executes the task.output_json branch changed in build_task_prompt_with_schema. Add a matching JSON-output case or parameterize the test over both output attributes.

Based on PR objectives: nullable prompt schemas must be preserved for both JSON and Pydantic outputs.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/tests/agents/test_agent_utils.py` around lines 25 - 27, Extend the
test around build_task_prompt_with_schema to cover Task configured with
output_json in addition to output_pydantic, either by adding a matching
JSON-output case or parameterizing both configurations. Ensure the assertions
verify nullable prompt schemas are preserved for both output attributes.

---

33-36: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Make the anyOf assertion order-independent.

The exact list comparison makes member order part of the test contract. JSON Schema alternatives are order-independent, so a valid generator change can fail this test.

Proposed assertion

-    assert schema["properties"]["note"]["anyOf"] == [
-        {"type": "string"},
-        {"type": "null"},
-    ]
+    assert {
+        entry["type"] for entry in schema["properties"]["note"]["anyOf"]
+    } == {"string", "null"}

Based on coding guidelines: tests should focus on behavior rather than implementation details.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/crewai/tests/agents/test_agent_utils.py` around lines 33 - 36, Update the
`anyOf` assertion in the agent schema test to compare alternatives without
relying on list order, while still requiring exactly the string and null schema
members. Preserve the existing validation of the `note` property’s schema.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@lib/crewai/tests/agents/test_agent_utils.py`:
- Around line 25-27: Extend the test around build_task_prompt_with_schema to
cover Task configured with output_json in addition to output_pydantic, either by
adding a matching JSON-output case or parameterizing both configurations. Ensure
the assertions verify nullable prompt schemas are preserved for both output
attributes.
- Around line 33-36: Update the `anyOf` assertion in the agent schema test to
compare alternatives without relying on list order, while still requiring
exactly the string and null schema members. Preserve the existing validation of
the `note` property’s schema.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fe8f54e9-1bab-4281-b4e9-a21ba24e21be

📥 Commits

Reviewing files that changed from the base of the PR and between c8f441c and 4ca325a.

📒 Files selected for processing (2)

• lib/crewai/src/crewai/agent/utils.py
• lib/crewai/tests/agents/test_agent_utils.py

[monkscode]

@lorenzejay @greysonlalonde — flagging this for triage since it touches agent/utils.py, which you two have most recently worked in.

One concrete blocker before the change itself: CI has never run here. All seven workflows on the head commit sit at action_required (first-time-contributor fork gate), so CodeRabbit is the only check and this reads as unverified. One "Approve and run workflows" click would give it a real signal.

The change is two lines, and there is precedent in-tree: utilities/agent_utils.py:268 already passes strip_null_types=False for tool schemas. This does the same at both branches of build_task_prompt_with_schema. required is unchanged — every property is still listed, which is the standard strict-mode idiom once the type is nullable.

On why it is worth the look: the bug is silent, not cosmetic. The provider-side response schema still permits nulls, so every task validates and returns normally — the damage is in what the model writes to avoid emitting a null it has been told is illegal. On our planner task we measured unrelated parameters concatenated into one required string field, a step silently dropped, and some runs extending a single JSON string value until it hit the output-token ceiling. Our end-to-end pass rate went 96.7% → 86.7% on 1.15.10, which is why we are still pinned to 1.8.1.

Happy to move or split the test file, or rebase — just say where.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[percymcn]

Ran #6774's repro on crewai==1.15.14 and simulated this diff verbatim before writing. It does what it says — post-fix the model can express "not applicable" again. Four things I measured that aren't visible from the diff.

1. This fixes the nullability half of #6774; the required half survives.

pydantic    required: ['name']
pre-fix     required: ['name', 'note']
POST-FIX    required: ['name', 'note']

POST-FIX    note: {"anyOf": [{"type": "string"}, {"type": "null"}], "default": null, ...}

ensure_all_properties_required runs regardless of strip_null_types. To be fair to the diff: required-but-nullable is the ordinary strict-mode encoding and the model can now emit null, so the user-facing harm in the issue is genuinely gone. But #6774's other complaint is that the in-prompt and provider-side contracts disagree, and on required they still do — ['name'] vs ['name','note']. Worth deciding deliberately rather than by omission; the new test doesn't assert on required.

2. Eight call sites feed this schema into the same prompt slice. This fixes two.

I18N.slice("formatted_task_instructions") is also fed by utilities/converter.py:549 and :556, lite_agent.py:698, agent/core.py:1841, and utilities/evaluators/task_evaluator.py:100 / :174 — none of which pass strip_null_types=False. agents/agent_adapters/base_converter_adapter.py:152/:154 return the same description to their caller. So after this merges, the same prompt built through Converter, LiteAgent or the task evaluator still strips null.

Useful precedent already in the tree: utilities/agent_utils.py:267 passes strip_null_types=False for tool parameters. The pattern exists; it's applied at one of the sites that needs it.

3. strip_null_types isn't the only pass that shouldn't run on a prompt, and the other one destroys data.

generate_model_description also runs force_additional_properties_false, which is unconditional on any type == "object" node. A Dict[str, str] field:

raw      : {"type": "object", "additionalProperties": {"type": "string"}, "title": "Tags"}
pre-fix  : {"type": "object", "additionalProperties": false, "properties": {}, "required": [], "title": "Tags"}
POST-FIX : (unchanged)

additionalProperties: false on a node with no properties doesn't close the object, it empties it — the only legal instance is {}. The prompt therefore tells the model that a dictionary field must be empty, which is the same class of contradiction #6774 reports, one step worse: the model can comply perfectly and still produce nothing.

Two details that make this hard to catch later:

• The pass also adds properties: {} and required: [], so the emptied map comes out byte-identical to a genuinely empty BaseModel. Nothing downstream can tell the two apart.
• On the strict paths, OpenAI's own transformer (openai@7.4.0, openai/lib/transform → toStrictJsonSchema()) throws on the raw open map (must set additionalProperties: false) and accepts the emptied form verbatim. So the sanitizer turns a precise, fixable 400 into a silent 200 with a field that can never be populated. That function is an offline oracle — no API key — if you want to pin any of this in CI.

4. The new test passes either way on that shape. _Output is name: str + note: str | None, so it stays green while the Dict damage remains. Adding tags: dict[str, str] to _Output, or a second case, would pin it.

None of this argues against merging — the diff is correct for what it does. It's about whether the remaining call sites and force_additional_properties_false belong here or in a follow-up.

[monkscode]

@percymcn — I loaded pydantic_schema_utils.py from main @ f7ba8e3 (v1.15.14) and ran the real generate_model_description over both shapes rather than reading the diff. All four points reproduce. The one thing I did not check is the openai-node toStrictJsonSchema() behaviour you cite in (3); everything below I ran.

1. required — unchanged by design, and I'll pin it if that's wanted.

pydantic required : ['name']
pre-fix  required : ['name', 'note']
POST-FIX required : ['name', 'note']

ensure_all_properties_required runs at pydantic_schema_utils.py:679, above the strip_null_types branch at :681, so this flag structurally cannot reach it. Required-plus-nullable is the encoding I was aiming for, but you're right that the test leaves it implicit: assert schema["required"] == ["name", "note"] passes today and would state it. I'll add it on request.

Making the in-prompt required mirror Pydantic's ['name'] instead means changing ensure_all_properties_required, which is shared with the provider strict paths. Different blast radius — not something I'd fold into this.

2. Other call sites — scoped deliberately, happy to widen.

Confirmed on main, with one addition: lite_agent.py:866 also embeds an unstripped schema, through the lite_agent_response_format slice rather than formatted_task_instructions. The full prompt-facing set that still strips null after this merges:

utilities/converter.py:549, :556 · lite_agent.py:698, :866 · agent/core.py:1841 · utilities/evaluators/task_evaluator.py:100, :174 · agents/agent_adapters/base_converter_adapter.py:152, :154 (reaches the model via enhance_system_prompt)

The llms/providers/* and tools/structured_tool.py sites are correctly excluded — those build provider response schemas, where stripping is the right behaviour.

I scoped this to the site #6774 reproduces on to keep a first contribution reviewable. Extending to the nine above is mechanical; I'll push it here if a maintainer prefers one PR, otherwise a follow-up.

3. force_additional_properties_false — reproduced, but the fix isn't prompt-only.

raw      tags: {"additionalProperties": {"type": "string"}, "type": "object", ...}
pre-fix  tags: {"additionalProperties": false, "properties": {}, "required": [], "type": "object", ...}
POST-FIX tags: identical

An object with properties: {} and additionalProperties: false admits exactly one instance, {} — so the prompt states that a dict field must be empty. Same class of contradiction as #6774, and worse, since the model can comply perfectly and still produce nothing.

Why I'd keep it out of this PR isn't scope hygiene. force_additional_properties_false is called at :670 from generate_model_description and also at :610 from _common_strict_pipeline, which backs sanitize_tool_params_for_openai_strict / _anthropic_strict / _bedrock_strict. On those paths the closure is arguably load-bearing — OpenAI strict mode cannot express an open map at all, so the real choice is fail-loudly versus degrade-silently. That is a maintainer decision, not a flag flip, and it needs its own issue. I'll file it with the repro unless you'd rather own it, since the measurements are yours.

4. Test coverage for that shape — lands with (3). Adding tags: dict[str, str] to _Output pins nothing on its own; the assertion that would pin it fails on this branch.

Blocker unchanged: the seven workflows on the head commit are still action_required behind the first-time-contributor fork gate, so nothing here has been executed by CI. @lorenzejay @greysonlalonde — one "Approve and run workflows" click would give this a real signal, and #6774 is still untriaged.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[monkscode]

@lucasgomide — flagging this to you specifically rather than broadly: #4579 is where strip_null_types was introduced, together with the utilities/agent_utils.py call site this PR cites as precedent. This is that same fix applied to the schema embedded in the task prompt — the one site of that pair which never received it.

@theCyberTech — a smaller, separate ask for whoever reaches it first: the seven workflows on the head commit are all still action_required behind the first-time-contributor fork gate. One "Approve and run workflows" click gives this a real signal and commits no one to a review.

What has changed since 4 Aug

The defect is still on main at HEAD: both branches of build_task_prompt_with_schema call generate_model_description(...) with the default. It also ships in the released 1.15.16, verified against the wheel.

The surface is wider than the PR title suggests:

• When the agent has tools, the provider-side model is dropped entirely — effective_response_model = None if self.original_tools else self.response_model (agents/crew_agent_executor.py:356, and :1170 async). The stripped prompt schema is then the only contract the model sees; there is no second schema to fall back on.
• On the native OpenAI paths (llms/providers/openai/completion.py:875 and :1824) the provider schema is built from the same generate_model_description(...) default and shipped with strict: True. There the required non-nullable string is enforced, not merely suggested.

Measured on 1.15.16, offline, for class Step(BaseModel): name: str; note: str | None = None:

                    required           note
pydantic            ['name']           {"anyOf":[{"type":"string"},{"type":"null"}],"default":null}
prompt, pre-fix     ['name','note']    {"default":null,"type":"string"}
prompt, post-#6775  ['name','note']    {"anyOf":[{"type":"string"},{"type":"null"}],"default":null}

Scope is yours to set. As it stands this is two lines plus a regression test in tests/agents/test_agent_utils.py, matching that directory's existing layout. Nine other prompt-facing call sites still strip null — enumerated earlier in this thread — and I will extend the change here or leave it as a follow-up, whichever you prefer. The force_additional_properties_false case raised above I would file separately, since that pass is shared with the OpenAI/Anthropic/Bedrock strict sanitizers and the decision there is not a flag flip.