Skip to content

fix: send output_schema to the Interactions API as response_format - #7871

Open
himanshu096 wants to merge 1 commit into
google:mainfrom
himanshu096:fix/interactions-output-schema
Open

himanshu096 wants to merge 1 commit into
google:mainfrom
himanshu096:fix/interactions-output-schema

Conversation

@himanshu096

@himanshu096 himanshu096 commented Oct 30, 2026 •

Copy link
Copy Markdown

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

Problem:
An LlmAgent's output_schema is set on llm_request.config.response_schema. generate_content_via_interactions never reads that field, so the schema is left out of every Interactions API request (Gemini(use_interactions_api=True)), and replies are unconstrained. This affects agents without tools, and agents with tools on Vertex AI, where output_schema_and_tools is True, so the set_model_response fallback is skipped.

Solution:

  • New private helper: _build_response_format() sits next to build_generation_config(). It turns config.response_json_schema or config.response_schema into the Interactions API's JSON text response format: {"type": "text", "mime_type": "application/json", "schema": ...}.
    • It reuses _schema_utils.schema_to_json_schema(), the converter the Anthropic connector already uses, for pydantic models, generic aliases such as list[Model], and dicts.
    • For types.Schema it uses Schema.json_schema.
    • response_json_schema takes precedence when both fields are set.
  • Request: generate_content_via_interactions sends response_format and response_mime_type="application/json", which google-genai documents as required alongside response_format. It sends them only when a schema is set, so requests without one are unchanged.
  • Compatibility: TextResponseFormatParam and the response_format / response_mime_type parameters of interactions.create() exist in google-genai 2.30, ADK's minimum, and in later releases (checked 2.31 and 2.34).

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

New TestOutputSchema in tests/unittests/models/test_interactions_utils.py, testing through generate_content_via_interactions and the runner:

  • test_schema_is_sent_as_response_format, parametrized over a pydantic model, list[Model] (keeping its $defs), types.Schema, a raw JSON schema dict, and response_json_schema taking precedence.
  • test_schema_is_sent_when_streaming.
  • test_no_schema_sends_neither_key: requests without a schema are unchanged.
  • test_agent_output_schema_reaches_the_request: an LlmAgent(output_schema=...) run through InMemoryRunner, on both the Gemini API and Vertex AI via the env_variables fixture.
  • test_agent_with_tools_sends_schema_with_tools_on_vertex: on Vertex AI, a schema agent with a function tool sends both the tool and the response format, with no set_model_response.

On main without the fix, 9 of these 10 tests fail. The tenth, test_no_schema_sends_neither_key, passes either way.

$ pytest tests/unittests/models/test_interactions_utils.py
215 passed in 2.37s

$ pytest tests/unittests/models/test_interactions_utils.py tests/unittests/models/test_google_llm.py \
    tests/unittests/models/test_llm_request.py tests/unittests/flows/llm_flows tests/unittests/utils
2179 passed, 22 skipped, 12 failed

The 12 failures are all in prompt/test_instructions_utils.py. They need the optional jinja2 package, which isn't in my environment, and they fail the same way on main without this change.

ruff, isort, pyink and codespell (pre-commit versions) pass on both changed files. mypy reports no new errors in interactions_utils.py.

Manual End-to-End (E2E) Tests:

This runs four LlmAgent(output_schema=...) agents against Vertex AI (global) with Gemini(model="gemini-3.6-flash", use_interactions_api=True), and checks whether each final reply validates against its schema. It compares released google-adk 2.13.0 with this branch.

===== BEFORE (google-adk 2.13.0)
[no_tools]        valid=False reply='Why was the cat sitting on the computer?\n\nTo keep an eye on the mouse! 🐾'
[no_tools_stream] valid=False reply='Why was the cat sitting on the computer?\n\nTo keep an eye on the mouse! 🐾'
[list_schema]     valid=False reply="Here are two quick facts about the Moon:\n\n1. **It's slowly drifting away:** ..."
[with_tool]       valid=False reply='The current weather in Paris is **sunny**, around **22°C (72°F)** with a **light breeze**. ...'
===== AFTER (this branch)
[no_tools]        valid=True reply='{\n  "ok": true,\n  "reason": "Why was the cat sitting on the computer? To keep an eye on the mouse!"\n}'
[no_tools_stream] valid=True reply='{\n  "ok": true,\n  "reason": "Why was the cat sitting on the computer? To keep an eye on the mouse!"\n}'
[list_schema]     valid=True reply='[\n  {\n    "ok": true,\n    "reason": "The Moon is Earth\'s only natural satellite, ..."\n  },\n  ...'
[with_tool]       valid=True reply='{\n  "ok": true,\n  "reason": "Successfully fetched Paris weather and determined picnic suitability."\n}'

The four cases:

  • no_tools: output_schema=Verdict (ok: bool, reason: str), with the prompt "Tell me a short joke about cats."
  • no_tools_stream: the same agent with StreamingMode.SSE.
  • list_schema: output_schema=list[Verdict]. This confirms the API accepts the $defs/$ref that pydantic generates.
  • with_tool: output_schema=Verdict plus a get_weather function tool on Vertex AI. This confirms the API accepts a response format together with function tools, and the model still calls the tool first.
E2E script
import asyncio
import pydantic
from google.adk.agents import LlmAgent
from google.adk.agents.run_config import RunConfig, StreamingMode
from google.adk.models import Gemini
from google.adk.runners import InMemoryRunner
from google.genai import types


class Verdict(pydantic.BaseModel):
  ok: bool
  reason: str


def get_weather(city: str) -> dict[str, str]:
  """Returns the current weather in a city."""
  return {"city": city, "weather": "sunny, 22C, light breeze"}


async def run(name, schema, prompt, tools=(), stream=False):
  agent = LlmAgent(
      name=name,
      model=Gemini(model="gemini-3.6-flash", use_interactions_api=True),
      instruction="You are a friendly assistant.",
      output_schema=schema,
      tools=list(tools),
  )
  runner = InMemoryRunner(agent=agent, app_name="live")
  session = await runner.session_service.create_session(app_name="live", user_id="u")
  final = ""
  async for event in runner.run_async(
      user_id="u",
      session_id=session.id,
      new_message=types.Content(role="user", parts=[types.Part(text=prompt)]),
      run_config=RunConfig(streaming_mode=StreamingMode.SSE if stream else StreamingMode.NONE),
  ):
    if event.partial or not event.content or not event.content.parts:
      continue
    text = "".join(p.text for p in event.content.parts if p.text and not p.thought)
    if text:
      final = text
  try:
    pydantic.TypeAdapter(schema).validate_json(final)
    valid = True
  except pydantic.ValidationError:
    valid = False
  print(f"[{name}] valid={valid} reply={final[:160]!r}")


async def main():
  await run("no_tools", Verdict, "Tell me a short joke about cats.")
  await run("no_tools_stream", Verdict, "Tell me a short joke about cats.", stream=True)
  await run("list_schema", list[Verdict], "Give me two quick facts about the moon.")
  await run("with_tool", Verdict, "What's the weather in Paris? Is it good for a picnic?", tools=[get_weather])


asyncio.run(main())

Run with GOOGLE_GENAI_USE_VERTEXAI=1 GOOGLE_CLOUD_PROJECT=<project> GOOGLE_CLOUD_LOCATION=global.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

  • Silently ignored keywords: the Interactions API ignores some JSON Schema keywords, such as minLength, maxLength and uniqueItems, without an error. It returns 400 for overly complex schemas. This PR sends the schema unchanged and leaves that behavior to the API.
  • Pydantic vs types.Schema: for pydantic models, generateContent converts response_schema into a types.Schema, while this path sends pydantic's JSON schema as-is. The live check above shows the API accepts it, including $defs/$ref.

🤖 Generated with Claude Code

@google-cla

google-cla Bot commented Oct 30, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@himanshu096
himanshu096 force-pushed the fix/interactions-output-schema branch from a947cfb to 8a07937 Compare October 30, 2026 22:30
An agent's output_schema is set on config.response_schema, which
generate_content_via_interactions never read, so the schema was dropped
from every Interactions API request and replies were unconstrained. This
hit agents without tools everywhere, and agents with tools on Vertex AI,
where the set_model_response fallback is skipped.

Send the schema as a JSON text response_format, with the
response_mime_type the API requires alongside it.

Fixes google#7870
@himanshu096
himanshu096 force-pushed the fix/interactions-output-schema branch from 8a07937 to 85fba8f Compare October 30, 2026 22:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

output_schema is silently dropped when Gemini uses the Interactions API (use_interactions_api=True)

2 participants