[sergqwer]

Claude Code returns a picture it opens with Read (a screenshot, a diagram) as an image block inside a tool_result. anthropic_to_messages turned every tool_result into its text (_text_of), so the image never reached the encoder and the model described a picture it had not seen. A user's own pasted image (an image block at the top level of a user message) already worked; only the tool path lost it.

Fix (serve/frontend.py, anthropic_to_messages):

• a tool_result's content goes through _parts_of, as the OpenAI path's tool messages already do; the chat template renders the image item inside <tool_response> (its render_content handles images in any role);
• an image block sent beside tool results in the same user turn is kept too (it used to be dropped by the same branch);
• a text-only tool result is still one string, so every text prompt renders exactly as before.

Test: ImageMarkers.test_anthropic_tool_result_image: an image alone, after text, beside a text tool result, and in both places; it counts the image tokens and checks where they land. It fails on main (0 image tokens instead of 3) and passes here. All serve tests: 159 OK (7 skipped), on 0.1.36.

End to end (Windows, RTX 5090, Flash-Next with the CPU encoder, Claude Code asked to Read a PNG with known text and shapes):

• before: the request carrying the tool result added 35 tokens; the answer was invented ("Hello Vision Test", a red circle);
• after: it added 391 tokens (the image's ~360); both texts with their colours and both shapes with their positions were right, and a dark-theme code screenshot was transcribed verbatim.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio

[sergqwer]

Follow-up, found in a review of this change: a tool's picture the server cannot read now became a 400. That happens on a server with no image encoder (every lazy-start config is text-only), and with an encoder that refuses the picture (WebP without Pillow, an error from strata-vision).

Clients resend the whole history every turn, so one such picture failed every later request of the conversation. Before this PR the picture was dropped silently.

Fixed in 271beca, rebased on v0.1.37:

• A tool's picture that cannot be read becomes [image omitted: <reason>] in its place.
• A picture the user sends keeps the 400, as before.
• The test covers both cases, and it fails without the fix.
• Serve tests: 173 OK (7 skipped).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VZy1yKaDDiA8a7svdwaHio

[sergqwer]

Rebased onto 0.1.39 (8ef230b). The only conflict was prepare, which now calls 0.1.39's encode_prompt(messages, tools, kwargs) after _note_unreadable_tool_images. Server tests on 0.1.39 with this branch: 269 run, 1 error. That error is test_responses.OverHttp.test_json_schema_text_format, which fails on a clean v0.1.39 too when jsonschema is not installed; #785 fixes it.

[sergqwer]

Reopened as #1041, the same branch and commit. GitHub closed this one when my fork was made private by mistake, which took it out of the fork network.

[Niko1221]

Merged for 0.1.40: an image in an Anthropic tool_result reaches the encoder. We added a log line when a tool's picture is left out, and a tool picture's URL is downloaded once.