[vintcessun]

What

A proposal, docs/proposals/chat-images.md (status: proposed), for sending an image from chat to a local model that reports vision. Docs only. No code, and no Known-gaps lines change yet.

It settles the two decisions #1991 leaves open:

• The stored shape of a user turn. {"text", "images": [{key, mime, size_bytes, sha256}]}, with the bytes as files under the workspace's folder, one folder per thread. images is present only when the turn carried any, so existing rows and message_text() are unchanged.
• What history resends. The newest user turn that carried images keeps sending them until a newer one carries its own; older images drop and their text stays. An image is priced at a fixed per-image cost, measured at implementation, so the history budget still holds.

And the choices those two force:

• Message gains a separate images field instead of widening content to str | list[Part], so Studio and every text-only path do not change.
• The downgrade stays at the provider seam (for_template() drops images when not can_see), and a conversation without images serialises byte-identically to today.
• The runtime decides and the catalog only predicts. The catalog's reads_images checks only the projector, while the runtime's can_see also needs the template to take typed content, which llama.cpp decides by running the template in its own Jinja engine; nothing outside it can compute that. So the provider records can_see whenever it reads /props (after an install and on the first turn after a load), and the badge and the composer use the recorded answer, falling back to the prediction before the first load. A turn with images for a model that cannot see gets a 409 with a sentence, never a silent drop.
• v1 is local models only. No remote model reports vision today; the remote manifest's modalities.input is the follow-up.

Why

#1991 asks for a proposal first, to agree on the stored turn shape before code. The four implementation PRs are listed under Ship in the proposal.

Refs #1991

How to test

python scripts/check_docs.py

High-level PR Summary

This PR proposes a design for sending images to vision-capable local models in chat. The proposal document defines how images will be stored (as files with metadata in JSON, not inline base64), how history will handle them (resending only the most recent set of images), and how the system will determine model capability (runtime detection recorded after model load, not just catalog prediction). The design prioritizes backward compatibility by adding a separate images field to Message rather than changing content, ensuring text-only models and existing conversations remain unchanged. The proposal is scoped to local models initially, with remote vision support deferred to follow-up work.

⏱️ Estimated Review Time: 5-15 minutes

💡 Review Order Suggestion

Order	File Path
1	docs/proposals/chat-images.md

[vercel]

@vintcessun is attempting to deploy a commit to the Rohan Verma's projects Team on Vercel.

A member of the Team first needs to authorize it.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 7716fd2a-8b13-4f27-b8b0-894e8064b025

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[AnishSarkar22]

@vintcessun thanks for this proposal, we are working on this currently.