feat(local): let the agent draw pictures

.pre-commit-config.yaml

@@ -100,6 +100,14 @@ repos:
   # in surfsense_local/frontend/package.json.
   - repo: local
     hooks:
+      # The desktop frontend's lint, on the files a commit changes: CI runs it
+      # before the tests, so a lint error there means no tests ran at all.
+      - id: eslint-desktop-frontend
+        name: eslint-desktop-frontend
+        entry: bash -c 'cd surfsense_local/frontend && pnpm exec eslint "${@#surfsense_local/frontend/}"' --
+        language: system
+        files: ^surfsense_local/frontend/src/.*\.(ts|tsx)$
+        stages: [pre-commit]
       - id: formatjs-extract
         name: formatjs-extract
         entry: pnpm --dir surfsense_local/frontend translations:extract

docs/adr/0087-the-agent-makes-pictures-with-its-own-surfsense-tool.md

@@ -0,0 +1,28 @@
+# ADR 0087: The agent makes pictures with its own SurfSense tool, not opencode's
+
+- **Status:** Accepted
+- **Date:** 2026-11-07
+- **Amends:** [ADR 0066](0066-web-pages-are-html-the-model-writes-without-network.md), in part: its rule that nothing is written into a page but the policy
+- **Source:** maintainer decision, 7 Nov 2026 (no permalink)
+
+## Context
+
+The agent could start Studio's image job only from ticked sources, with a chat model rewriting its request, and got nothing back to look at or place. Documents could place only source figures and analysis charts.
+
+opencode 2.2 added `experimental_image_generate`. A spike on 2.2.5 with SurfSense's configuration found it writes files and edits through `/images/generations` and `/images/edits`, but runs in the background: the turn goes idle before the picture is ready, and a second run brings the notice. It is listed only when a provider has an image model.
+
+## Decision
+
+- The agent draws through `surfsense_make_image`, a SurfSense tool over Studio's job, not opencode's tool.
+- A picture is a normal Studio artifact, with versions, the viewer and the Edit box, and needs only an image model: the agent writes the prompt, and no chat model rewrites it.
+- `image-<artifact id>` names one version, and only the chat that made it can place or edit it.
+- A tool call keeps its limit. A picture past it answers as queued, and no tool polls.
+- A page names an image as `surfsense-image:<name>`, and saving makes it a `data:` URL. The agent has no shell, and typing base64 through a model is not practical.
+
+## Consequences
+
+- opencode threads and Claude plan threads get the same tool, which opencode's would not reach. The pinned opencode stays at 2.1.2, with no new model routes or second-run handling.
+- Local pictures run through the existing sd-server and text-runtime hold.
+- A stored page is what the model wrote, its images inline, plus the policy. Its working file keeps the names.
+- A document needing a picture still being drawn waits for the user's next message.
+- Studio's own buttons keep drafting from sources through a chat model, and Studio's drafts cannot place these pictures.

docs/adr/README.md

@@ -115,3 +115,4 @@ The Date is the day the decision was first written down. Source links are permal
 | 0084 | [ElevenLabs is a connection with its own client](0084-elevenlabs-is-a-connection-with-its-own-client.md) | Accepted |
 | 0085 | [A user can move models or all data to another drive, and Electron's settings stay behind](0085-the-data-folder-moves-and-electrons-settings-stay.md) | Accepted |
 | 0086 | [The contract redline ships with the Business switch, as an app runner on SurfSense's own Word engine](0086-the-contract-redline-ships-with-the-business-switch-on-the-word-engine.md) | Accepted |
+| 0087 | [The agent makes pictures with its own SurfSense tool, not opencode's](0087-the-agent-makes-pictures-with-its-own-surfsense-tool.md) | Accepted |

docs/architecture/studio.md

@@ -231,7 +231,7 @@ worker/studio/
 ├── web/html/
 └── media/
     ├── audio/podcast/    outline, draft, roster
-    └── visual/           image, infographic, and edit: either's picture changed to an instruction
+    └── visual/           image, infographic, drawn: a prompt the agent wrote, and edit: any picture changed to an instruction
 ```
 
 Every pipeline returns a `Built`: a `title`, the `markdown` that is always the indexed body, and an optional `primary` (and `preview`) with its MIME type and filename. Adding a format is a folder with a `render`, a `case` in `job_router.py` and a row in `formats.py`. Frontend entries below customize its translated label, icon, tooltip and viewer; an unknown key still appears with the server's label, a file icon, the generic tooltip and the document viewer.
@@ -317,9 +317,15 @@ Every pipeline returns a `Built`: a `title`, the `markdown` that is always the i
 - sd-server runs only while Studio needs it, because it keeps a model's weights in memory until it exits. The API answers Electron's poll from the jobs ([`local_image_demand.py`](../../surfsense_local/backend/modules/artifacts/local_image_demand.py)): the chosen model of the image type the oldest processing job needs, read from its recipe, so an edit needs `image_edit` though its format is still `image` or `infographic`; else, for 5 minutes after the last such job ended, the same one, so a second image does not reload it; else nothing, and Electron stops it. A cancel ends the window at once, since sd-server cannot stop a generation, and so does any other Studio job running, since a podcast voices in that memory and a text job loads the chat model into it. The job first gives up the text runtime through the API, held for the whole image, because the two share the graphics card and its writing is done by then: the API lets running replies finish, unloads the text models and keeps local text waiting until the image is drawn, and unloads the speech recognition model, starting no recording's window meanwhile ([admission](local-models/admission.md#the-image-jobs-hold)). Then it waits up to 2 minutes for `/sdapi/v1/sd-models` to name its weights before it posts, and fails with a reason if sd-server never comes up.
 - sd-server is launched with each file of the build on its flag and the model's reviewed `image` defaults as its own (`-W`, `-H`, `--steps`, `--cfg-scale`, `--sampling-method`, `--flow-shift`), since a request that carries none of them starts from the launch settings ([`local-models/catalog.md`](local-models/catalog.md)).
 - The client posts `{model, prompt}` to `/images/generations`, falls back once to `/images` only on `404` or `405`, and remembers the route that worked for the life of the process. It accepts `b64_json`, a data URL or a URL, caps the sizes, and checks the bytes against the claimed MIME type ([`connections.md`](connections.md)).
-- The `text_gen` model writes first in both formats. The markdown body is the image prompt or the brief, so an image can be found by what it shows.
+- The `text_gen` model writes first in both formats, except a picture the agent asked for ([below](#pictures-the-agent-asks-for)). The markdown body is the image prompt or the brief, so an image can be found by what it shows.
 - A drawn picture is v1 of its own document: the job records `version` and `recipe: {"kind": "draft"}` as a Word draft's, so its edits group under it. Regenerate on it still draws from the sources and replaces v1 in place.
 
+### Pictures the agent asks for
+
+`surfsense_make_image` ([agent](agent.md#pictures-the-agent-makes)) creates an `image` or `infographic` with no sources, the chat's `chat_thread_id` and the recipe `{"kind": "image_drawn", "prompt"}` ([`drawn_images/`](../../surfsense_local/backend/modules/artifacts/drawn_images/), [ADR 0087](../adr/0087-the-agent-makes-pictures-with-its-own-surfsense-tool.md)). The job paints the prompt as written, asking no chat model; an infographic adds Studio's sketchnote style around it ([`media/visual/drawn/`](../../surfsense_local/backend/worker/studio/media/visual/drawn/)). The body is the title and the prompt, so it can be found by what it shows.
+
+It needs only `image_gen`, its Regenerate paints the same prompt, and Huey never retries it, since each try may be billed. It is v1 of its own document and takes Studio's Edit box like any picture. Every edit keeps its base's chat, from either box, so that chat can still place it. While a script of its chat names a picture, deleting that picture is refused with `placed_in_document`, since the script names it rather than copying it ([`placed_in.py`](../../surfsense_local/backend/modules/artifacts/drawn_images/placed_in.py)).
+
 ### Editing a picture
 
 `POST /artifacts/{id}/image-edit` with `{instruction}` (1 to 2,000 characters, as Refine's) makes an image's or infographic's next version from the open version's picture ([`image_edits/`](../../surfsense_local/backend/modules/artifacts/image_edits/), [ADR 0077](../adr/0077-an-image-edit-is-the-next-version.md)).

surfsense_local/backend/modules/agent/agent_threads/steps.py

@@ -5,6 +5,7 @@
 
 from modules.agent.tool_endpoint.confirm_job import TOOL_NAME as CONFIRM_JOB_TOOL
 from modules.agent.tool_endpoint.convert_document import TOOL_NAME as CONVERT_TOOL
+from modules.agent.tool_endpoint.make_image import TOOL_NAME as MAKE_IMAGE_TOOL
 from modules.agent.tool_endpoint.registration import SERVER
 from modules.agent.tool_endpoint.rendered_label import TOOL_NAME, rendered_artifact
 from modules.agent.tool_endpoint.revise_document import TOOL_NAME as REVISE_TOOL
@@ -29,6 +30,8 @@
 CONVERT_STEP = f"{SERVER}_{CONVERT_TOOL}"
 # A saved page's result opens as a render's does.
 SAVE_PAGE_STEP = f"{SERVER}_{SAVE_PAGE_TOOL}"
+# So does a picture's.
+MAKE_IMAGE_STEP = f"{SERVER}_{MAKE_IMAGE_TOOL}"
 # A job run opens the revised copy it filled in, and a confirmation the version it made.
 RUN_JOB_STEP = f"{SERVER}_{RUN_JOB_TOOL}"
 CONFIRM_JOB_STEP = f"{SERVER}_{CONFIRM_JOB_TOOL}"
@@ -37,6 +40,7 @@
     REVISE_STEP,
     CONVERT_STEP,
     SAVE_PAGE_STEP,
+    MAKE_IMAGE_STEP,
     RUN_JOB_STEP,
     CONFIRM_JOB_STEP,
 )

surfsense_local/backend/modules/agent/prompts/agent.md

@@ -28,7 +28,9 @@ When the user asks for Word, a PDF, slides or a spreadsheet, make that. Only whe
 
 For a Word document, a PDF, a PowerPoint deck or an Excel workbook, load the `surfsense-documents` skill, then make it with `surfsense_render_document`. To change a document you made, read its script with `surfsense_read_document` and render the change with its `artifact_id`. Make every change the user asks for in that render, reading loose words the natural way, and say what you assumed; ask first only when a change cannot be made as asked. A render of a Word document, a PDF or a deck comes back with its pages as images: look at every one before you answer, and render again if one is wrong.
 
-For a quiz, flashcards, a podcast, a mind map or an image, start it in Studio with `surfsense_create_artifact`, naming the sources by the number at the end of their file names, then tell the user it will appear in Studio. Studio reads those sources itself, so you need not read them first.
+For a quiz, flashcards, a podcast or a mind map, start it in Studio with `surfsense_create_artifact`, naming the sources by the number at the end of their file names, then tell the user it will appear in Studio. Studio reads those sources itself, so you need not read them first.
+
+For a picture or infographic, when asked: `surfsense_make_image`.
 
 For any other document, table or file, write it to `outputs/` with a clear file name, then say in one line what you wrote and where.
 

surfsense_local/backend/modules/agent/skills/surfsense-documents/SKILL.md

@@ -13,6 +13,7 @@ You make a file by writing a Python script and passing it to `surfsense_render_d
 - Use only Python's standard library, python-docx (Word), ReportLab (PDF), python-pptx (PowerPoint), xlsxwriter or openpyxl (Excel), matplotlib (charts), Pillow and numpy. Do not use the network.
 - The script runs alone, from an empty folder, for at most 120 seconds. It cannot read the user's sources: write the content into the script, from what you found in them.
 - To place an image from the sources, find its name with `surfsense_list_images`, list every name you use in the call's `images`, and open it at `os.path.join(os.environ["IMAGES_DIR"], name + ".png")`.
+- A picture from `surfsense_make_image` goes the same way: list the name it gave, `image-<id>`, in `images`, and open it at the same path. If the render says it is still being drawn, tell the user and stop. Draw one only when the user asks for a picture, never for data: a chart stays matplotlib or a native chart.
 - To look at a source image yourself, open the copy `surfsense_list_images` names, `sources/figures/<name>.png`, with `read`. These copies are read-only.
 - With `template_source_id`, a copy of that source's file is at `os.environ["TEMPLATE_PATH"]` (see [Starting from the user's own file](#starting-from-the-users-own-file)).
 - Save a chart or any other intermediate file in the working folder, then place it.

surfsense_local/backend/modules/agent/skills/surfsense-html/SKILL.md

@@ -27,7 +27,8 @@ You write the page yourself, as one HTML file under `outputs/`, and keep it with
 ## The contract
 
 - One complete HTML document: `<!doctype html>`, `<html lang>`, `<head>` with `<meta charset="utf-8">`, `<meta name="viewport" content="width=device-width, initial-scale=1">` and a `<title>`, then `<body>`.
-- Everything inline: CSS in `<style>`, code in `<script>`, images as `data:` URLs or inline SVG.
+- Everything inline: CSS in `<style>`, code in `<script>`, drawn graphics as inline SVG.
+- For an image from `surfsense_make_image`, `surfsense_list_images` or `surfsense_analyze_data`, write `<img src="surfsense-image:<name>" alt="…">`. Saving makes it inline; never type base64 yourself. A page with its images must still fit the size limit, so use few.
 - **The page has no network.** The viewer blocks every request: no CDN, no web font, no `fetch`, no image from a URL, no form that submits. Use system fonts, draw charts as SVG, and write the data into the page. A save lists anything the page asks to load, so you can take it out.
 - No `eval` or `new Function`; they are blocked too.
 - At most 5 MB. A page is usually 10 to 80 KB.

surfsense_local/backend/modules/agent/tool_endpoint/create_artifact.py

@@ -6,12 +6,14 @@
 from sqlalchemy.orm import Session
 
 from modules.agent.thread_folder.layout import SOURCES
+from modules.agent.tool_endpoint.refusal_sentence import refusal_sentence
 from modules.agent.tool_endpoint.registration import SERVER
 from modules.agent.tool_endpoint.rendered_label import TOOL_NAME as RENDER_TOOL
 from modules.agent.tool_endpoint.save_page import TOOL_NAME as SAVE_TOOL
 from modules.agent.tool_endpoint.tool import Tool, ToolCallError
 from modules.agent.tool_endpoint.turn_scope import TurnScope
 from modules.artifacts.formats import FORMATS, FORMATS_BY_KEY
+from modules.artifacts.image_edits.editable import EDITABLE_FORMATS
 from modules.artifacts.schemas import StudioJobCreate
 from modules.artifacts.script_documents.spec import DOCUMENT_FORMATS
 from modules.artifacts.service import create_artifact_job
@@ -22,8 +24,13 @@
 INSTRUCTIONS_CHARS = 2000
 
 # Office files and PDFs are scripts the agent renders, and a web page it writes
-# itself (ADR 0066): Studio's draft keeps none to edit.
-_KEYS = [fmt.key for fmt in FORMATS if fmt.key not in (*DOCUMENT_FORMATS, "html")]
+# itself (ADR 0066): Studio's draft keeps none to edit. A picture has its own tool,
+# since a draft hands the agent nothing to look at or place.
+_KEYS = [
+    fmt.key
+    for fmt in FORMATS
+    if fmt.key not in (*DOCUMENT_FORMATS, "html", *EDITABLE_FORMATS)
+]
 
 # Written out flat, and the same on every turn, so a local model's prompt cache holds.
 LISTING: dict[str, Any] = {
@@ -114,7 +121,7 @@ def start(session: Session, scope: TurnScope, arguments: dict[str, Any]) -> str:
         create_artifact_job(session, workspace, payload)
     except HTTPException as refused:
         raise ToolCallError(
-            f'Studio could not start "{fmt.label}": {_sentence(refused.detail)}'
+            f'Studio could not start "{fmt.label}": {refusal_sentence(refused.detail)}'
         ) from refused
     count = len(set(source_ids))
     sources = "source" if count == 1 else "sources"
@@ -124,15 +131,4 @@ def start(session: Session, scope: TurnScope, arguments: dict[str, Any]) -> str:
     )
 
 
-def _sentence(detail: object) -> str:
-    """Studio's reason, ended as a sentence.
-
-    A refusal the interface translates carries its reason as
-    ``{"message", "code"}``; the model reads the message."""
-    if isinstance(detail, dict) and isinstance(detail.get("message"), str):
-        detail = detail["message"]
-    text = str(detail).strip()
-    return text if text.endswith(".") else f"{text}."
-
-
 CREATE_ARTIFACT = Tool(listing=LISTING, run=start)

surfsense_local/backend/modules/agent/tool_endpoint/failed_renders.py

@@ -36,6 +36,7 @@ class Runs:
 RENDERS = Runs("render", "renders", "renders or revisions")
 REVISIONS = Runs("revision", "revisions", "renders or revisions")
 ANALYSES = Runs("analysis run", "analysis runs", "analysis runs")
+IMAGES = Runs("picture", "pictures", "pictures")
 
 _guard = threading.Condition()
 
@@ -49,6 +50,8 @@ class _Turn:
     failed: list[Runs] = field(default_factory=list)
     # Runs under way, each of which may yet fail.
     running: int = 0
+    # Artifacts still being made when their call answered: no call sees them fail.
+    queued: set[int] = field(default_factory=set)
 
 
 # A new turn drops the thread's entries, so a run still going from the last one
@@ -68,6 +71,30 @@ def begin_turn(thread_id: int) -> None:
         _guard.notify_all()
 
 
+def note_queued(runs: Runs, thread_id: int, artifact_id: int) -> None:
+    """A run whose call answered before it ended, to settle with `settle_queued`."""
+    with _guard:
+        _turns.setdefault((runs.limit, thread_id), _Turn()).queued.add(artifact_id)
+
+
+def queued_runs(runs: Runs, thread_id: int) -> frozenset[int]:
+    with _guard:
+        turn = _turns.get((runs.limit, thread_id))
+        return frozenset(turn.queued) if turn is not None else frozenset()
+
+
+def settle_queued(runs: Runs, thread_id: int, artifact_id: int, failed: bool) -> None:
+    """A queued run has ended; one that failed counts as if its call had seen it."""
+    with _guard:
+        turn = _turns.get((runs.limit, thread_id))
+        if turn is None or artifact_id not in turn.queued:
+            return
+        turn.queued.discard(artifact_id)
+        if failed:
+            turn.failed.append(runs)
+        _guard.notify_all()
+
+
 def stopped(failed: list[Runs]) -> str:
     """The refusal once a turn's limit is spent, naming what failed: "2 renders and 1 revision"."""
     counts = Counter(failed)