Repository navigation
Inbox mobile: attachments and annotation in the iPhone app (M2) - #1785
Merged
Merged
Conversation
backnotprop
force-pushed
the
feat/mobile-m2-attachments
branch
from
October 8, 2026 12:28
459479e to
203c131
Compare
backnotprop
force-pushed
the
feat/mobile-m1-app
branch
from
October 8, 2026 12:43
08711d5 to
975bb6c
Compare
backnotprop
force-pushed
the
feat/mobile-m2-attachments
branch
from
October 8, 2026 13:07
203c131 to
af0ee68
Compare
backnotprop
force-pushed
the
feat/mobile-m2-attachments
branch
from
October 8, 2026 13:37
af0ee68 to
59da3d9
Compare
Attachments full screen in Plannotator's bundled surface (4.1 to 4.4): a long press and Comment in the edit menu on markdown, a pin with Parent and Child on HTML, a diagram node, each saved through the device door from a native panel; the "3 annotations" sheet (3.6); the changed line and the version it sent; Share; annotations riding the next Send as Plannotator's feedback text. The shell takes bridge messages from the main frame only, page-reported pins only right after a touch, and opens links only as https or mailto after a tap. Contract section 5 gains comment_selection, export_feedback and feedback; ui gains HtmlViewer hostNavigates and ViewerHandle.takeSelection.
…aces and accents, first paint, Send with a deleted file, fixture, contract links)
…one bad address never empties the list
backnotprop
force-pushed
the
feat/mobile-m2-attachments
branch
from
October 8, 2026 13:56
59da3d9 to
4823f4c
Compare
…urface's CSP on every asset answer, http and https blocked in the web view)
…er; the viewer's sandbox spelled as the isolation test pins it
…ments # Conflicts: # AGENTS.md
…o surface without its rule list
…l-filter has no alternation)
…ments # Conflicts: # AGENTS.md
…ments # Conflicts: # AGENTS.md # apps/ios/Plannotator.xcodeproj/project.pbxproj
…ments # Conflicts: # AGENTS.md # apps/ios/Plannotator.xcodeproj/project.pbxproj # apps/ios/Plannotator/App/AppModel.swift # apps/ios/Plannotator/App/SourceSession.swift # apps/ios/project.yml
…oad says so instead of spinning, the proof traces the surface and leaves the app fresh after every class
backnotprop
force-pushed
the
feat/mobile-m2-attachments
branch
from
October 8, 2026 19:34
b14005c to
d1c4bab
Compare
…ments # Conflicts: # .github/workflows/ios.yml # AGENTS.md # apps/ios/Plannotator.xcodeproj/project.pbxproj # apps/ios/Plannotator/Thread/ReplyBar.swift # apps/ios/README.md # apps/ios/scripts/proof.ts
…computer's phones revoked); the plan's comment falls back to the toolstrip's pinpoint when the simulator raises no edit menu, naming the path
… every proof class starts from a cold install
…ments # Conflicts: # .github/workflows/ios.yml # AGENTS.md # apps/ios/Plannotator.xcodeproj/project.pbxproj # apps/ios/PlannotatorKit/Sources/PlannotatorKit/InboxClient.swift # apps/ios/project.yml # apps/ios/scripts/proof.ts
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The story
An agent's files now open on the iPhone the way they do on the desktop: full screen in Plannotator's own surface, with comments by touch (a long press and Comment in the system edit menu on markdown, a pin on an HTML page with Parent and Child, a tap on a diagram node), each written in a native sheet and saved through the device door. The comments wait in a "3 annotations" sheet and ride the next Send as Plannotator's feedback text, so the agent reads the same thing the desktop window would have sent. The page an agent wrote can neither reach the shell nor open anything on its own: bridge messages count only from the main frame, page-reported pins only right after the person's touch, and links open only as https or mailto, only after a tap.
Stacked on #1778 (M1,
feat/mobile-m1-app), with main merged in for S1 (#1777) and its follow-up (#1782). The overseer retargets this to main when M1 merges. CI has not run on this PR:ios.ymlandinbox-e2e.ymltrigger on pull requests tomain, so they first run once this PR is retargeted; the local runs below stand in until then.What changed
apps/ios/Plannotator/Attachments/SurfaceHost.swift: one warmWKWebViewloading the bundledsurface.htmlfromplannotator-surface://app/surface.html;plannotator-asset://inbox/api/html-assets/<token>/<path>served from the door'sGET html-assets(7.16) for the open page's own token only, anything else fails; theWKScriptMessageHandlerdrops any message whoseframeInfo.isMainFrameis false;pin,selectionanddraftfor an HTML page are taken only within 1.2 s of the person's own touch on the web view (aUIGestureRecognizerthat observes and never cancels), as Inbox mobile S1 follow-up: the review's items #1782 wrote into section 5;decidePolicyForcancels every navigation away from the surface except the page's ownabout:srcdocframe and frames from its folder;createWebViewWithnever makes a web view.SurfaceWebViewputs Comment first in the edit menu (buildMenu(with:)).Attachments/AttachmentCover.swift(4.1 to 4.4: the bars, the changed line, Annotate and Interact with Fit, Parent and Child, the comment panels, the saved comment with Edit and Remove, Share and the version switch in the "..." menu),AnnotationsSheet.swift(3.6, and the Edit sheet),AttachmentTiles.swift(the files at the foot of a message, 3.4A),LinkViews.swift(Safari View Controller, the mail sheet).Thread/ReplyBar.swiftgains the "3 annotations" chip;Thread/ThreadScreen.swiftthe tiles, the cover and the sheet.App/SourceSession.swift: each open thread's files and waiting annotations (cached like the thread), save and remove, and Send carryingfeedbackandannotation_idson the last reply.PlannotatorKit/Attachments.swift: the attachment and annotation wire shapes with explicit keys and an opaqueJSONValue, so Plannotator'sAnnotationcrosses the app untouched (a round-trip test pins it), and the door calls 7.14 to 7.18.packages/inbox/surface,packages/core/inbox-surface-bridge.ts):export_feedback/feedbackandcomment_selection(contract section 5, below);hostNavigateson the HTML viewer; on iOS the document's text follows Dynamic Type through-webkit-text-size-adjust(surface.css, below).packages/ui/HANDOFF.md"unreleased":HtmlViewerhostNavigates;ViewerHandle.takeSelectionwith no touch selection painted while a host composes.apps/ios/project.ymlrunsbuild:surfacewhen its sources are newer thanapps/inbox/dist/surface.htmland copies it into the app (ENABLE_USER_SCRIPT_SANDBOXING: NOon the app target: the phase runs Vite over the repo's packages).WarningandAccentjoin the generated colours.apps/ios/PlannotatorUITests/AttachmentProofTests.swift, the shared helpers moved intoProofSupport.swift(M1's test unchanged in what it does),scripts/proof.tsgains/attachand/edit-plan, an--onlyflag and the data-dir assertion (row 5109),scripts/fixtures/the three files..github/workflows/ios.ymlstays one job; its paths add the surface's files.apps/ios/README.md, the contract.The review's fixes (REVIEW-PLANNOTATOR-MOBILE-M2-2026-10-08, one push)
ViewerHandle.revealPin, bridge messageplannotator-bridge-reveal-pin, HANDOFF "unreleased"). The proof asserts the pinned button is on screen and wholly above the switch after the pin, after Parent and after Child (4.3-pin,4.3-pin-parent).my pic.pngwent out asmy%2520pic.png. The path is now decoded once (url.path(percentEncoded: false)) and encoded once when the door URL is built. The ticket page's three "going" photos come from its folder asgoing/mira.png,going/dj tern.pngandgoing/joão.png; the page writes how many drew and the proof asserts "Photos: 3 of 3".open_attachment); the recording shows white, then the plan..btnxrule:font-size: 18px; font-weight: 900as in the record; the 4.3 frames are retaken.createWebViewWith, within the touch window; the unreachable.linkActivatedarms are gone). Also from the items: the folder token is read from the door's<base href>only, and an edit made in the 3.6 sheet over an open file reaches the surface.packages/uiandpackages/guide-viewer, the 4.4 node ring and zoom control under the panel, heading tracking at the largest type size.plannotator-ops's review: agent HTML cannot phone home (one push)
A page the agent's page embeds from its own folder (
<iframe src="extra.html">) loaded throughplannotator-asset:with no policy of its own, so its script could send an image beacon. Now every asset answer carries the surface's own CSP (no network; scripts, styles, images, fonts, media and frames from the asset scheme,data:andblob:only), and the web view carries aWKContentRuleListthat blocks every http and https load. One exception in that list, named: the request to open a new window, which is how a tapped link reachescreateWebViewWith; no web view is ever made for it, and the URL goes to Safari View Controller. Proof: the ticket page embedsvenue-notes.htmlfrom its folder; its script runs (the test reads the text it writes) and sends an image beacon and afetchto the proof script's own loopback server; the server records none. Contract section 5 and CLAUDE.md say so.plannotator-ops's delta review (one push)
wsandwsstoo (adata:orblob:frame may not inherit the CSP); the nested embed's proof now also opens a WebSocket to the proof's loopback server, which records nothing.proof-run-rule-list-refused.log); ws and wss are now a rule of their own. No standing test forces the failure, since that needs a hook in the app."allow-scripts" + (hostNavigates ? " allow-popups" : ""), so the isolation test's pin onsandbox: "allow-scripts"holds (it failed CI at 4823f4c); the proof scrolls the commented line clear of the toolstrip before the long press (it hit the toolstrip on the slower CI runner).The CI fix (run 37826449162)
The attachments proof failed in CI on the first open of the plan: the spinner for 100 s, the surface never reported ready; not reproduced locally (on a warm build folder or a fresh one); the trace names the cause next time. What changed:
The surface starts loading when a thread's files show, not when one is opened.
If the surface page itself fails to load, the screen says "Files can't be shown on this iPhone right now." instead of spinning.
With
-PlannotatorProofthe screen carries a trace: loads, web content restarts, the rule list's state, ready, the load error. A failing expect prints it in the screen dump.The proof's first wait is main's
ProofWait.opening(120 s, from M3's cold-runner measurements) instead of 30 s.Every proof class of this kind (
ProofCase) starts from the app reinstalled from the run's own build and the computer's paired phones revoked (/reset-app), and resets again when it fails, so no class starts paired because the one before it failed partway. The first version only deleted a preferences domain outside the app's container, which left the app paired: in run on dc03ea9 Decisions and ProofTests failed at their first lines after the attachments class failed. Proved locally by forcing the attachments class to fail right after pairing: ProofTests then passed.The plan's comment: on the CI simulator a synthesized long press raised no edit menu in 120 s (run on dc03ea9; the trace showed the surface ready). The proof presses again once, and if the menu still does not come, comments through the toolstrip's pinpoint (a tap on the line opens the same sheet); the log and the results name the path that ran ("Comment on the plan through ..."). Both paths pass locally (the pinpoint path forced once). The edit menu itself is still proved where the simulator raises it, and on the device.
/derived*/is ignored at the repo root: a build folder kept in a worktree was committed once and taken out of the history.On the CI simulator the surface twice loaded and never said ready (trace on c0c4c60:
loads=1 ended=0 rules=ok ready=false error=noneafter 120 s; not reproduced locally). The shell now loads the surface again when someone has waited 30 s on a silent load, twice at most; the trace counts the loads.Record sections made real
3.4A (files and the annotations chip), 3.6, 4.1, 4.2, 4.3, 4.4, light and dark; contract section 5 (both schemes, main frame only, the touch window, links), exchanges 7.14 to 7.18 and 7.11 with
feedback; BRIEF section 7 "An attachment"; row 5120 (no URL from content reaches the app's handler).Proofs and how to run them
bun apps/ios/scripts/proof.ts --binary .local/plannotator(afterbun run --cwd apps/review build && bun run build:hook && bun build apps/hook/server/index.ts --compile --no-compile-autoload-bunfig --define '__CLI_VERSION__="0.0.0-dev"' --outfile .local/plannotator), iPhone 17 simulator on iOS 26.5, Xcode 26.6, on this branch rebased onto M1's head (7f714b7) with main merged in, after the review's fixes: both XCUITest flows pass in one run (M2's in 213 s, M1's in 241 s).--only PlannotatorUITests/AttachmentProofTestsruns M2's alone. What M2's asserts, against the compiled Inbox and an agent throughinbox mcp:retry-plan.md, the record's Low Tideticket-page.html(rebuilt from the iPhone record'spartyPage()inindex.html, with a footer of two links and a script that tries the forgeries) andinstall-flow.mmd.httpsopens Safari View Controller on docs.stripe.com; aplannotator://pairlink opens nothing (no pairing, no Safari, the file still on screen).location.href, 1.5 s after load with nobody touching: no panel opens, the shell counted 5 frame messages and 1 untouched pin dropped, the page stays.plannotator://pair) opens nothing.wait_for_replybody namesretry-plan.md,ticket-page.htmlandinstall-flow.mmdand carries all three comments; the chip goes.Light and dark frames of every screen beside their renders, and the recording of the comment flow, are in
research/operational/mobile-proofs/M2-2026-10-08/in the Workspaces meta repo (built/,compare/,M2-comment-flow.mp4).Also run after the fixes:
tests/e2e/surface.spec.ts(9 of 9, Playwright WebKit against the rebuilt surface and the compiled Inbox);swift testinPlannotatorKit(the record round trip with the rest).Gates
bun run typecheck;bun test packages/ui/components/html-viewer packages/ui/hooks packages/core packages/inbox(393 pass);DOM_TESTS=1 bun test --isolateon every html-viewer, Viewer and highlighter test file (376 tests, 0 fail);bun test tests/entry-assets.test.ts tests/favicon-surfaces.test.ts;bun run --cwd packages/ui smoke:package;bun run check:release-version;bun apps/ios/scripts/gen-colors.ts --check; Gitleaks on the range.Contract changes (section 5, edited in this PR)
export_feedback/feedback: Send'sfeedbacktext is Plannotator's export, which needs Plannotator's parser for line numbers; the phone has it only inside the surface, so the surface builds it with the window's ownattachmentFeedbackinstead of a second implementation in Swift.comment_selection: on the iPhone the viewer painted a markdown selection as a pending highlight when it settled, and the library painted it attouchend; either replaced the system selection and closed the edit menu before the person reached Comment (seen in the simulator). On a touch screen the selection now stays the system's until Comment asks for it..linkActivatedframe navigation. The surface's CSP (frame-src plannotator-asset: about: data: blob:) refuses that load before WebKit asks the shell, so a tapped link could never open. WithhostNavigatesa person's tap (isTrusted) opens the link as a new window instead, whichcreateWebViewWithopens only within the touch window and never makes a web view; the frame never navigates. Written into the section.Deviations from the record, each with its reason
HtmlViewerdraws a page at the phone's width; a desktop-width page is not laid out at its own width here.--background(light grey) where 4.4 draws white.proof.ts), not a named one.-PlannotatorProofthe attachment screen carries an invisible element with the shell's dropped and cancelled counts, which the test reads.zoomto font sizes (the spacing grew, the letters did not: measured in the simulator, computed 22 px at both sizes), so S1's Dynamic Type did nothing to the document's text on a phone. The surface now also sets-webkit-text-size-adjustby the same factor on the document; desktop WebKit ignores it (the S1 spec still passes). The guide reader's card (.sf-gcard) has the same zoom and is left for M4.What it leaves out
Multi-block selections, multi-target HTML comments, Pencil, guides (M4). Fetching
viewagain when an asset answers 404 after the Inbox restarted while a page is open (reopening the file does it). Remote images and web fonts in an agent's page (no network by design). What needs the device: the edit menu's feel under a finger, a finger pin and Parent and Child on the Low Tide page, a node tap on a dense diagram, the mail sheet with an account, haptics, the 20 MB surface's first open on the iPhone 15.Limits
One number, from the contract: a page-reported pin, selection or draft, or a link, counts within 1.2 s of the person's touch on the web view. No others.