Repository navigation
feat(preview): open heic, heif and avif images - #4009
Merged
Merged
Conversation
dschmidt
force-pushed
the
feat/preview-heic-avif
branch
from
October 24, 2026 06:11
dc7b350 to
f3c46ca
Compare
dschmidt
force-pushed
the
feat/preview-heic-avif
branch
2 times, most recently
from
October 24, 2026 06:50
48e671d to
1cb37eb
Compare
dschmidt
force-pushed
the
feat/preview-heic-avif
branch
4 times, most recently
from
October 24, 2026 08:04
af6f04f to
54100f9
Compare
dschmidt
marked this pull request as ready for review
October 24, 2026 09:05
dschmidt
force-pushed
the
feat/preview-heic-avif
branch
from
October 26, 2026 08:53
6e12172 to
0a04d36
Compare
dschmidt
force-pushed
the
feat/preview-heic-avif
branch
from
October 26, 2026 09:15
0a04d36 to
2c7ebd6
Compare
The viewer never hands the original bytes of a raster image to the browser, it shows a preview the server rendered. So whether a heic opens is not a question of the browser's decoders or of a format list compiled into the app, but of whether the server has a preview for that file, and the server already says so per file via `oc:has-preview` and the graph thumbnails relationship. The preview app therefore claims heic, heif and avif like any other image and decides per resource. `canOpen` decides whether the viewer takes a file at all, from the file and the server only, so the action and the carousel come out the same in every browser: audio, video, svg and vault images are taken as they are, every other image only if `hasPreview()` says yes. A new `canOpen` hook on `ApplicationFileExtension` carries that into the file actions, so on a server whose thumbnailer does not read a format the editor stays away and download takes over, instead of opening into an empty stage. A jpeg the server has no preview for is treated the same way, it had nothing to show before either. `canRender` then decides in the viewer whether this browser can show a file it took. Only vault images depend on it: the browser gets their decrypted original, and whether it reads heic, avif or tiff differs between browsers. That is probed once when the app is set up, by decoding a 2x2 image of each format that ships inlined with the app, and read synchronously from then on. A vault image in a format the browser doesn't read is not fetched at all, the viewer says so instead. Two more things are reported as errors now rather than left blank: a resolved load without a url, and an original the browser fails to decode after all, via an `error` event on the image components. heic, heif and avif also get the image icon in the file list instead of the preview app's own icon.
The fixtures are a 128x96 test pattern encoded with heif-enc, so there is nothing to license. In a folder both get a thumbnail and open in the viewer, which needs the server to render previews for them: the CI stack gets an imagor service for the new preview suite, announced the same way as in the dev stack. In a vault the browser gets the original, so what happens depends on its decoders. An avif opens in every browser. A heic opens only in a browser that decodes heic, and is otherwise listed but not fetched, the viewer says so. Both are their own scenario, tagged @heic-decoder and @no-heic-decoder, and the playwright config picks the one that fits the project: chromium and firefox never decode heic, WebKit does on macOS through the system image frameworks and not elsewhere. So the suite runs on a developer's mac as well as in CI.
The pinned commit predates opencloud-eu/opencloud#3397, so the server built for CI ignored WEBDAV_THUMBNAILS_GENERATOR_URL and OC_THUMBNAILS_ADDITIONAL_SOURCE_TYPES and the preview suite found no thumbnails. Pinned to the current main.
canRender answers false for a probed format until the probes finished, so a vault avif, heic, heif or tiff that loaded before that was marked unsupported and never fetched. performLoad now awaits the probes and sets isUnsupported from the result, so a file that recovered no longer shows the browser message for an unrelated decode error.
kulmann
approved these changes
Oct 26, 2026
kulmann
enabled stacked auto-merge
October 26, 2026 11:23
1 of 7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Stacked on #3997, which makes the dev stack render previews for
heic,heif,avifand the two HEIF image sequence types through imagor and announce them. This PR makes the web client open those files.The viewer never hands the original bytes of a raster image to the browser, it shows a preview the server rendered. So whether a heic opens is not a question of the browser's decoders or of a format list compiled into the app, but of whether the server has a preview for that file, and the server already says so per file via
oc:has-previewand the graph thumbnails relationship.image/avif,image/heic,image/heic-sequence,image/heifandimage/heif-sequencelike any other image. All five are in reva's extension-to-type table (avif,heic,heics,heif,heifs), so the server reports them for uploaded files..hif, the extension Canon and Sony cameras use for HEIF, is added to that table in Map .hif files to image/heif reva#911.canOpendecides whether the viewer takes a file at all, from the file and the server only, so the "open with" action and the carousel come out the same in every browser: audio, video, svg and vault images are taken as they are, every other image only ifhasPreview()says yes.canRenderdecides in the viewer whether this browser can show a file it took.ApplicationFileExtensiongets acanOpenhook, evaluated inuseFileActions, so an app can narrow down the files of a claimed type it is offered for. The preview app passes itscanOpen. On a server whose thumbnailer does not read a format the editor stays away and download takes over, instead of opening into an empty stage. A jpeg the server has no preview for is treated the same way, it had nothing to show before either.useBrowserImageSupportprobes that once per viewer, by decoding a 2x2 image of each of the three formats. The probe images live insrc/assets/probesand are imported with?inline, so they ship inside the bundle and no request leaves the browser;assetsIncludein the shared vite config teaches vite that.heicand.tifare assets. A vault image in a format the browser doesn't read is not fetched at all, the viewer says "cannot be displayed in this browser" instead.vaultMimeType.ts), that table now knowsavif,heic,heics,heif,heifs,hif,tifandtiff. Without it a heic in a vault wasapplication/octet-streamand did not open at all.errorevent onMediaImageandMediaMotionPhoto.avif,heicandheifget the image icon in the file list instead of the preview app's own icon.Supersedes #3187, which gated the same formats with a compiled-in
requiresServerPreviewflag.Needs a server image newer than the merge of opencloud-eu/opencloud#3397, see #3997.
Docs: the new
canOpenhook onApplicationFileExtensionis public extension API and needs a mention in the developer docs (https://docs.opencloud.eu/docs/dev/web/, where the file extensions of an app are described), next tomimeType,extension,routeNameandsecureView: signature(resource: Resource) => boolean, called after the type matched, false keeps the app out of the "open with" actions for that one file.Follow-up, not part of this PR: every vault upload scenario in
rclone-crypt/fails in Playwright WebKit, the new ones included, in the pre-existing upload and save steps. They prove the encryption by reading the request body throughpage.route()and looking for theRCLONEmagic, and Playwright's WebKit driver returnsnullfrompostDataBuffer()for Blob bodies (fetchand XHR alike, checked with a probe in both browsers), which is what Uppy/tus sends. That is the driver, not the OS, so it holds on a mac too, and it keeps the@heic-decoderscenario below from ever reaching the viewer in WebKit. CI runsrclone-cryptin Chromium only, so it does not show there. The fix is isolated: one API helper that lists the vault folder over WebDAV (ciphertext names), fetches each blob and checks it starts withRCLONEand holds no cleartext, replacing the threepage.route()blocks inresource/upload.ts(two) andresource/viewer.ts(one), plus passing theuserinto those page object calls. Roughly 60 lines in the e2e support code, no product change.Related Issue
How Has This Been Tested?
opencloudeu/opencloud-rolling:dailywithshumc/imagor:1.9.11and the announced types from feat(dev): make previews with imagor and announce heic, heif and avif #3997; the probe images decoded in both browsers via Playwright (png and avif load, heic and tiff fireerror, in Linux WebKit too)samplePhoto.heicandsamplePhoto.avif: a 128x96 test pattern from ffmpeg, encoded with heif-enc, so there is nothing to licensepreview/serverRenderedImages.feature: a heic and an avif uploaded to a folder show a thumbnail in the list and a preview in the sidebar, and open in the media viewer. New CI suitepreviewwith an imagor service and the announced types, like the dev stack. Passes in Chromium and WebKitrclone-crypt/vault.feature, three scenarios: an avif in a vault opens from the original in every browser; a heic in a vault opens from the original in a browser that decodes heic (@heic-decoder); a heic in a vault is listed, opens, and the viewer says "cannot be displayed in this browser" without a single GET for the file (@no-heic-decoder). The playwright config picks the tagged scenario per project: Chromium and Firefox never decode heic, WebKit does on macOS through the system image frameworks and not on Linux, so the suite can run on a developer's mac as well as in CI. Verified here in Chromium; in WebKit they hit the follow-up above before reaching the viewer, and the@heic-decoderscenario additionally needs a machas-previewis in the carousel and gets the preview action, one without is left out and falls back to download; svg, vault images, audio and video stay in the carousel without a server preview, a vault heic included; a vault image in a format the browser doesn't read is not fetched and shows the unsupported state; the browser probe takes jpeg, png, gif and webp for granted, reports what the browser decoded for avif, heic and tiff, lets heif and the sequence types follow heic; a vaultIMG_0001.HEICgetsimage/heic.hifis typedapplication/octet-streamby the server and getshas-preview0 without Map .hif files to image/heif reva#911, so it stays download-only herefiles.thumbnailin its capabilities, so the announced types cannot be read from therehas-preview/thumbnailsfor heic; a heic live photo in the viewer; Firefox was not run locallyTypes of changes