Sharp Synth is a desktop UI for running Apple's Sharp monocular view synthesis pipeline locally. Currently focusing on MacOS, but might adapt to windows or linux in the future if there is sufficient interest.
- Download the latest release zip from GitHub Releases and unzip it.
- Open the app and choose a model:
- Download new copy (saves into the app folder), or
- Choose different file (use a model you already downloaded).
- Drag images into the dropzone or use Try examples.
Outputs are saved to ~/Documents/Sharp Synth Outputs by default and can be changed in Setup.
You can use the "Open Outputs" button to open the folder, or use the "Reveal" button in the output panel to reveal the generated .ply file. You can transfer this to a Vision Pro over Airdrop and view in software like MetalSplatter (https://github.com/scier/MetalSplatter)
Output files are rotated to try and render upright in MetalSplatter's default orientation.
It could be useful for the desktop application to also start a webserver in order to submit images for processing via a browser from within VisionOS, and download ply files directly without needing to airdrop them.
There may also be utiltiy in adding an interface to generate images via image generation models from the same interfaces (either running locally or by entering an API key) to step into new images, or to explore alterations to existing images.
This could allow for visualising how a room might look with different interior design, for example, using an image and a prompt. If this would be worthwhile, it may make sense to write a VisionOS app that communicates with the webserver on MacOS and also wraps or extends MetalSplatter itself to generate a splat from an image, view it, and prompt alterations to it in natural language text or voice.
cd electron-app && bun installcd electron-app && bun start./scripts/package_release.shto rebuild icons, PyApp, and release zips.
- Use Copy log in the Live console and attach the log output when opening a GitHub issue.
- Model weights are not bundled; the download is opt-in and subject to Apple's research-only license.
- The preview renderer in the gallery seems to be quite slow. You may want to hit the pause button if it begins to hang. Might fix this down the line. Also the output preview image is tricky to rotate and present correctly for various images.
- You can open ply files in MacOS preview, but they may open upside down as MetalSplatter's default orientation is different from MacOS Preview. Preview is much faster, though.
If you have feedback, you are welcome to either open an issue on this repo or to contact me on X where I am @Digital_Reverie (link in app).
See THIRD_PARTY_NOTICES.md for third-party licenses and notices.