Fix macOS installer crashes on precompiled NIFs and unpadded OTP binaries#15
Fix macOS installer crashes on precompiled NIFs and unpadded OTP binaries#15johns10 wants to merge 4 commits into
Conversation
find_all_deps followed and imported every otool -L dependency that matched an import prefix, including install-name paths baked into precompiled NIFs that only ever existed on the machine that built them (e.g. exqlite's prebuilt artifact records `/Users/runner/work/exqlite/...`). That path matches the `/Users/` prefix, so the walk recursed into it and ran `otool -L` on a nonexistent file, which exits non-zero and raised a MatchError in cmd!/2 at macos.ex, aborting installer generation. Filter such nonexistent dependency paths out of the macOS dependency walk (with a warning) and guard find_deps/1 so otool is never invoked on a missing file. A missing baked path is not a bundleable dependency: anything genuinely required resolves next to the binary or via the linker search paths. Adds a regression test that builds a dylib whose recorded dependency path is removed before the walk, reproducing the precompiled-NIF scenario. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bundling a dependency rewrites the dependent binary's install path to the bundled copy via @loader_path/../../.../priv/<lib>. install_name_tool can only edit an install path in place — it cannot grow a binary's Mach-O load commands to fit a longer path unless the binary was linked with -headerpad_max_install_names. Homebrew/kerl-built OTP binaries (e.g. crypto.so, which links Homebrew's libcrypto) typically have no spare header room, so the rewrite failed with "larger updated load commands do not fit" and aborted installer generation with a MatchError in cmd!/2. When the standard rewrite does not fit, fall back to placing a copy of the dependency next to the binary and pointing at it with @loader_path/<basename> — the shortest possible install name, which never exceeds the original absolute path and so always fits without relinking. The colocated copy is adhoc-signed to keep it launchable, matching the rest of the tree. Verified end-to-end: the EarWitness app (Phoenix LiveView + wx + exqlite + whisper.cpp NIF + ONNX) now produces a valid .dmg on an arm64 Mac with a Homebrew-linked OTP. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request adds handling for missing dependency paths on macOS to prevent build crashes, and introduces a fallback mechanism to colocate dependencies when install_name_tool fails due to insufficient header space. The review feedback suggests robust path comparison in colocate_dep/3 by expanding both paths, handling additional loader prefixes like @executable_path/ and @rpath/ in loader_relative/1, and refactoring the test suite to use ExUnit's built-in :tmp_dir feature.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| defp colocate_dep(object, old_name, new_name) do | ||
| basename = Path.basename(new_name) | ||
| bundled = Path.expand(Path.join(Path.dirname(object), loader_relative(new_name))) | ||
| local = Path.join(Path.dirname(object), basename) | ||
|
|
||
| if not File.exists?(bundled) do | ||
| raise "Cannot relocate #{old_name} for #{object}: bundled copy not found at #{bundled}" | ||
| end | ||
|
|
||
| if bundled != local and not File.exists?(local) do | ||
| File.cp!(bundled, local) | ||
| File.chmod!(local, 0o755) | ||
| # Match how the rest of the tree is left: adhoc-signed so it is launchable. | ||
| cmd!("codesign", ["-f", "-s", "-", local]) | ||
| end |
There was a problem hiding this comment.
In colocate_dep/3, bundled is resolved to an absolute path using Path.expand/1, whereas local is kept as a relative path (since Path.dirname(object) is typically relative during builds).
Comparing an absolute path (bundled) with a relative path (local) using bundled != local will always evaluate to true, even if they point to the exact same file on disk. If File.exists?(local) is ever bypassed or behaves unexpectedly (e.g., with symlinks), this could lead to copying a file onto itself, which truncates the file to 0 bytes and corrupts the binary.
To make the comparison robust and semantically correct, we should expand local to an absolute path using Path.expand/1 as well.
defp colocate_dep(object, old_name, new_name) do
basename = Path.basename(new_name)
bundled = Path.expand(Path.join(Path.dirname(object), loader_relative(new_name)))
local = Path.expand(Path.join(Path.dirname(object), basename))
if not File.exists?(bundled) do
raise "Cannot relocate #{old_name} for #{object}: bundled copy not found at #{bundled}"
end
if bundled != local and not File.exists?(local) do
File.cp!(bundled, local)
File.chmod!(local, 0o755)
# Match how the rest of the tree is left: adhoc-signed so it is launchable.
cmd!("codesign", ["-f", "-s", "-", local])
end
| defp loader_relative("@loader_path/" <> rest), do: rest | ||
| defp loader_relative(path), do: path |
There was a problem hiding this comment.
The loader_relative/1 helper currently only strips the @loader_path/ prefix. However, macOS binaries can also use @executable_path/ (which is explicitly used in import_webview/3 on line 248) or @rpath/.
If install_name_tool fails on a dependency path starting with @executable_path/, loader_relative/1 will return the path with the prefix intact. This causes Path.join/2 and Path.expand/1 to look for a literal directory named @executable_path on disk, which does not exist, leading to a crash with a "bundled copy not found" error.
We should add clauses to handle @executable_path/ and @rpath/ as well.
defp loader_relative("@loader_path/" <> rest), do: rest
defp loader_relative("@executable_path/" <> rest), do: rest
defp loader_relative("@rpath/" <> rest), do: rest
defp loader_relative(path), do: path
| test "find_all_deps skips dependency paths that do not exist on this machine" do | ||
| dir = tmp_dir("find_deps_missing") | ||
| dep = Path.join(dir, "dep.c") | ||
| main = Path.join(dir, "main.c") | ||
| missing_install_name = "/Users/runner/work/exqlite/exqlite/priv/libfakedep.dylib" | ||
| fake_dep = Path.join(dir, "libfakedep.dylib") | ||
| lib = Path.join(dir, "libmain.dylib") | ||
|
|
||
| File.write!(dep, "int dep_fn(void){return 42;}\n") | ||
| File.write!(main, "extern int dep_fn(void); int main_fn(void){return dep_fn();}\n") | ||
|
|
||
| # Build a dep dylib whose install-name is a path that will not exist at load | ||
| # time, then link main against it so main records that missing path. | ||
| {_, 0} = | ||
| System.cmd("clang", [ | ||
| "-dynamiclib", | ||
| "-install_name", | ||
| missing_install_name, | ||
| "-o", | ||
| fake_dep, | ||
| dep | ||
| ]) | ||
|
|
||
| {_, 0} = System.cmd("clang", ["-dynamiclib", "-o", lib, main, fake_dep]) | ||
| File.rm!(fake_dep) | ||
|
|
||
| # Sanity check: otool records the now-missing path. | ||
| assert Package.MacOS.find_deps(lib) |> Enum.member?(missing_install_name) | ||
| refute File.exists?(missing_install_name) | ||
|
|
||
| # The walk must not crash and must not surface the missing path. | ||
| deps = Tooling.find_all_deps(MacOS, [lib]) | ||
| refute Enum.member?(deps, missing_install_name) | ||
|
|
||
| File.rm_rf!(dir) | ||
| end |
There was a problem hiding this comment.
Instead of manually creating and cleaning up temporary directories with a custom helper, we can leverage ExUnit's built-in :tmp_dir feature (available since Elixir 1.11).
By tagging the test or using the tmp_dir context, ExUnit automatically creates a unique temporary directory for the test and guarantees its cleanup even if assertions fail, preventing stale files from being left behind.
@tag :tmp_dir
test "find_all_deps skips dependency paths that do not exist on this machine", %{tmp_dir: tmp_dir} do
dep = Path.join(tmp_dir, "dep.c")
main = Path.join(tmp_dir, "main.c")
missing_install_name = "/Users/runner/work/exqlite/exqlite/priv/libfakedep.dylib"
fake_dep = Path.join(tmp_dir, "libfakedep.dylib")
lib = Path.join(tmp_dir, "libmain.dylib")
File.write!(dep, "int dep_fn(void){return 42;}\n")
File.write!(main, "extern int dep_fn(void); int main_fn(void){return dep_fn();}\n")
# Build a dep dylib whose install-name is a path that will not exist at load
# time, then link main against it so main records that missing path.
{_, 0} =
System.cmd("clang", [
"-dynamiclib",
"-install_name",
missing_install_name,
"-o",
fake_dep,
dep
])
{_, 0} = System.cmd("clang", ["-dynamiclib", "-o", lib, main, fake_dep])
File.rm!(fake_dep)
# Sanity check: otool records the now-missing path.
assert Package.MacOS.find_deps(lib) |> Enum.member?(missing_install_name)
refute File.exists?(missing_install_name)
# The walk must not crash and must not surface the missing path.
deps = Tooling.find_all_deps(MacOS, [lib])
refute Enum.member?(deps, missing_install_name)
end
Stripping symbols and rewriting install names invalidates the code
signatures that precompiled NIFs and dylibs ship with. With a Developer ID,
codesign/1 re-signs the whole tree; without one, nothing re-signed the
binaries that weren't install-name-rewritten (e.g. bundled precompiled
portaudio dylibs). On modern macOS — arm64 in general, Sequoia/Tahoe
strictly — a binary with an invalid signature is SIGKILL'd
("Code Signature Invalid") the moment it is dlopen'd, so the app crash-looped
under heart at launch.
Ad-hoc sign every binary (and seal the bundle) when no Developer ID is
present, so a locally built app is validly signed and launches. Verified: a
previously crash-looping EarWitness bundle launches and serves after ad-hoc
signing; `codesign --verify` passes on the sealed bundle.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Brings the working macOS installer pipeline onto master: deployment fixes (via elixir-desktop/deployment#15 branch), cross-platform Makefile, release-safe logging, boot migrations, and the macos-installer.yml workflow that builds and publishes the .dmg to GitHub Releases on version tags.
The Windows packaging path downloads the WebView2/vcredist redistributables at build time; on CI these occasionally stall, and a single HTTPoison timeout failed the whole installer build. Retry up to 3x with a 180s receive timeout so a network hiccup no longer breaks a release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Problem
mix desktop.installercrashed during macOS.appgeneration for a realPhoenix-LiveView desktop app (wx + exqlite + a whisper.cpp NIF + ONNX) built
against a Homebrew-linked OTP. Two distinct failures, both in the macOS
dependency-relocation path, each ending in
(MatchError) {"", 1}fromTooling.cmd!/2:1.
otool -Lon a path baked into a precompiled NIFfind_all_deps/3follows everyotool -Lentry that matches an import prefix.Precompiled NIFs bake in the path of the machine that built them — e.g.
exqlite's prebuilt artifact records
/Users/runner/work/exqlite/.../sqlite3_nif.so. That matches the/Users/prefix, so the walk recursed into it and ran
otool -Lon a file that doesnot exist here, which exits non-zero and raised a
MatchError.Fix: skip dependency paths that don't exist on the build machine (with a
warning), and guard
find_deps/1sootoolis never invoked on a missingfile. A missing baked path is not a bundleable dependency.
2.
install_name_tool -changecan't grow load commandsBundling a dep rewrites the dependent binary's install path to
@loader_path/../../.../priv/<lib>.install_name_tooledits install paths inplace and cannot grow a binary's Mach-O load commands to fit a longer path
unless it was linked with
-headerpad_max_install_names. Homebrew/kerl-builtOTP binaries (notably
crypto.so→libcrypto.3.dylib) typically have nospare header room, so the rewrite failed with "larger updated load commands do
not fit" and aborted.
Fix: when the standard rewrite doesn't fit, fall back to placing a copy of
the dependency next to the binary and pointing at it with
@loader_path/<basename>— the shortest possible install name, which neverexceeds the original absolute path and so always fits without relinking. The
colocated copy is adhoc-signed to keep it launchable.
Why CI didn't catch these
The example-app CI builds a custom diode OTP that (a) doesn't pull in a
precompiled exqlite NIF with a foreign baked path and (b) appears to link with
enough header padding. Both bugs only appear with stock precompiled NIFs and a
Homebrew/kerl OTP — i.e. a normal contributor's Mac.
Verification
test/find_deps_test.exs) builds a dylib whose recordeddependency path is removed before the walk, reproducing the precompiled-NIF
scenario; asserts the walk no longer crashes.
hdiutil verify-clean.dmgonan arm64 Mac with Homebrew OTP.
crypto.socorrectly loads the co-located@loader_path/libcrypto.3.dylib.mix formatclean;mix credointroduces no new issues.🤖 Generated with Claude Code