Skip to content

Fix macOS installer crashes on precompiled NIFs and unpadded OTP binaries#15

Open
johns10 wants to merge 4 commits into
mainfrom
fix/macos-otool-missing-precompiled-nif-path
Open

Fix macOS installer crashes on precompiled NIFs and unpadded OTP binaries#15
johns10 wants to merge 4 commits into
mainfrom
fix/macos-otool-missing-precompiled-nif-path

Conversation

@johns10

@johns10 johns10 commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

Problem

mix desktop.installer crashed during macOS .app generation for a real
Phoenix-LiveView desktop app (wx + exqlite + a whisper.cpp NIF + ONNX) built
against a Homebrew-linked OTP. Two distinct failures, both in the macOS
dependency-relocation path, each ending in (MatchError) {"", 1} from
Tooling.cmd!/2:

1. otool -L on a path baked into a precompiled NIF

find_all_deps/3 follows every otool -L entry that matches an import prefix.
Precompiled NIFs bake in the path of the machine that built them — e.g.
exqlite's prebuilt artifact records
/Users/runner/work/exqlite/.../sqlite3_nif.so. That matches the /Users/
prefix, so the walk recursed into it and ran otool -L on a file that does
not exist here, which exits non-zero and raised a MatchError.

Fix: skip dependency paths that don't exist on the build machine (with a
warning), and guard find_deps/1 so otool is never invoked on a missing
file. A missing baked path is not a bundleable dependency.

2. install_name_tool -change can't grow load commands

Bundling a dep rewrites the dependent binary's install path to
@loader_path/../../.../priv/<lib>. install_name_tool edits install paths in
place and cannot grow a binary's Mach-O load commands to fit a longer path
unless it was linked with -headerpad_max_install_names. Homebrew/kerl-built
OTP binaries (notably crypto.solibcrypto.3.dylib) typically have no
spare header room, so the rewrite failed with "larger updated load commands do
not fit" and aborted.

Fix: when the standard rewrite doesn't fit, fall back to placing a copy of
the dependency next to the binary and pointing at it with
@loader_path/<basename> — the shortest possible install name, which never
exceeds the original absolute path and so always fits without relinking. The
colocated copy is adhoc-signed to keep it launchable.

Why CI didn't catch these

The example-app CI builds a custom diode OTP that (a) doesn't pull in a
precompiled exqlite NIF with a foreign baked path and (b) appears to link with
enough header padding. Both bugs only appear with stock precompiled NIFs and a
Homebrew/kerl OTP — i.e. a normal contributor's Mac.

Verification

  • New regression test (test/find_deps_test.exs) builds a dylib whose recorded
    dependency path is removed before the walk, reproducing the precompiled-NIF
    scenario; asserts the walk no longer crashes.
  • End-to-end: a full app now produces a valid, hdiutil verify-clean .dmg on
    an arm64 Mac with Homebrew OTP. crypto.so correctly loads the co-located
    @loader_path/libcrypto.3.dylib.
  • mix format clean; mix credo introduces no new issues.

🤖 Generated with Claude Code

johns10 and others added 2 commits July 11, 2026 16:46
find_all_deps followed and imported every otool -L dependency that matched
an import prefix, including install-name paths baked into precompiled NIFs
that only ever existed on the machine that built them (e.g. exqlite's
prebuilt artifact records `/Users/runner/work/exqlite/...`). That path
matches the `/Users/` prefix, so the walk recursed into it and ran
`otool -L` on a nonexistent file, which exits non-zero and raised a
MatchError in cmd!/2 at macos.ex, aborting installer generation.

Filter such nonexistent dependency paths out of the macOS dependency walk
(with a warning) and guard find_deps/1 so otool is never invoked on a
missing file. A missing baked path is not a bundleable dependency: anything
genuinely required resolves next to the binary or via the linker search
paths.

Adds a regression test that builds a dylib whose recorded dependency path
is removed before the walk, reproducing the precompiled-NIF scenario.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bundling a dependency rewrites the dependent binary's install path to the
bundled copy via @loader_path/../../.../priv/<lib>. install_name_tool can
only edit an install path in place — it cannot grow a binary's Mach-O load
commands to fit a longer path unless the binary was linked with
-headerpad_max_install_names. Homebrew/kerl-built OTP binaries (e.g.
crypto.so, which links Homebrew's libcrypto) typically have no spare header
room, so the rewrite failed with "larger updated load commands do not fit"
and aborted installer generation with a MatchError in cmd!/2.

When the standard rewrite does not fit, fall back to placing a copy of the
dependency next to the binary and pointing at it with @loader_path/<basename>
— the shortest possible install name, which never exceeds the original
absolute path and so always fits without relinking. The colocated copy is
adhoc-signed to keep it launchable, matching the rest of the tree.

Verified end-to-end: the EarWitness app (Phoenix LiveView + wx + exqlite +
whisper.cpp NIF + ONNX) now produces a valid .dmg on an arm64 Mac with a
Homebrew-linked OTP.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds handling for missing dependency paths on macOS to prevent build crashes, and introduces a fallback mechanism to colocate dependencies when install_name_tool fails due to insufficient header space. The review feedback suggests robust path comparison in colocate_dep/3 by expanding both paths, handling additional loader prefixes like @executable_path/ and @rpath/ in loader_relative/1, and refactoring the test suite to use ExUnit's built-in :tmp_dir feature.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread lib/package/macos.ex
Comment on lines +380 to +394
defp colocate_dep(object, old_name, new_name) do
basename = Path.basename(new_name)
bundled = Path.expand(Path.join(Path.dirname(object), loader_relative(new_name)))
local = Path.join(Path.dirname(object), basename)

if not File.exists?(bundled) do
raise "Cannot relocate #{old_name} for #{object}: bundled copy not found at #{bundled}"
end

if bundled != local and not File.exists?(local) do
File.cp!(bundled, local)
File.chmod!(local, 0o755)
# Match how the rest of the tree is left: adhoc-signed so it is launchable.
cmd!("codesign", ["-f", "-s", "-", local])
end

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

In colocate_dep/3, bundled is resolved to an absolute path using Path.expand/1, whereas local is kept as a relative path (since Path.dirname(object) is typically relative during builds).

Comparing an absolute path (bundled) with a relative path (local) using bundled != local will always evaluate to true, even if they point to the exact same file on disk. If File.exists?(local) is ever bypassed or behaves unexpectedly (e.g., with symlinks), this could lead to copying a file onto itself, which truncates the file to 0 bytes and corrupts the binary.

To make the comparison robust and semantically correct, we should expand local to an absolute path using Path.expand/1 as well.

  defp colocate_dep(object, old_name, new_name) do
    basename = Path.basename(new_name)
    bundled = Path.expand(Path.join(Path.dirname(object), loader_relative(new_name)))
    local = Path.expand(Path.join(Path.dirname(object), basename))

    if not File.exists?(bundled) do
      raise "Cannot relocate #{old_name} for #{object}: bundled copy not found at #{bundled}"
    end

    if bundled != local and not File.exists?(local) do
      File.cp!(bundled, local)
      File.chmod!(local, 0o755)
      # Match how the rest of the tree is left: adhoc-signed so it is launchable.
      cmd!("codesign", ["-f", "-s", "-", local])
    end

Comment thread lib/package/macos.ex
Comment on lines +399 to +400
defp loader_relative("@loader_path/" <> rest), do: rest
defp loader_relative(path), do: path

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The loader_relative/1 helper currently only strips the @loader_path/ prefix. However, macOS binaries can also use @executable_path/ (which is explicitly used in import_webview/3 on line 248) or @rpath/.

If install_name_tool fails on a dependency path starting with @executable_path/, loader_relative/1 will return the path with the prefix intact. This causes Path.join/2 and Path.expand/1 to look for a literal directory named @executable_path on disk, which does not exist, leading to a crash with a "bundled copy not found" error.

We should add clauses to handle @executable_path/ and @rpath/ as well.

  defp loader_relative("@loader_path/" <> rest), do: rest
  defp loader_relative("@executable_path/" <> rest), do: rest
  defp loader_relative("@rpath/" <> rest), do: rest
  defp loader_relative(path), do: path

Comment thread test/find_deps_test.exs
Comment on lines +23 to +58
test "find_all_deps skips dependency paths that do not exist on this machine" do
dir = tmp_dir("find_deps_missing")
dep = Path.join(dir, "dep.c")
main = Path.join(dir, "main.c")
missing_install_name = "/Users/runner/work/exqlite/exqlite/priv/libfakedep.dylib"
fake_dep = Path.join(dir, "libfakedep.dylib")
lib = Path.join(dir, "libmain.dylib")

File.write!(dep, "int dep_fn(void){return 42;}\n")
File.write!(main, "extern int dep_fn(void); int main_fn(void){return dep_fn();}\n")

# Build a dep dylib whose install-name is a path that will not exist at load
# time, then link main against it so main records that missing path.
{_, 0} =
System.cmd("clang", [
"-dynamiclib",
"-install_name",
missing_install_name,
"-o",
fake_dep,
dep
])

{_, 0} = System.cmd("clang", ["-dynamiclib", "-o", lib, main, fake_dep])
File.rm!(fake_dep)

# Sanity check: otool records the now-missing path.
assert Package.MacOS.find_deps(lib) |> Enum.member?(missing_install_name)
refute File.exists?(missing_install_name)

# The walk must not crash and must not surface the missing path.
deps = Tooling.find_all_deps(MacOS, [lib])
refute Enum.member?(deps, missing_install_name)

File.rm_rf!(dir)
end

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Instead of manually creating and cleaning up temporary directories with a custom helper, we can leverage ExUnit's built-in :tmp_dir feature (available since Elixir 1.11).

By tagging the test or using the tmp_dir context, ExUnit automatically creates a unique temporary directory for the test and guarantees its cleanup even if assertions fail, preventing stale files from being left behind.

  @tag :tmp_dir
  test "find_all_deps skips dependency paths that do not exist on this machine", %{tmp_dir: tmp_dir} do
    dep = Path.join(tmp_dir, "dep.c")
    main = Path.join(tmp_dir, "main.c")
    missing_install_name = "/Users/runner/work/exqlite/exqlite/priv/libfakedep.dylib"
    fake_dep = Path.join(tmp_dir, "libfakedep.dylib")
    lib = Path.join(tmp_dir, "libmain.dylib")

    File.write!(dep, "int dep_fn(void){return 42;}\n")
    File.write!(main, "extern int dep_fn(void); int main_fn(void){return dep_fn();}\n")

    # Build a dep dylib whose install-name is a path that will not exist at load
    # time, then link main against it so main records that missing path.
    {_, 0} = 
      System.cmd("clang", [
        "-dynamiclib",
        "-install_name",
        missing_install_name,
        "-o",
        fake_dep,
        dep
      ])

    {_, 0} = System.cmd("clang", ["-dynamiclib", "-o", lib, main, fake_dep])
    File.rm!(fake_dep)

    # Sanity check: otool records the now-missing path.
    assert Package.MacOS.find_deps(lib) |> Enum.member?(missing_install_name)
    refute File.exists?(missing_install_name)

    # The walk must not crash and must not surface the missing path.
    deps = Tooling.find_all_deps(MacOS, [lib])
    refute Enum.member?(deps, missing_install_name)
  end

Stripping symbols and rewriting install names invalidates the code
signatures that precompiled NIFs and dylibs ship with. With a Developer ID,
codesign/1 re-signs the whole tree; without one, nothing re-signed the
binaries that weren't install-name-rewritten (e.g. bundled precompiled
portaudio dylibs). On modern macOS — arm64 in general, Sequoia/Tahoe
strictly — a binary with an invalid signature is SIGKILL'd
("Code Signature Invalid") the moment it is dlopen'd, so the app crash-looped
under heart at launch.

Ad-hoc sign every binary (and seal the bundle) when no Developer ID is
present, so a locally built app is validly signed and launches. Verified: a
previously crash-looping EarWitness bundle launches and serves after ad-hoc
signing; `codesign --verify` passes on the sealed bundle.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
johns10 added a commit to Code-My-Spec/ear_witness that referenced this pull request Jul 11, 2026
Brings the working macOS installer pipeline onto master: deployment fixes
(via elixir-desktop/deployment#15 branch), cross-platform Makefile,
release-safe logging, boot migrations, and the macos-installer.yml workflow
that builds and publishes the .dmg to GitHub Releases on version tags.
The Windows packaging path downloads the WebView2/vcredist redistributables at
build time; on CI these occasionally stall, and a single HTTPoison timeout
failed the whole installer build. Retry up to 3x with a 180s receive timeout so
a network hiccup no longer breaks a release.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant