Skip to content

perry-codegen: parallel-module codegen race produces colliding object files (consumer_type_only.o ≡ consumer_value.o) #509

Closed

Description

@proggeramlug

Background

Surfaced after v0.5.589 (closes #493 — closure rest registry). The test_issue_446_import_type_method_typeof test moved from a parity-mismatch (typeof returned 'object' instead of 'function') to a hard compile failure where the per-module `.o` files for two siblings have IDENTICAL CONTENT.

Repro

rm -rf .perry-cache target/perry-auto-* fixtures_issue_446_pkg_*.o
PERRY_NO_OBJECT_CACHE=1 ./target/release/perry compile \
  test-files/test_issue_446_import_type_method_typeof.ts -o /tmp/t446
Undefined symbols for architecture arm64:
  "_fixtures_issue_446_pkg_consumer_type_only_ts__init", referenced from:
      _main in test_issue_446_import_type_method_typeof_ts.o
  "_perry_fn_fixtures_issue_446_pkg_consumer_type_only_ts__consumeTypeOnly", referenced from:
      _main in test_issue_446_import_type_method_typeof_ts.o

`nm` shows the bug:

=== fixtures_issue_446_pkg_consumer_type_only_ts.o
T ___perry_init_strings_fixtures_issue_446_pkg_consumer_value_ts   ← WRONG
T _fixtures_issue_446_pkg_consumer_value_ts__init                   ← WRONG
T ___perry_wrap_perry_fn_fixtures_issue_446_pkg_consumer_value_ts__consumeValue
                                                                     ← WRONG

=== fixtures_issue_446_pkg_consumer_value_ts.o
(same symbols)

Both files have IDENTICAL content with consumer_VALUE's symbols. Whatever thread wins the race produces the consumer_value bytes, and that gets persisted to BOTH filenames. `consumer_inline_type_ts.o` (the third sibling) has the correct inline_type symbols — it must produce a different rayon scheduling shape.

Why now

The 4 fixture modules share a long common prefix (`fixtures_issue_446_pkg_consumer_*`). Possible causes:

  1. Hash collision in object cache — but `PERRY_NO_OBJECT_CACHE=1` reproduces, so this isn't it.
  2. Filename truncation / mistaking — perry's per-module output filename comes from `sanitize(hir.name)`. Both produce different names (`consumer_type_only_ts` vs `consumer_value_ts`), so straight collision is unlikely.
  3. rayon parallel codegen race — perry compiles modules in parallel via rayon. If a shared mutable state is racing, the same compiled bytes could be written to both output files. After v0.5.589 the runtime-side `CLOSURE_REST_REGISTRY` is thread_local — but module init emits `js_register_closure_rest` calls into the LL text. The codegen-side `closure_rest_params` HashMap is per-module and shouldn't race. But the `emit_string_pool` function reads strings.iter() and emits init-function bodies — if two modules' codegen threads share an `StringPool` or LlModule by accident, they'd both write to the same memory.
  4. Object cache writeback race — even with PERRY_NO_OBJECT_CACHE=1, the keep-intermediates path writes `.o` to disk. Two threads could swap filenames if the in-memory bytes are passed by reference instead of being owned.

The symptoms (perfect content collision, surviving cache disable) point at (3) or (4). Untriaged which one.

Workaround

Pre-v0.5.589 the same test was a parity-mismatch but compiled cleanly. The fix in v0.5.589 doesn't logically interact with cross-module `.o` content — most likely the race was already present and v0.5.589's added codegen calls just timing-shifted it into the failure window.

Tracked in `test-parity/known_failures.json` so the parity gate still passes the release; the failure is non-blocking for #421/#420 followups but blocks any future CI tightening.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions