Skip to content

Improve startup fault capture, attribution and experiment reports - #34

Draft
benjaminRomano wants to merge 3 commits into
mainfrom
codex/reliable-startup-fault-attribution
Draft

benjaminRomano wants to merge 3 commits into
mainfrom
codex/reliable-startup-fault-attribution

Conversation

@benjaminRomano

@benjaminRomano benjaminRomano commented Sep 10, 2026 •

Copy link
Copy Markdown
Owner

Startup captures could exhaust fixed native arrays, lose Simpleperf samples, or overwrite initial Perfetto data. Reports also conflated independently recorded managed stacks with fault-page evidence and lacked a reproducible way to re-symbolicate and compare repeated runs.

This change:

  • Makes native sample, mapping, callchain and ring capacities configurable; reports each loss source separately; signals every recorder before waiting for exports; adds lean Perfetto with explicit omitted evidence and preserves startup data with discard buffers.
  • Requires a same-record fault address for exact managed attribution. Adds a custom Simpleperf recorder option, address-bearing raw-record parsing and a Google Simpleperf source patch; stock recordings retain their independent stack view. Documents -m and --user-buffer-size and links the upstream stack/deobfuscation reports.
  • Adds atomic --report-only rendering with input/renderer/tool hashes, architecture/build-ID matched native symbols, exact-build mapping assertions and explicit merged/ambiguous Java origins.
  • Separates faulted DEX, caller physical DEX and page contents; adds a secondary DEX caller filter, consumed startup/baseline profile audits distinguishing class and <clinit> roots, and staged OAT failure diagnostics.
  • Adds shared time axes, repeated/return-control cohorts and provenance checks. Readouts lead with fully drawn latency and aggregate app-owned major faults, retaining DEX 3+ as diagnostic evidence.

Validation:

  • ./gradlew build shadowJar generateDocs: passed; 267 JVM tests (10 environment-gated skips), 30 Node tests.
  • Native ARM64 collector compiled with -Wall -Wextra -Werror; Simpleperf patch applies cleanly to Google's event-selection source.
  • Explicitly enabled live Android integration passed on a rooted Android 16 emulator with 16 KB pages: fully drawn cutoff 1,461 ms, zero native/Perfetto loss, 495 lossless stock Simpleperf samples and an independent stack view. A separate lean capture exercised rejection/preservation of lossy Simpleperf data.
  • CLI report-only cohort rendering over two real captures left all 385 captured files unchanged and produced byte-identical HTML on repeat.
  • Chromium desktop/mobile checks passed for plots, selection, filters, zoom and cohort curves, with no JavaScript errors. Experiment screenshot shows an exploratory comparison of different fixture setups, not a treatment effect.

Draft limitation: the modified address-bearing Simpleperf recorder has binary parser regression coverage but has not been AOSP-built or validated on a device. Exact managed fault-page attribution remains experimental until that check; the stock independent-stack path was validated live. The supplied mapping/profile build identity is an explicit assertion tied to the captured APK hash, not independent proof of the build pipeline's consumed inputs.

Usage and evidence semantics: startup fault attribution guide.

Follow-up end-to-end verification (2026-09-11, 5b01fd2):

  • Found and fixed APK-relative native frame symbolication; real stripped JNI APK + matching debug ELF now resolve correctly, and a different real ELF build is rejected.
  • A real R8-obfuscated, deliberately partitioned three-DEX fixture verifies ART deobfuscation, DEX2/3 caller filters, class-only versus explicit initializer roots, and missing method roots. This tests attribution, not R8 layout decisions.
  • Native stress recorded 624,161 samples with zero loss, including all 614,400 induced target faults. Separate one-entry limits independently trigger sample/mapping/callchain failures; the one-sample CLI capture shows 22,800 drops and only health diagnostics.
  • Fresh enabled Android integration passed. Three same-APK full/lean/return-full captures had healthy native/Perfetto/Simpleperf evidence and 384 app-owned major faults each; cohort curves share an axis and full cohort n=2.
  • Real symbolicated report repeat rendering preserved all 370 input files and identical HTML; wrong build binding preserved the previous output. Desktop/mobile filters, expanded profile audits, cohort and failure views passed with no JavaScript errors.
  • Final local checks: 268 JVM tests (10 environment-gated skips), 30 JavaScript tests; independent review reports no unresolved actionable findings. Patched-Simpleperf same-sample managed attribution remains explicitly unvalidated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant