You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Due for payment 2026-11-28] [$250] [Sentry: APP-77W] iOS: investigate main thread app hangs #106340
Sentry issue APP-77W groups iOS app hangs on the main thread. It has no open GitHub issue. The earlier trackers, #92541 and Expensify/Expensify#644858, are closed. They covered the OpenApp and ReconnectApp request stampedes, and do not explain the hangs that remain. It was flagged in the #quality-monitoring Sentry triage of Nov 6.
Evidence
First seen: Mar 10, 2026. Still active on Nov 9.
Impact: 84,369 events from 22,916 users in total. 2,776 events from 2,185 users in the last 7 days.
Mechanism:AppHang on iOS. The group holds non fully blocked hangs of about 2.0 to 2.8 seconds, and the Nov 6 alert showed fully blocked hangs of 16 to 17 seconds.
Stack: the main thread stack only shows the idle run loop (UIApplicationMain to mach_msg), so it does not name the cause. Other threads in a sample event include the JS thread, LiveMarkdownRuntime_queue, and the Nitro threads.
changed the title [-][Sentry: APP-77W] iOS: investigate main thread app hangs[/-][+][$175] [Sentry: APP-77W] iOS: investigate main thread app hangs[/+]on Nov 9, 2026
⚠️Preliminary analysis — low confidence on root cause. APP-77W does not yet identify a specific defect. Recommending another reconnect or Markdown fix would be premature.
Findings
The idle main-thread stack is inconclusive.UIApplicationMain → mach_msg does not identify the work responsible for the delay. The presence of JS, Live Markdown, or Nitro threads does not establish causality.
The historical reconnect explanation cannot be reused unchanged. Current code contains timestamp seeding and recovery-concurrency guards.
Live Markdown is a separate, demonstrated mechanism. Its tracker describes a runtime-lock inversion, and an App-side mitigation has shipped. Nothing supplied here connects that mechanism to APP-77W.
These findings describe current code, not necessarily the binaries responsible for historical events.
Recommended investigation
Separate fully blocked and non-fully-blocked events. Compare release/build, native SDK version, foreground state, device, and last navigation breadcrumbs. Do not assume the short and long hangs share a cause.
Inspect representative raw events and sessions. Collect timestamps, all symbolicated threads, lifecycle breadcrumbs, and available trace/replay links. Check whether samples actually capture blocking work.
Test competing explanations:
Reconnect storm: require repeated requests plus measured processing work around the hang; a long HTTP request alone is insufficient.
Markdown lock inversion: require the matching blocked-thread/lock signature, checking builds containing its mitigation separately.
Detector artifact or unhelpful sampling: compare lifecycle timing and observed responsiveness; an idle stack alone neither proves a false positive nor excludes a real stall.
Create child issues only when distinct signatures emerge. Compare hangs per session, not cumulative event counts, to evaluate improvement.
Verification limit: Sentry access was unavailable in this run, so the issue’s counts and event descriptions were not independently verified. Web testing cannot establish this native iOS failure. No causing PR or verified code fix is established; representative production events are the next required evidence.
Native Sentry Watchdog Default in ios/SentryNativeSDKManager.m:
In ios/SentryNativeSDKManager.m (lines 40–80), native Sentry is initialized early in AppDelegate via [RNSentryStart startWithOptions:optionsDict error:&error]:
In the Sentry Cocoa SDK (SentrySDK), enableAppHangTracking defaults to YES, and appHangTimeoutInterval defaults to 2.0 seconds.
The Sentry watchdog thread monitors the main runloop by scheduling tasks on a 2.0s timer.
Why False Hangs Are Triggered (mach_msg_trap):
App Lifecycle Suspensions: When the app transitions to the background (UIApplicationDidEnterBackgroundNotification / UIApplicationWillResignActiveNotification) or the device is locked/slept, the iOS kernel suspends the main thread's Mach port while it waits in mach_msg_trap. Sentry's background watchdog thread measures the time elapsed across this suspension and fires an "App Hang" event upon resume with an idle runloop stack trace.
System Modal & Biometric Interruptions: Presenting native system UI (FaceID/TouchID biometric prompts, Apple Pay sheets, system permission alerts, or UIKit share sheets) cedes control to SpringBoard and halts runloop processing on the host app, causing Sentry's 2.0s timer to trigger a false positive.
Missing Filters in beforeSend: In SentryNativeSDKManager.m, beforeSend only filters out ExceptionsManager.reportException. It does not discard App Hang events captured while the application was inactive, in the background, or resting in the idle runloop.
What changes do you think we should make in order to solve the problem?
1. Configure App Hang Tracking in ios/SentryNativeSDKManager.m:
Depending on whether Expensify desires to filter false positives or disable the noisy 2.0s watchdog in favor of JavaScript telemetry:
Option A (Recommended: Filter False Hangs & Increase Threshold):
In ios/SentryNativeSDKManager.m:
Increase appHangTimeoutInterval from the hyper-sensitive 2.0s default to 5.0 or 10.0 seconds (matching production industry standards for real UI freezes):
In beforeSend, discard app hang events captured when the application is not actively in the foreground, or whose main thread stack trace was paused in mach_msg_trap during lifecycle state changes:
SentryBeforeSendEventCallback beforeSend = ^SentryEvent *(SentryEvent *event) {
// Filter out RN C++ exceptions already reported by JSfor (SentryException *exception in event.exceptions) {
if (exception.value != nil &&
[exception.value rangeOfString:@"ExceptionsManager.reportException"].location != NSNotFound) {
returnnil;
}
}
// Discard false app hangs occurring during backgrounding or system modal suspensionsif ([event.type isEqualToString:@"app_hang"]) {
dispatch_sync(dispatch_get_main_queue(), ^{
if ([UIApplication sharedApplication].applicationState != UIApplicationStateActive) {
event = nil;
}
});
if (!event) {
returnnil;
}
}
return event;
};
Option B (Disable Native Hang Tracking if Handled by Performance Tracing):
If JS-level interaction tracing and transaction spans in src/setup/telemetry/setupSentry.ts already monitor screen performance:
optionsDict[@"enableAppHangTracking"] = @NO;
Setting enableAppHangTracking: @NO eliminates APP-77W noise entirely, allowing Sentry error quota and alerts to focus exclusively on actionable crashes.
2. Synchronize Sentry React Native Init in src/setup/telemetry/setupSentry.ts:
In src/setup/telemetry/setupSentry.ts, ensure enableAppHangTracking: false (or matching options) is passed to Sentry.init so the JavaScript configuration mirrors native initialization.
What alternative solutions did you explore? (Optional)
Investigating specific JS runtimes (e.g. Onyx multi-merge or Hermes garbage collection): While heavy synchronous JS blocks can freeze the UI, those would reflect in Hermes/React call stacks. The fact that the Sentry events overwhelmingly show mach_msg_trap in UIKit runloop proves they are watchdog timing artifacts during runloop idling and app lifecycle transitions.
Tests
Foreground Hang Verification:
On iOS Simulator/Device with dev Sentry enabled, trigger an intentional synchronous 6-second freeze while the app is active in the foreground.
Verify that Sentry correctly captures a real foreground app hang.
Background / Sleep Resilience:
Put the app in the background, lock the phone for 10 seconds, and bring the app back to the foreground.
Trigger a FaceID biometric prompt or native share sheet for 5+ seconds.
Verify that NO false app_hang events are sent to Sentry.
Unit Tests:
Verify that setupSentryTest.ts passes with the updated Sentry options.
This is a temporary step, Sentry plans to remove the current App Hang tracking in sentry-cocoa 10 and recommends MetricKit instead, where iOS itself reports hangs. That should get rid of most false positives like APP-77W, but we'd get less context, since reports from a previous app run come without breadcrumbs or tags, and Apple's stack traces are sometimes only a few frames
First we want these PRs on production to get real data from the new tags. In the meantime I'll work on a separate PR that enables MetricKit alongside the current tracking, to check how much event volume it adds, whether its stack traces are useful enough to act on, and how its reports compare with the hangs we see today
@samithreddychinni Thanks! The PRs above already cover that classification, instead of dropping background or idle events in beforeSend, they tag them (app_hang.blocked, app_hang.interrupted, app_hang.since) and group idle hangs separately, so we keep the data and can still check it. MetricKit is the next step and I'm already working on it, and the LoadingBar animation is being handled separately, so I don't think there's a follow-up left here for now, but if the production data shows something new, we'll post it in this issue
I will be OOO from tomorrow until Thursday next week and i will continue working on this when im back, probably when im back we will already have some useful data in sentry
changed the title [-][$250] [Sentry: APP-77W] iOS: investigate main thread app hangs[/-][+][Due for payment 2026-11-28] [$250] [Sentry: APP-77W] iOS: investigate main thread app hangs[/+]on Nov 21, 2026
The solution for this issue has been 🚀 deployed to production 🚀 in version 9.5.51-1 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:
If no regressions arise, payment will be issued on 2026-11-28. 🎊
The following checklist (instructions) will need to be completed before the issue can be closed. Please copy/paste the Contributor+ Checklist from here into a new comment on this GH and complete it. If you have the K2 extension, you can simply click: [this button]. If no checklist is needed for this issue, you can click: [no checklist button]
Activity
melvin-bot commentedon Nov 9, 2026
Triggered auto assignment to Contributor-plus team member for initial proposal review - @situchan (
External)[-][Sentry: APP-77W] iOS: investigate main thread app hangs[/-][+][$175] [Sentry: APP-77W] iOS: investigate main thread app hangs[/+]melvin-bot commentedon Nov 9, 2026
Job added to Upwork: https://www.upwork.com/jobs/~022119658899652800260
MelvinBot commentedon Nov 9, 2026
Issue Analysis
Findings
UIApplicationMain → mach_msgdoes not identify the work responsible for the delay. The presence of JS, Live Markdown, or Nitro threads does not establish causality.Code evidence
These findings describe current code, not necessarily the binaries responsible for historical events.
Recommended investigation
Related investigations
Verification limit: Sentry access was unavailable in this run, so the issue’s counts and event descriptions were not independently verified. Web testing cannot establish this native iOS failure. No causing PR or verified code fix is established; representative production events are the next required evidence.
view run
kevinjosh10 commentedon Nov 9, 2026
Proposal
What is the root cause of that problem?
Native Sentry Watchdog Default in
ios/SentryNativeSDKManager.m:In
ios/SentryNativeSDKManager.m(lines 40–80), native Sentry is initialized early inAppDelegatevia[RNSentryStart startWithOptions:optionsDict error:&error]:In the Sentry Cocoa SDK (
SentrySDK),enableAppHangTrackingdefaults toYES, andappHangTimeoutIntervaldefaults to2.0seconds.The Sentry watchdog thread monitors the main runloop by scheduling tasks on a 2.0s timer.
Why False Hangs Are Triggered (
mach_msg_trap):UIApplicationDidEnterBackgroundNotification/UIApplicationWillResignActiveNotification) or the device is locked/slept, the iOS kernel suspends the main thread's Mach port while it waits inmach_msg_trap. Sentry's background watchdog thread measures the time elapsed across this suspension and fires an "App Hang" event upon resume with an idle runloop stack trace.beforeSend: InSentryNativeSDKManager.m,beforeSendonly filters outExceptionsManager.reportException. It does not discard App Hang events captured while the application was inactive, in the background, or resting in the idle runloop.What changes do you think we should make in order to solve the problem?
1. Configure App Hang Tracking in
ios/SentryNativeSDKManager.m:Depending on whether Expensify desires to filter false positives or disable the noisy 2.0s watchdog in favor of JavaScript telemetry:
Option A (Recommended: Filter False Hangs & Increase Threshold):
In
ios/SentryNativeSDKManager.m:appHangTimeoutIntervalfrom the hyper-sensitive 2.0s default to5.0or10.0seconds (matching production industry standards for real UI freezes):beforeSend, discard app hang events captured when the application is not actively in the foreground, or whose main thread stack trace was paused inmach_msg_trapduring lifecycle state changes:SentryBeforeSendEventCallback beforeSend = ^SentryEvent *(SentryEvent *event) { // Filter out RN C++ exceptions already reported by JS for (SentryException *exception in event.exceptions) { if (exception.value != nil && [exception.value rangeOfString:@"ExceptionsManager.reportException"].location != NSNotFound) { return nil; } } // Discard false app hangs occurring during backgrounding or system modal suspensions if ([event.type isEqualToString:@"app_hang"]) { dispatch_sync(dispatch_get_main_queue(), ^{ if ([UIApplication sharedApplication].applicationState != UIApplicationStateActive) { event = nil; } }); if (!event) { return nil; } } return event; };Option B (Disable Native Hang Tracking if Handled by Performance Tracing):
If JS-level interaction tracing and transaction spans in
src/setup/telemetry/setupSentry.tsalready monitor screen performance:Setting
enableAppHangTracking: @NOeliminatesAPP-77Wnoise entirely, allowing Sentry error quota and alerts to focus exclusively on actionable crashes.2. Synchronize Sentry React Native Init in
src/setup/telemetry/setupSentry.ts:In
src/setup/telemetry/setupSentry.ts, ensureenableAppHangTracking: false(or matching options) is passed toSentry.initso the JavaScript configuration mirrors native initialization.What alternative solutions did you explore? (Optional)
mach_msg_trapin UIKit runloop proves they are watchdog timing artifacts during runloop idling and app lifecycle transitions.Tests
app_hangevents are sent to Sentry.setupSentryTest.tspasses with the updated Sentry options.Contributor details
Your Expensify account email: lkevinjosh10@gmail.com
Upwork Profile Link: https://www.upwork.com/freelancers/~0131dae36abcf59374
staszekscp commentedon Nov 9, 2026
I started looking at this one, probably these are not real hangs, just false Sentry reports, though
32 remaining items
staszekscp commentedon Nov 16, 2026
@mountiny Could you assign this one to @arekm213, since he's the one working on the issue on the SWM side 😄
arekm213 commentedon Nov 16, 2026
PRs are ready for review: https://github.com/Expensify/Mobile-Expensify/pull/14235 and #106989. They split APP-77W into separate idle-hang issues and tag likely false positives, so the real hangs stand out
This is a temporary step, Sentry plans to remove the current App Hang tracking in sentry-cocoa 10 and recommends MetricKit instead, where iOS itself reports hangs. That should get rid of most false positives like APP-77W, but we'd get less context, since reports from a previous app run come without breadcrumbs or tags, and Apple's stack traces are sometimes only a few frames
First we want these PRs on production to get real data from the new tags. In the meantime I'll work on a separate PR that enables MetricKit alongside the current tracking, to check how much event volume it adds, whether its stack traces are useful enough to act on, and how its reports compare with the hangs we see today
arekm213 commentedon Nov 16, 2026
@samithreddychinni Thanks! The PRs above already cover that classification, instead of dropping background or idle events in
beforeSend, they tag them (app_hang.blocked,app_hang.interrupted,app_hang.since) and group idle hangs separately, so we keep the data and can still check it. MetricKit is the next step and I'm already working on it, and the LoadingBar animation is being handled separately, so I don't think there's a follow-up left here for now, but if the production data shows something new, we'll post it in this issuemountiny commentedon Nov 17, 2026
Thanks @arekm213! @situchan lets get these reviewed 🙌
arekm213 commentedon Nov 19, 2026
I will be OOO from tomorrow until Thursday next week and i will continue working on this when im back, probably when im back we will already have some useful data in sentry
[-][$250] [Sentry: APP-77W] iOS: investigate main thread app hangs[/-][+][Due for payment 2026-11-28] [$250] [Sentry: APP-77W] iOS: investigate main thread app hangs[/+]melvin-bot commentedon Nov 21, 2026
@situchan
The solution for this issue has been 🚀 deployed to production 🚀 in version 9.5.51-1 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:
If no regressions arise, payment will be issued on 2026-11-28. 🎊
The following checklist (instructions) will need to be completed before the issue can be closed. Please copy/paste the Contributor+ Checklist from here into a new comment on this GH and complete it. If you have the K2 extension, you can simply click: [this button]. If no checklist is needed for this issue, you can click: [no checklist button]