Repository navigation
Paginate live to-do searches instead of rendering every cached row - #100941
Conversation
Codecov Report✅ Changes either increased or maintained existing code coverage, great job!
|
|
🚧 mountiny has triggered a test Expensify/App build. You can view the workflow run here. |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 31b41a449b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Reviewer Checklist
Screenshots/VideosAndroid: HybridAppAndroid-Native-1.mp4Android: mWeb ChromeAndroid-mWeb-1.mp4iOS: HybridAppiOS-Native-2.mp4iOS: mWeb SafariMacOS: Chrome / SafariMac-Chrome-1.mp4 |
| return; | ||
| } | ||
|
|
||
| if (!searchResults?.search?.hasMoreResults) { |
There was a problem hiding this comment.
Offline, reaching the bottom shows more cached rows, but online the only way past the cap is hasMoreResults. When the server says there's nothing more but the device has more to-do reports, those rows can't be reached online.
Two ways to get there IMO:
- On a first visit, the tab's Search request fails. To-do tabs don't show the error view, so the list still renders, but
hasMoreResultsis never set and it stays at 50 rows. - The server's list is shorter than the device's. The app and the server don't use exactly the same rules for these tabs.
Before this PR every cached row was visible. Could we also show more cached rows online when hasMoreResults is false?
There was a problem hiding this comment.
I removed the ability to reveal additional cached rows while offline, so offline users can only see the rows that have already been synced from the backend. The cache fallback should only be used when hasMoreResults === false in a successful search response. A failed request should not be treated as proof that the server is exhausted, wdyt? 🤔
There was a problem hiding this comment.
Agreed, a failed request doesn't prove the server has nothing more, so paging should keep going after a failure.
Tho Two things are still open:
- On a first visit, if the first page fails,
hasMoreResultsis never set (the failure data doesn't write it), sofetchMoreResultsreturns straight away and the list stays at 50 rows until the tab is opened again. The failed page should be asked for again, for example at the next scroll to the bottom. - The fallback for a successful
hasMoreResults: falseisn't in the code yet. When I made the first page returnhasMoreResults: false, the list stayed at 50 of 108 rows.
About offline: Carlos pointed out that to-dos are built from Onyx on purpose so they work offline from slack right, and before this PR every cached row showed. Now offline shows only the pages already loaded (50 of 108 in my test). I'd show more cached rows when the user scrolls to the bottom offline, and after a failed page too, so the list isn't stuck while it waits to retry. Should we defer this for mountiny and luacmartins?
WDYT?
There was a problem hiding this comment.
| // don't force isLoading off, the load-more skeleton reads it to tell a page is in flight | ||
| currentSearchResults = { | ||
| search: {...searchInfo, isLoading: false, hasResults}, | ||
| search: {...searchInfo, hasResults}, |
There was a problem hiding this comment.
Since isLoading isn't forced to false anymore, a stale value can leave a to-do tab stuck.
search.isLoading is saved to disk with the snapshot. If the app is closed or reloaded while a page request is running, it stays true. After reopening, nothing sends a request that would clear it:
useSearchPageSetupskips to-do tabs- the search effect in
<Search>returns early whileisLoadingis true fetchMoreResultsandsearch()return early too
The loading rows stay at the bottom and no more pages load until the sort changes.
To reproduce, on a to-do tab with 50+ reports, run this in the console and reload:
Onyx.merge('snapshot_<hash>', {search: {isLoading: true, offset: 50}})On main the tab recovers by itself.
useInsightData.ts handles the same case by calling search() with isLoading: false and letting the duplicate check skip a request that's really still running. Could we do something similar here?
This causes this bug
Mac-Chrome.mp4
There was a problem hiding this comment.
thanks for finding this, mind checking again?
There was a problem hiding this comment.
It fixes a reload while a page request is running, because that request also saves state: 'loading'. A to-do tab can still get stuck in two ways:
- First visit. The snapshot only gets its
hashwhen the first answer comes back, so a reload before that fails the newisSnapshotForCurrentQuerycheck and nothing sends the request again. The effect in<Search>still returns early whileisLoadingis true, so switching tabs doesn't fix it either. To reproduce, run this and reload:Onyx.set('snapshot_<hash>', {search: {isLoading: true, state: 'loading', offset: 0}});
isLoading: truewithoutstate. The console line from my first comment does this. So does deleting selected expenses in a report opened from Search, if the app is closed beforehandlePreventSearchAPIturns loading off again.isSearchPendingis false in both cases, so the new check skips them.
A leftover isLoading: true at offset 0 also keeps the footer total as a skeleton, and offline nothing can clear it until the app reconnects.
Since the snapshot's isLoading is shared with other requests, could live results keep isLoading: false like on main, and let to-do paging track its own requests?
|
Bug: Mac-Chrome-2.mp4 |
Looks like it's broken on prod too Screen.Recording.2026-09-15.at.15.05.58.mov |
|
@TaduJR friendly bump |
This comment was marked as resolved.
This comment was marked as resolved.
|
🤖 Code review — 2 things I'd want answered before approving, plus a stale PR description. The approach is sound: cap 1. The local-reveal path is missing the scroll guard — it can un-cap the whole listYou added
2. The selected/highlighted escape hatch in
|
This comment was marked as resolved.
This comment was marked as resolved.
|
🚧 mountiny has triggered a test Expensify/App build. You can view the workflow run here. |
This comment has been minimized.
This comment has been minimized.
| const rowLimitEnd = visibleRowLimit === undefined ? stableSortedData.length : getRowLimitEnd(stableSortedData, visibleRowLimit, selectedTransactions, isOffline); | ||
| const isRowLimitApplied = rowLimitEnd < stableSortedData.length; | ||
| const visibleData = isRowLimitApplied ? stableSortedData.slice(0, rowLimitEnd) : stableSortedData; | ||
| // selection runs off this, so cap it too or "select all" reaches rows that were never rendered; live rows are all report groups |
There was a problem hiding this comment.
| // selection runs off this, so cap it too or "select all" reaches rows that were never rendered; live rows are all report groups | |
| // selection runs off this, so cap it too or "select all" reaches rows that were never rendered; live rows are all report groups |
| // Wide layout floats the bulk action bar over the end of the list, so the list has to leave room for it. | ||
| const shouldReserveBulkActionBarSpace = useShouldShowBulkActionBar(queryJSON); | ||
| const [offset, setOffset] = useState(0); | ||
| // a page left running by a previous mount is still in flight, so wait for it rather than revealing or skipping its rows |
There was a problem hiding this comment.
| // a page left running by a previous mount is still in flight, so wait for it rather than revealing or skipping its rows | |
| // Page left running by a previous mount is still in flight, so wait for it rather than revealing or skipping its rows |
| const [offset, setOffset] = useState(0); | ||
| // a page left running by a previous mount is still in flight, so wait for it rather than revealing or skipping its rows | ||
| const isLivePageRunningAtMount = () => shouldUseLiveData && isSearchPending(searchResults) && (searchResults?.search?.offset ?? 0) > 0; | ||
| // live paging owns its in-flight flag, keyed to the offset it asked for so other searches on this hash don't read as ours |
There was a problem hiding this comment.
| // live paging owns its in-flight flag, keyed to the offset it asked for so other searches on this hash don't read as ours | |
| // Live paging owns its in-flight flag, keyed to the offset it asked for so other searches on this hash don't read as ours |
| wantedOffsetRef.current = nextOffset; | ||
| return; | ||
| } | ||
| // the server has nothing left, but the live scan can still hold rows past the cap: page those in locally |
There was a problem hiding this comment.
| // the server has nothing left, but the live scan can still hold rows past the cap: page those in locally | |
| // The server has nothing left, but the live scan can still hold rows past the cap: page those in locally |
|
🚧 mountiny has triggered a test Expensify/App build. You can view the workflow run here. |
|
Should we create create a new issue for #100941 (comment)? I can help with the review. |
|
✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release. |
|
🧪🧪 Use the links below to test this adhoc build on Android, iOS, and Web. Happy testing! 🧪🧪
|
|
🚀 Deployed to staging by https://github.com/mountiny in version: 9.4.94-0 🚀
|
|
Prerequisite: Have more than 25 Draft Expenses
Expected: N selected is decremented Mac-Chrome.mp4 |
|
@MelvinBot can you create an issue for #100941 (comment) in App |
|
No help site changes are required for this PR, so I didn't create a docs PR. This PR changes how the to-do lists on the Spend page (Submit, Approve, Pay, Export) load rows. That is a performance change. It doesn't add or rename any feature, setting, or button. Why no docs update is needed
|
|
I created Spend - Select all count is stale after submitting a report for I didn't find a duplicate. A related issue is [Bug] Mark as exported processes only the first page after Reports “Select all matching”. I didn't add labels or an assignee. |
|
🚀 Deployed to production by https://github.com/Julesssss in version: 9.4.94-0 🚀
Bundle Size Analysis (Sentry): |
Explanation of Change
The four to-do searches (Submit, Approve, Pay, Export) render straight from a live Onyx scan, which hands back every matching row at once. If you have a big local cache, that means hundreds of rows on the very first render.
This PR caps those lists at 50 rows and pages them like any other search.
The cap lives in
useSearchSnapshot. We slice the rows after sorting, so the next page picks up right where the current one ends. A few rows get to stay even past the cap: anything you've selected and anything that's being highlighted. Rows that are being deleted don't count toward it while online, since they're hidden anyway. Selection data is capped too, so "select all" only picks what you can actually see. The chart and totals still use the full set.A small new hook,
useLiveRowLimit, decides how many rows to show. It's one page past the last page the server actually answered. It only ever grows, so a refresh (which resets the offset to 0) doesn't suddenly shrink the list — those rows are still sitting in Onyx.In
<Search>, hitting the end of the list now fires a regular Search request instead of bailing out early for live searches. Live paging tracks its own "loading" flag, tied to the offset it asked for, so a refresh on the same query doesn't flash the load-more skeleton.SearchResultsProviderkeeps forcingisLoading: falsefor live searches, because that flag survives a reload and would otherwise leave skeletons stuck on screen.A few edge cases worth calling out:
hasMoreResultsis only a default at that point.errors, so we look atresponseJsonCodeto spot a failed page. It only retries when you scroll to the end again. The list also reports "end reached" whenever its rows rebuild, so we use an actual scroll to tell the two apart.<Search>remounted, we wait for it rather than skipping or duplicating its rows, and ask for it once more if a reload left it hanging.Related to #100409, which removes the live to-do path entirely. This PR keeps it and paginates it instead, so to-dos keep working offline.
Fixed Issues
$ #101061
PROPOSAL:
Tests
SearchAPI is being called, and new items appear in the list once it resolvesSearchrequest in devtools, scroll to the end, and verify the skeleton goes away and no request loop startsOffline tests
QA Steps
// TODO: These must be filled out, or the issue title must include "[No QA]."
Same as tests.
PR Author Checklist
### Fixed Issuessection aboveTestssectionOffline stepssectionQA stepssectioncanBeMissingparam foruseOnyxtoggleReportand notonIconClick)src/languages/*files and using the translation methodSTYLE.md) were followedAvatar, I verified the components usingAvatarare working as expected)StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))npm run compress-svg)Avataris modified, I verified thatAvataris working as expected in all cases)Designlabel and/or tagged@Expensify/designso the design team can review the changes.ScrollViewcomponent to make it scrollable when more elements are added to the page.mainbranch was merged into this PR after a review, I tested again and verified the outcome was still expected according to theTeststeps.Screenshots/Videos
Android: Native
Android: mWeb Chrome
iOS: Native
iOS: mWeb Safari
MacOS: Chrome / Safari
Screen.Recording.2026-09-14.at.10.15.24.mov