Repository navigation
Description
If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!
Action Performed:
This bug comes from discussion in #26592
TL;DR: Onyx processes the successData and API response in the wrong order on iOS and Android.
Dive deeper:
App/src/libs/actions/Policy.js
Line 408 in 3352cc8
| API.write('AddMembersToWorkspace', params, {optimisticData, successData, failureData}); |
When adding a member to workspace here, we send the following data:
"optimisticData": {
"onyxMethod": "set",
"key": "report_3008335418540078",
"value": {
"type": "chat",
"chatType": "policyExpenseChat",
"ownerAccountID": 899964195,
"participantAccountIDs": [
15664021,
899964195
],
"policyID": "41A8E180FAEB43DF",
"reportID": "3008335418540078",
"reportName": "Chat Report",
"pendingFields": {
"createChat": "add"
},
"isOptimisticReport": true
...
}
},
"successData": {
"onyxMethod": "merge",
"key": "report_3008335418540078",
"value": {
"pendingFields": {
"createChat": null
},
"errorFields": {
"createChat": null
},
"isOptimisticReport": false
}
}
And the API call returns the following onyx merge_collection action:
{
"jsonCode": 200,
"requestID": "806fecea0afd3512-WAW",
"onyxData": [
{
"onyxMethod": "mergecollection",
"key": "report_",
"value": {
...
"report_3008335418540078": {
"reportID": "3008335418540078",
"reportName": "",
"type": "chat",
"chatType": "policyExpenseChat",
"ownerEmail": "[pasha@test.co](mailto:pasha@test.co)",
"ownerAccountID": 15671904,
"policyID": "41A8E180FAEB43DF",
"participants": [
"[pasha@test.co](mailto:pasha@test.co)",
"[paultsimura+3@gmail.com](mailto:paultsimura+3@gmail.com)"
],
"participantAccountIDs": [
15664021,
15671904
],
"lastActorAccountID": 15671904,
},
}
},
]
}
It should replace the temporary accountID of the report.ownerAccountID with the real one, which comes from the server.
On the web, the onyx actions are executed in the following order:
- persist optimisticData;
- persist successData;
- persist API response;
The issue is that on iOS, the onyx actions are executed in the following order: - persist optimisticData;
- persist API response;
- persist successData;
ThesuccessDataholds merge operation of only several certain fields, which shouldn't affect the report much.
However, here's the tricky part:
This piece of code, which creates a promise for future merge operation, is executed before Onyx persists the API response:
https://github.com/Expensify/react-native-onyx/blob/616c4c5286b7a7fadac18d477cd75edd60865b12/lib/Onyx.js#L1136-L1147
Meaning the following execution chain: - persist optimisticData;
- create
persistSuccessDatapromise; - persist API response;
- execute
persistSuccessDatapromise;
This means, at the momentpersistSuccessDatais created, the existingValue is still not updated, so even though it should update only a couple of fields, themodifiedDatahere is saved in a state of pre-API-response.
Then the report is updated by the "persist API response" operation, but gets overwritten by thepersistSuccessDataoperation with the outdated data.
If we debug step-by-step, you'll notice that the chat name is displayed correctly between the steps 3 and 4 from above.
As an outcome of this complete operation, the state of Onyx is the following:
- personalDetailsList contains the new permanent ownerAccountID;
- report has the temporary ownerAccountID, which is already removed from Onyx.
Expected Result:
Onyx updates are executed in correct order without data being overwritten
Actual Result:
Onyx updates face a race condition and some data is overwritten
Workaround:
Unknown
Platforms:
Which of our officially supported platforms is this issue occurring on?
- Android / native
- Android / Chrome
- iOS / native
- iOS / Safari
- MacOS / Chrome / Safari
- MacOS / Desktop
Version Number: 1.3.76.0
Reproducible in staging?: n/a
Reproducible in production?: n/a
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
n/a
Expensify/Expensify Issue URL:
Issue reported by: @@paultsimura
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1696287156456449
Upwork Automation - Do Not Edit
- Upwork Job URL: https://www.upwork.com/jobs/~013373cff08a2575e9
- Upwork Job ID: 1709846441891180544
- Last Price Increase: 2023-10-05
Activity
melvin-bot commentedon Oct 3, 2023
Triggered auto assignment to @CortneyOfstad (
Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.melvin-bot commentedon Oct 3, 2023
Bug0 Triage Checklist (Main S/O)
Platformsin OP are ✅)ospfranco commentedon Oct 4, 2023
I believe I found the cause.
When the
updatefunction is called, all the operations are fired in an async manner. However, this causes a race condition where certain operations will be written into the DB before others. It's a built-in race condition.Just for documentation purposes: I suggested we need to make the update operations sequentially await. @marcaaron @tgolen are however worried about the app taking a big performance hit. I will try another solution and report back if I manage to get the app to do reliable updates.
pradeepmdk commentedon Oct 5, 2023
for note: that is what I already told in my proposal. #26592 (comment)
cc: @Julesssss @ospfranco
pradeepmdk commentedon Oct 5, 2023
Another problem in the App side:
Proposal
Please re-state the problem that we are trying to solve in this issue.
Onyx updating on race condition
What is the root cause of that problem?
https://github.com/Expensify/App/assets/16595846/736946e5-183f-4b16-969a-c0ef118ad228

@marcaaron I saw recently it changed to onyx to memory here
#25614
we are flushed out to Onyx once all the api is completed in the queue (networkRequestQueue).
I think this will create a performance issue right?ex: here in the queue2035is pending after coming online it gets a response object in thequeuedOnyxUpdatesto46kobject.Save queued updates to memory only #25614 Due to this change incase the middle of the response I refresh the page means it will never update to onyx.
What changes do you think we should make in order to solve the problem?
App/src/libs/Network/SequentialQueue.js
Line 126 in 41ddd74
we need to flush out each api success or failure.
cc: @marcaaron @tgolen @ospfranco @mountiny
[-]Web - Onyx: out-of-order updates[/-][+][$500] Web - Onyx: out-of-order updates[/+]melvin-bot commentedon Oct 5, 2023
Job added to Upwork: https://www.upwork.com/jobs/~013373cff08a2575e9
melvin-bot commentedon Oct 5, 2023
Triggered auto assignment to Contributor-plus team member for initial proposal review - @jjcoffee (
External)63 remaining items
Julesssss commentedon Dec 13, 2023
I found them on Twitter, but I got blocked for typing a message that sounded like a scam 😆
lets see if they get my second message
ospfranco commentedon Dec 13, 2023
Ah, so actually my PR was reverted, I don't know which steps the team took to fix the issue if any at all. As it had entailed doing a large refactoring of the Onyx internals. I cannot in good conscience accept any payment since I'm no longer helping Margelo with the project. You can forgo any payment you are planning for me, it's fine :)
Julesssss commentedon Dec 18, 2023
@ospfranco thanks, I see. It's hard to tell but it looks like we would have reverted, then resubmitted with a small fix which would still be eligible for payment.
But I'll leave it up to you
CortneyOfstad commentedon Dec 20, 2023
@ospfranco bump on the comment above ^^^ Thanks!
Julesssss commentedon Dec 21, 2023
Sounds like he's happy not to. We can always reopen if that changes though!
melvin-bot commentedon Dec 23, 2023
If you are the assigned CME please investigate whether the linked PR caused a regression and leave a comment with the results.
If a regression has occurred and you are the assigned CM follow the instructions here.
If this regression could have been avoided please consider also proposing a recommendation to the PR checklist so that we can avoid it in the future.