Skip to content

[HOLD for payment 2024-07-10] [HOLD for payment 2024-07-02] LOW: [$500] Stop using reportAction.originalMessage or reportAction.message.text #39797

Closed

Description

@quinthar

Problem:

Each comment in the server database is stored in reportAction.message in HTML form. However, for some reason lost to time, when we send that same comment out as an Onyx update, we:

  1. Rename the original message in the database to reportAction.originalMessage in the update (which is confusing)
  2. Create a new message object that contains a copy of the html message, as well as a stripped text version.
  3. Put the message object inside a totally unnecessary one-element array

That means every comment is actually sent out three times (four, if you include the update to the report object, but that's out of scope for now). This is clearly wasteful in a number of ways:

  1. It requires postprocessing on the server to do this extraneous operation
  2. It increases network bandwidth by 3x
  3. It increases RAM use 3x
  4. It increases on-device storage 3x
  5. It's confusing AF

Additionally, it's particularly problematic when sending Onyx updates out via UrbanAirship, which is extremely limited in the payload sizes allowed -- with anything over a certain limit just dropped quietly and never delivered, causing "gaps" in our update stream (which require more network calls to "backfill" the Onyx data on app open, which makes things slow). Basically, this was just a mistake introduced for reasons we can't remember, but that we want to undo.

Solution:

To solve this, please:

  1. Phase 1 (External / clientside): Stop using originalMessage, start using array-message
    1. Update every instance of originalMessage.html to use message.html || message[0].html
    2. Update every instance of message.text to dynamically strip the text from message.html || message[0].html
    3. Update every key within originalMessage to use the original message
  2. Phase 2 (Internal / serverside): Once we've confirmed the client no longer uses originalMessage or message.text or any other key within originalMessage:
    1. Stop sending originalMessage
    2. Start sending message without an array
  3. Phase 3 (External / clientside): Remove checking for "array message"

This issue is for phase 1 and phase 3.

Upwork Automation - Do Not Edit
  • Upwork Job URL: https://www.upwork.com/jobs/~01856d1399295ef760
  • Upwork Job ID: 1776729925881405440
  • Last Price Increase: 2024-07-12
  • Automatic offers:
    • cubuspl42 | Reviewer | 0
Issue OwnerCurrent Issue Owner: @puneetlath

Activity

  1. converted this from a draft issue on Apr 6, 2024
  2. added
    ExternalAdded to denote the issue can be worked on by a contributor
    ImprovementItem broken or needs improvement.
    on Apr 6, 2024
  3. changed the title [-]HIGH: [Reliability] Remove `text` from the `reportActions` sent to the client to reduce payload size[/-] [+][$250] HIGH: [Reliability] Remove `text` from the `reportActions` sent to the client to reduce payload size[/+] on Apr 6, 2024
  4. melvin-bot commented on Apr 6, 2024

    @melvin-bot
  5. added
    Help WantedApply this label when an issue is open to proposals by contributors
    on Apr 6, 2024
  6. melvin-bot commented on Apr 6, 2024

    @melvin-bot

    Triggered auto assignment to Contributor-plus team member for initial proposal review - @cubuspl42 (External)

  7. maxim-grin commented on Apr 6, 2024

    @maxim-grin

    Hey I’m going to investigate why the payload is so big in Onyx storage. Possibly track what change exactly caused the problem. Then either revert that piece of change or use one of suggestions by @thienlnam

    Contributor details
    Your Expensify account email: maxim.grin@gmail.com
    Upwork Profile Link: https://www.upwork.com/freelancers/~01fb7b172c01252917

  8. melvin-bot commented on Apr 6, 2024

    @melvin-bot

    📣 @maxim-grin! 📣
    Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
    Please follow these steps:

    1. Make sure you've read and understood the contributing guidelines.
    2. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
    3. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
    4. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
      Screen Shot 2022-11-16 at 4 42 54 PM
      Format:
    Contributor details
    Your Expensify account email: <REPLACE EMAIL HERE>
    Upwork Profile Link: <REPLACE LINK HERE>
    
  9. maxim-grin commented on Apr 6, 2024

    @maxim-grin

    Hey I’m going to investigate why the payload is so big in Onyx storage. Possibly track what change exactly caused the problem. Then either revert that piece of change or use one of suggestions by @thienlnam

    Contributor details
    Your Expensify account email: maxim.grin@gmail.com
    Upwork Profile Link: https://www.upwork.com/freelancers/~01fb7b172c01252917

  10. melvin-bot commented on Apr 6, 2024

    @melvin-bot

    ✅ Contributor details stored successfully. Thank you for contributing to Expensify!

  11. ShridharGoel commented on Apr 7, 2024

    @ShridharGoel
    Contributor

    Proposal

    Please re-state the problem that we are trying to solve in this issue.

    We don't need to use text from reportAction, it would be better to use originalMessage.html.

    What is the root cause of that problem?

    New change.

    text from message in reportAction is being used at multiple places.

    What changes do you think we should make in order to solve the problem?

    We have text defined here:

    /** The text content of the fragment. */
    text: string;

    Example usage:

    const actionMessage = action.message?.[0]?.text ?? '';

    We can change this to:

    const parser = new ExpensiMark();
    const actionMessage = parser.htmlToText(action.message?.[0]?.html ?? '');
    

    Add the above in a common method getActionMessagePostParsing in ReportUtils.
    Similarly, check all usages and update them to use the method mentioned above.

    Finally, we can remove text.

    There's one instance of originalMessage?.html in ReportUtils.ts which also needs to be updated to use the common method.

    if (ReportActionsUtils.isWhisperAction(reportAction)) {
        // Allow flagging welcome message whispers as they can be set by any room creator
        if (report?.description && !isCurrentUserAction && isOriginalMessageHaveHtml && `getActionMessagePostParsing(reportAction)` === report.description) {
            return true;
        }
    
        // Disallow flagging the rest of whisper as they are sent by us
        return false;
    }
    
  12. 101 remaining items

  13. melvin-bot commented on Jul 3, 2024

    @melvin-bot

    The solution for this issue has been 🚀 deployed to production 🚀 in version 9.0.3-7 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

    If no regressions arise, payment will be issued on 2024-07-10. 🎊

    For reference, here are some details about the assignees on this issue:

    • @c3024 requires payment (Needs manual offer from BZ)
    • @nkdengineer requires payment (Needs manual offer from BZ)
  14. melvin-bot commented on Jul 3, 2024

    @melvin-bot
  15. melvin-bot commented on Jul 10, 2024

    @melvin-bot

    Payment Summary

    Upwork Job

    • ROLE: @c3024 paid $(AMOUNT) via Upwork (LINK)
    • ROLE: @nkdengineer paid $(AMOUNT) via Upwork (LINK)

    BugZero Checklist (@puneetlath)

    • I have verified the correct assignees and roles are listed above and updated the neccesary manual offers
    • I have verified that there are no duplicate or incorrect contracts on Upwork for this job (https://www.upwork.com/ab/applicants/1776729925881405440/hired)
    • I have paid out the Upwork contracts or cancelled the ones that are incorrect
    • I have verified the payment summary above is correct
  16. changed the title [-][HOLD for payment 2024-07-10] [HOLD for payment 2024-07-02] LOW: [$250] Stop using `reportAction.originalMessage` or `reportAction.message.text`[/-] [+][HOLD for payment 2024-07-10] [HOLD for payment 2024-07-02] LOW: [$500] Stop using `reportAction.originalMessage` or `reportAction.message.text`[/+] on Jul 12, 2024
  17. melvin-bot commented on Jul 12, 2024

    @melvin-bot

    Upwork job price has been updated to $500

  18. puneetlath commented on Jul 12, 2024

    @puneetlath
    Contributor

    @nkdengineer please accept offer here: https://www.upwork.com/nx/wm/offer/103096930
    @c3024 please accept offer here: https://www.upwork.com/nx/wm/offer/103096934

    Both of you, please ping me on this issue when you've accepted.

  19. c3024 commented on Jul 12, 2024

    @c3024
    Contributor

    @puneetlath thanks, accepted!

  20. puneetlath commented on Jul 12, 2024

    @puneetlath
    Contributor

    @c3024 has been paid. Just waiting for you now @nkdengineer.

  21. nkdengineer commented on Jul 12, 2024

    @nkdengineer
    Contributor

    @puneetlath Accepted thanks 🙇

  22. puneetlath commented on Jul 12, 2024

    @puneetlath
    Contributor

    All paid. Thanks everyone!

  23. deleted a comment from melvin-bot on Jul 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Awaiting PaymentAuto-added when associated PR is deployed to productionBugSomething is broken. Auto assigns a BugZero manager.ImprovementItem broken or needs improvement.InternalRequires API changes or must be handled by Expensify staffWeeklyKSv2

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions