Skip to content

Streamline testing every pull request on all platforms #13558

Closed

Description

@AndrewGable

Problem

Over the last year or so, we have seen many regressions introduced in our application. At best, this means that deployers of our application have to spend time tracking the source of the regression and fixing it while it’s still only deployed to our staging environment, and at worst these regressions are deployed to our production app.

Furthermore, we’ve got a single codebase where most code changes affect all platforms (iOS, Android, macOS desktop, web, and mobile web). To help prevent these regressions, we need all pull requests to be tested on all platforms. However, building the app on all these platforms takes a lot of time – even for a simple 1-line change you could easily spend 30-45 minutes building the app on all different platforms, running the testing steps, and collecting screen recordings as proof that you’ve done your due diligence. This takes up a lot of our team’s time, and is also just usually not very fun.

Solution

Build a CI integration that performs a release build of the app on every platform for every pull request. The integration would leave a comment on the pull request when it’s created, including links to the built code:

  • On web, this would likely be a URL to the site built from the pull request head, deployed to an S3 bucket (similar to our actual production deploy)
  • The desktop app for the PR could be downloaded and installed from the web URL in the previous step
  • On Android, maybe this could use the Expo Go client or internal app sharing of some clicking the link or scanning the QR code would install the build on your device or emulator
  • On iOS, we could have a solution similar to the one used for Android

This comment would update anytime new code is pushed to the pull request. A good example of a similar CI integration exists in the react-navigation repo: example

Activity

  1. added
    InternalRequires API changes or must be handled by Expensify staff
    on Dec 13, 2022
  2. self-assigned this
    on Dec 13, 2022
  3. locked and limited conversation to collaborators on Dec 13, 2022
  4. AndrewGable commented on Dec 13, 2022

    @AndrewGable
    ContributorAuthor

    @staszekscp and I are working on this. Can you link to this on all future PRs @staszekscp? Thanks!

  5. AndrewGable commented on Dec 13, 2022

    @AndrewGable
    ContributorAuthor

    @staszekscp I tested #13433 and here are the results:

    1. Looks like workflow dispatch did not work ❌ https://github.com/Expensify/App/actions/runs/3689373686
    2. PR Test did pass ✅ https://github.com/Expensify/App/actions/runs/3689448500/jobs/6245401950
    3. However clicking the desktop link does not work, attached error here:

    Screenshot 2022-12-13 at 3 07 06 PM

  6. unlocked this conversation on Jan 6, 2023
  7. AndrewGable commented on Jan 6, 2023

    @AndrewGable
    ContributorAuthor

    High level: this is working! Going to close for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

InternalRequires API changes or must be handled by Expensify staffMonthlyKSv2ReviewingHas a PR in review

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions