Repository navigation
Description
Background
Back in November we started requiring unit tests for App changes. We updated the review checklist and proposal template to include a section on adding unit tests, and announced to contributors that they should be prioritizing writing unit tests both in their proposals and their code reviews
I compared our total Jest code coverage from 6 months ago to today:
| Metric | 6 Months Ago | Today | Change |
|---|---|---|---|
| Statements | 25.77% | 32.29% | +6.52% |
| Branches | 21.94% | 29.50% | +7.56% |
| Functions | 22.48% | 27.71% | +5.23% |
| Lines | 26.27% | 32.81% | +6.54% |
| Total Avg | 24.62% | 30.58% | +5.96% |
While this shows an objective increase in code coverage, it's only ~6%. Subjectively, many proposals and PRs still don't include unit tests.
Problem
A lack of unit tests and a lack of knowledge on how well tested our PRs are leads to:
- More regressions
- More deploy blockers
- Wasted engineering time
Solution
Build a new GitHub Action that automatically updates each Expensify/App PR description with a "Test Coverage" section. This section should:
- Show code coverage metrics related to the PR's changes. This should be concise and informative.
- (ideally) link to an artifact that jest creates with the coverage report (
coverage/lcov-report/index.html) so that the PR author can dig into exactly what isn't covered by current tests. - The solution should not require a certain level of coverage. It is purely informative, not a check on what can and can't be merged.
This will give the PR Author, the C+ reviewing the changes, and the assigned internal Expensify engineer knowledge on where we lack code coverage for a given PR, which will inform our decision to add coverage or not
Since this isn't a user-facing feature the contributor and C+ who end up working on this will get a lot of creative liberty for how this will end up looking. If they need help I'll be available to provide internal engineering resources. - @blimpich
Next steps
- get assigned C+ and contributor for this
- go through a mini-design process with C+ and contributor for how this will look like
- estimate how much work this will be
- implement
- follow up: remove test cases section from proposal template (slack)
- follow up: do we need to automatically cleanup S3? Are we going to bloat it?
Upwork Automation - Do Not Edit
- Upwork Job URL: https://www.upwork.com/jobs/~021934697255819850080
- Upwork Job ID: 1934697255819850080
- Last Price Increase: 2025-07-28
Issue Owner
Current Issue Owner: @Issue Owner
Current Issue Owner: @blimpichMetadata
Metadata
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
blimpich commentedon Jun 14, 2025
post to #expert-contributors here
blimpich commentedon Jun 16, 2025
Assigning @ikevin127 as C+ reviewer and @DylanDylann as implementer as per this thread: https://expensify.slack.com/archives/C08CZDJFJ77/p1749862652715049
melvin-bot commentedon Jun 16, 2025
Job added to Upwork: https://www.upwork.com/jobs/~021934697255819850080
[-]Create a GitHub Action to Update PR descriptions with code coverage metrics[/-][+][$250] Create a GitHub Action to Update PR descriptions with code coverage metrics[/+]melvin-bot commentedon Jun 16, 2025
Current assignee @ikevin127 is eligible for the External assigner, not assigning anyone new.
164 remaining items
blimpich commentedon Sep 26, 2025
I've completed all the steps outlined here: #64206 (comment) and the only one left in the checklist I made here is to monitor this. I think we can close this issue for now and just deal with changes to the CodeCov bot as they come up going forward. Closing.
ikevin127 commentedon Sep 29, 2025
Nevermind, I mixed up PRs, everything is working good 👍
blimpich commentedon Sep 30, 2025
Keeping this issue closed but posting this here for posterity: I'm still having some issues with getting the config to update the layout of the code comment. Unsure what's going on. Comment is working correctly but I want to tweak it a bit, but it's not responding to my changes. Created issue here to handle that: #71505