Skip to content

Create a GitHub Action to Update PR descriptions with code coverage metrics #64206

Closed
2 of 2 issues completed

Description

@blimpich

Background

Back in November we started requiring unit tests for App changes. We updated the review checklist and proposal template to include a section on adding unit tests, and announced to contributors that they should be prioritizing writing unit tests both in their proposals and their code reviews

I compared our total Jest code coverage from 6 months ago to today:

Metric 6 Months Ago Today Change
Statements 25.77% 32.29% +6.52%
Branches 21.94% 29.50% +7.56%
Functions 22.48% 27.71% +5.23%
Lines 26.27% 32.81% +6.54%
Total Avg 24.62% 30.58% +5.96%

While this shows an objective increase in code coverage, it's only ~6%. Subjectively, many proposals and PRs still don't include unit tests.


Problem

A lack of unit tests and a lack of knowledge on how well tested our PRs are leads to:

  • More regressions
  • More deploy blockers
  • Wasted engineering time

Solution

Build a new GitHub Action that automatically updates each Expensify/App PR description with a "Test Coverage" section. This section should:

  • Show code coverage metrics related to the PR's changes. This should be concise and informative.
  • (ideally) link to an artifact that jest creates with the coverage report (coverage/lcov-report/index.html) so that the PR author can dig into exactly what isn't covered by current tests.
  • The solution should not require a certain level of coverage. It is purely informative, not a check on what can and can't be merged.

This will give the PR Author, the C+ reviewing the changes, and the assigned internal Expensify engineer knowledge on where we lack code coverage for a given PR, which will inform our decision to add coverage or not

Since this isn't a user-facing feature the contributor and C+ who end up working on this will get a lot of creative liberty for how this will end up looking. If they need help I'll be available to provide internal engineering resources. - @blimpich


Next steps

  • get assigned C+ and contributor for this
  • go through a mini-design process with C+ and contributor for how this will look like
  • estimate how much work this will be
  • implement
  • follow up: remove test cases section from proposal template (slack)
  • follow up: do we need to automatically cleanup S3? Are we going to bloat it?
Upwork Automation - Do Not Edit
  • Upwork Job URL: https://www.upwork.com/jobs/~021934697255819850080
  • Upwork Job ID: 1934697255819850080
  • Last Price Increase: 2025-07-28
Issue OwnerCurrent Issue Owner: @
Issue OwnerCurrent Issue Owner: @blimpich

Activity

  1. self-assigned this
    on Jun 14, 2025
  2. blimpich commented on Jun 14, 2025

    @blimpich
    ContributorAuthor
  3. blimpich commented on Jun 16, 2025

    @blimpich
    ContributorAuthor

    Assigning @ikevin127 as C+ reviewer and @DylanDylann as implementer as per this thread: https://expensify.slack.com/archives/C08CZDJFJ77/p1749862652715049

  4. added
    ExternalAdded to denote the issue can be worked on by a contributor
    on Jun 16, 2025
  5. melvin-bot commented on Jun 16, 2025

    @melvin-bot
  6. changed the title [-]Create a GitHub Action to Update PR descriptions with code coverage metrics[/-] [+][$250] Create a GitHub Action to Update PR descriptions with code coverage metrics[/+] on Jun 16, 2025
  7. added
    Help WantedApply this label when an issue is open to proposals by contributors
    on Jun 16, 2025
  8. melvin-bot commented on Jun 16, 2025

    @melvin-bot

    Current assignee @ikevin127 is eligible for the External assigner, not assigning anyone new.

  9. removed
    Help WantedApply this label when an issue is open to proposals by contributors
    on Jun 16, 2025
  10. 164 remaining items

  11. added and removed
    ReviewingHas a PR in review
    on Sep 26, 2025
  12. added and removed
    ReviewingHas a PR in review
    on Sep 26, 2025
  13. blimpich commented on Sep 26, 2025

    @blimpich
    ContributorAuthor

    I've completed all the steps outlined here: #64206 (comment) and the only one left in the checklist I made here is to monitor this. I think we can close this issue for now and just deal with changes to the CodeCov bot as they come up going forward. Closing.

  14. ikevin127 commented on Sep 29, 2025

    @ikevin127
    Contributor

    Nevermind, I mixed up PRs, everything is working good 👍

  15. blimpich commented on Sep 30, 2025

    @blimpich
    ContributorAuthor

    Keeping this issue closed but posting this here for posterity: I'm still having some issues with getting the config to update the layout of the code comment. Unsure what's going on. Comment is working correctly but I want to tweak it a bit, but it's not responding to my changes. Created issue here to handle that: #71505

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

BugSomething is broken. Auto assigns a BugZero manager.DailyKSv2

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions