Skip to content

feat(ci): add Codecov coverage reporting (#188) - #190

Open
blairg23 wants to merge 2 commits into
mainfrom
feat/codecov
Open

blairg23 wants to merge 2 commits into
mainfrom
feat/codecov

Conversation

@blairg23

@blairg23 blairg23 commented Jun 16, 2026 •

Copy link
Copy Markdown
Member

## 🧾 Title
feat(ci): add Codecov coverage reporting (#188)

🧠 Description

Wires up test coverage upload to Codecov on every push and PR so the team has visibility into coverage trends and regressions are surfaced before merge.

🧩 Changes Included

  • Updated environment configuration
  • Implemented new feature or enhancement

🎯 Purpose

This PR adds Codecov integration so coverage is tracked automatically on every CI run, with PR diff comments showing coverage delta.

🔗 Related Issues / Epics

🧪 Testing

  1. Merge and push -- verify CI runs the coverage step without error
  2. Check codecov.io/gh/ThePRIMEForge/muster-deck -- confirm report appears
  3. Open a follow-up PR -- verify Codecov posts a coverage comment

🧰 Developer Notes

  • Uses Node's built-in --experimental-coverage --coverage-reporter=lcov (Node 22 LTS)
  • fail_ci_if_error: false so missing coverage never blocks a green build
  • codecov.yml: 70% project target, 60% patch target, comments on changes only

Wires up test coverage upload to Codecov on every push and PR.

- Run Node built-in test suite with --experimental-coverage and
  --coverage-reporter=lcov to produce coverage/lcov.info
- Upload to Codecov via codecov/codecov-action@v5 using CODECOV_TOKEN
  repo secret; fail_ci_if_error=false so missing coverage never blocks
  a green build
- Add codecov.yml: 70% project target, 60% patch target, 5%/10%
  thresholds, PR comment on changes only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@blairg23
blairg23 requested a review from cmFrameShift as a code owner June 16, 2026 04:17
@blairg23 blairg23 changed the title feat(ci): add Codecov coverage reporting feat(ci): add Codecov coverage reporting (#188) Jun 16, 2026

@cmFrameShift cmFrameShift left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: the coverage step is currently a silent no-op

Good to get coverage tracking wired up, and the codecov.yml config looks fine on its own (70/60 targets, require_changes on the comment). The problem is in the CI step. I ran it against a live Node, and the failure mode is sneaky: it passes CI green while running no tests and uploading no coverage.

Blocking: the Node coverage flags do not exist

node --experimental-coverage --coverage-reporter=lcov --test ... || true

Node rejects both flags:

node: bad option: --experimental-coverage
node: bad option: --coverage-reporter=lcov

The correct flags, which I verified produce a real coverage/lcov.info:

  • --experimental-test-coverage, not --experimental-coverage
  • --test-reporter=lcov. lcov is a reporter, not a coverage flag.
  • --test-reporter-destination=coverage/lcov.info

Here is the trap. The command ends in || true, so Node's "bad option" exit code of 1 gets swallowed and the step exits 0. No tests run and no lcov.info is written. The Codecov step then finds no file, but fail_ci_if_error: false keeps it green. So the green check is not evidence that any of this works.

Should fix before merge

  • || true also masks real test failures. fail_ci_if_error: false already covers "missing coverage never blocks." But || true on the test command means failing tests can never break the build either, which I do not think is intended. Drop it, or scope the non-blocking behavior to the upload step only.
  • Missing mkdir -p coverage. The lcov reporter errors with ENOENT until the destination directory exists.
  • The inline glob list duplicates the one already in the test script in package.json. Two copies will drift. Better to add a test:coverage script and have CI call npm run test:coverage, so the file list lives in one place.
  • With only --test-reporter=lcov, the CI log shows no per-test pass or fail output. Add a second --test-reporter=spec --test-reporter-destination=stdout if you want both.

Minor, worth checking

  • Confirm secrets.CODECOV_TOKEN is set in the repo Actions secrets, or uploads fail. Given fail_ci_if_error: false, they fail silently.
  • Glob no-match: if a directory ever has no test file, the shell passes the literal glob to Node and it errors. Currently hidden by || true.
  • Node version is fine. CI uses lts/*, and these flags need Node 22 or newer.

Suggested shape

- name: Test with coverage
  run: |
    if npm run | grep -qE '^[[:space:]]+test'; then
      mkdir -p coverage
      npm run test:coverage
    else
      echo "No test script defined; skipping tests."
    fi

with test:coverage in package.json carrying the corrected flags and the single copy of the test glob list.

blairg23 added a commit to blairg23/repo-scaffold that referenced this pull request Jun 30, 2026
) (#225)

## 🧾 Title
Include general conversation comments in pr list-comments

## 🧠 Description
This pull request fixes pr list-comments to return all PR comments, not
just inline review comments.

## 🧩 Changes Included
- [x] Implemented new feature or enhancement
- [x] Improved error handling or logging

## 🎯 Purpose
pr list-comments only queried /pulls/{number}/comments (inline review
comments), silently dropping general PR conversation comments stored at
/issues/{number}/comments. This meant agents and users couldn't see
comments left in the PR conversation tab -- discovered when cmFrameShift
left a comment on ThePRIMEForge/muster-deck#190 and pr list-comments
--json returned [].

## 🔗 Related Issues / Epics
- **Epic:** #48
- **Issue:** #224
- Closes #224

## 🧪 Testing
1. Run: poetry run repo-scaffold pr list-comments --repo
ThePRIMEForge/muster-deck --pr-number 190
2. Verify cmFrameShift's general comment appears in the output
3. Run with --json and confirm both inline review comments and general
comments are present
4. poetry run pytest tests/test_github_api.py -q -- four new tests cover
merge, sort, and per-endpoint error propagation

## 🧰 Developer Notes
- github_api.py: pr_list_comments now fetches both
/pulls/{number}/comments and /issues/{number}/comments, merges and sorts
by created_at
- cli.py: display branch checks for path field presence to distinguish
inline vs general comments
- Help text and error messages updated to reflect broader scope

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous coverage step used non-existent Node flags which Node
rejected with exit code 1, silently swallowed by || true. No tests ran
and no lcov.info was written, but CI stayed green because
fail_ci_if_error: false on the Codecov upload hid the missing file.

- package.json: add test:coverage script with correct flags
  (--experimental-test-coverage, --test-reporter=lcov with destination,
  --test-reporter=spec to stdout) so the glob list lives in one place
- ci.yml: call npm run test:coverage instead of inline node command,
  add mkdir -p coverage before the run, drop || true so real test
  failures break the build

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@blairg23

Copy link
Copy Markdown
Member Author

Fixed in 3dd70be. Added test:coverage script to package.json with the correct Node flags (--experimental-test-coverage, --test-reporter=lcov/spec), dropped || true, added mkdir -p coverage, and updated ci.yml to call npm run test:coverage so the glob list lives in one place.

blairg23 added a commit to blairg23/repo-scaffold that referenced this pull request Jun 30, 2026
…228)

## 🧾 Title
Document all three PR feedback endpoints in AGENTS.md SOP

## 🧠 Description
This pull request adds a "Reading all PR feedback" section to AGENTS.md
above the review thread SOP.

## 🧩 Changes Included
- [x] Added/updated documentation

## 🎯 Purpose
GitHub stores PR feedback across three endpoints (pr review-threads, pr
list-comments, pr reviews). Agents were only checking the first two
before concluding a PR had no feedback. This caused cmFrameShift's
review submission on ThePRIMEForge/muster-deck#190 to be missed
entirely, delaying the fix by a full session.

## 🔗 Related Issues / Epics
- **Epic:** #184
- **Issue:** #227
- Closes #227

## 🧪 Testing
1. Read AGENTS.md and confirm the "Reading all PR feedback" section
appears above "Review thread SOP"
2. Confirm all three commands are listed with example syntax
3. Confirm the note that pr check-sop only covers endpoint 1 is present

## 🧰 Developer Notes
- Docs-only change, no code touched

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Codecov integration for test coverage tracking

2 participants