Skip to content

fix(ci): exclude projects without previous report from bulk compare - #1288

Merged
matejchalk merged 1 commit into
mainfrom
filter-projects-in-compare
Sep 11, 2026
Merged

fix(ci): exclude projects without previous report from bulk compare#1288
matejchalk merged 1 commit into
mainfrom
filter-projects-in-compare

Conversation

@matejchalk

Copy link
Copy Markdown
Collaborator

In monorepo mode with a bulk run-many command, a project which isn't configured in the base branch (typically a package newly added in the PR) is correctly skipped when collecting previous reports, but it was still included in the subsequent compare command. Since it has no report-before.json, code-pushup compare fails for that project, which fails the whole run-many task and with it the CI job - so no comment is posted, even though every other project compared fine.

The logs of an affected run look like this (Nx monorepo, one new package):

<✓> 14 out of 15 projects loaded previous report from artifact cache
<✓> Switched to base branch main
✖ $ npx nx run @acme/new-pkg:code-pushup -- print-config --output=…
<✓> Executing print-config for project @acme/new-pkg failed, assuming code-pushup not installed in base branch main and skipping comparison
<✓> Skipping 1 projects which aren't configured in base branch main
<✓> Collecting previous reports for 14 projects
<✓> Switched back to PR/MR branch
- $ npx nx run-many --targets=code-pushup --parallel=true --projects=…,@acme/new-pkg,… -- compare
❌ > nx run @acme/new-pkg:code-pushup compare
ENOENT: no such file or directory, open '/…/packages/new-pkg/.code-pushup/report-before.json'
 NX   Running target code-pushup for 15 projects failed

The previous reports are collected with an explicit projects filter (Collecting previous reports for 14 projects), but the compare command called runManyCommand() without any arguments, so the handler fell back to all projects.

Changes:

  • compareManyProjects passes the names of the projects which actually have a previous report to compareMany, mirroring what collectMany already does for the base branch collect.
  • Comparison is skipped entirely when no project has a previous report. Previously that case ran compare for every project, none of which had a report-before.json.

As a side effect, the compare command is now always filtered explicitly, so for pnpm/Turborepo/Yarn ≥2 it no longer runs across the whole workspace, but only over the projects which were collected.

npm workspaces and Yarn v1 support no projects filter, so they remain affected by this bug - the same pre-existing limitation applies to the previous report collect.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@pkg-pr-new

pkg-pr-new Bot commented Sep 10, 2026

Copy link
Copy Markdown

Open in StackBlitz

@code-pushup/ci

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/ci@1288

@code-pushup/cli

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/cli@1288

@code-pushup/core

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/core@1288

@code-pushup/models

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/models@1288

@code-pushup/create-cli

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/create-cli@1288

@code-pushup/axe-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/axe-plugin@1288

@code-pushup/nx-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/nx-plugin@1288

@code-pushup/coverage-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/coverage-plugin@1288

@code-pushup/eslint-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/eslint-plugin@1288

@code-pushup/js-packages-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/js-packages-plugin@1288

@code-pushup/jsdocs-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/jsdocs-plugin@1288

@code-pushup/lighthouse-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/lighthouse-plugin@1288

@code-pushup/typescript-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/typescript-plugin@1288

@code-pushup/utils

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/utils@1288

commit: 5f815aa

@matejchalk
matejchalk requested a review from vmasek September 10, 2026 18:22
@github-actions

Copy link
Copy Markdown
Contributor

Code PushUp

🤨 Code PushUp report has both improvements and regressions – compared current commit 28d014d with previous commit ba41f92.

🕵️ See full comparison in Code PushUp portal 🔍

🏷️ Categories

🏷️ Category ⭐ Previous score ⭐ Current score 🔄 Score change
Performance 🔴 29 🔴 28 ↓ −0.9
Code coverage 🟢 92 🟢 92 ↑ +0.1
Bug prevention 🟡 75 🟡 75
Code style 🟢 100 🟢 100
Security 🔴 0 🔴 0
Updates 🟡 65 🟡 65
Type Safety 🟡 67 🟡 67
Miscellaneous 🟡 67 🟡 67
Documentation 🟡 51 🟡 51
Accessibility 🟢 92 🟢 92
Best Practices 🟢 97 🟢 97
SEO 🟢 92 🟢 92
Axe Accessibility 🟡 90 🟡 90
👍 1 group improved, 👎 1 group regressed, 👍 2 audits improved, 👎 1 audit regressed, 15 audits changed without impacting score

🗃️ Groups

🔌 Plugin 🗃️ Group ⭐ Previous score ⭐ Current score 🔄 Score change
Lighthouse Performance 🔴 29 🔴 28 ↓ −0.9
Code coverage Code coverage metrics 🟢 92 🟢 92 ↑ +0.1

32 other groups are unchanged.

🛡️ Audits

🔌 Plugin 🛡️ Audit 📏 Previous value 📏 Current value 🔄 Value change
Lighthouse Total Blocking Time 🟥 1,710 ms 🟥 1,930 ms ↑ +12.9 %
Code coverage Line coverage 🟩 92.3 % 🟩 92.3 % ↑ +0.1 %
Code coverage Branch coverage 🟨 89.2 % 🟨 89.2 % ↑ +0.1 %
Lighthouse Minimizes main-thread work 🟥 8.0 s 🟥 9.3 s ↑ +16.3 %
Lighthouse Largest Contentful Paint 🟥 15.6 s 🟥 14.7 s ↓ −6 %
Lighthouse JavaScript execution time 🟥 2.9 s 🟥 3.7 s ↑ +25.1 %
Lighthouse Metrics 🟩 100% 🟩 100% ↑ +2.6 %
Lighthouse Time to Interactive 🟥 16.2 s 🟥 16.6 s ↑ +2.6 %
Lighthouse Avoids enormous network payloads 🟨 Total size was 2,723 KiB 🟨 Total size was 2,723 KiB ↑ +0.1 %
Lighthouse Remove duplicate modules in JavaScript bundles 🟥 Potential savings of 65 KiB 🟥 Potential savings of 68 KiB ↓ −43.3 %
Lighthouse Max Potential First Input Delay 🟥 1,230 ms 🟥 1,050 ms ↓ −14.4 %
Lighthouse Server Backend Latencies 🟩 360 ms 🟩 540 ms ↑ +47.4 %
Lighthouse Reduce unused CSS 🟥 Potential savings of 188 KiB 🟥 Potential savings of 188 KiB ↓ −14.3 %
Lighthouse Initial server response time was short 🟩 Root document took 580 ms 🟩 Root document took 510 ms ↓ −12 %
Lighthouse Uses efficient cache policy on static assets 🟨 31 resources found 🟨 31 resources found ↓ −0.1 %
Lighthouse First Contentful Paint 🟥 10.6 s 🟥 10.6 s ↓ −0.1 %
Lighthouse Speed Index 🟥 10.6 s 🟥 10.6 s ↓ −0.1 %
Lighthouse Network Round Trip Times 🟩 50 ms 🟩 50 ms ↓ −1.3 %

660 other audits are unchanged.

@github-actions

Copy link
Copy Markdown
Contributor

Code PushUp

🥳 Code PushUp report has improved – compared current commit 28d014d with previous commit ba41f92.

💼 Project ci

🥳 Code PushUp report has improved.

🕵️ See full comparison in Code PushUp portal 🔍

🏷️ Category ⭐ Previous score ⭐ Current score 🔄 Score change
Code coverage 🟢 92 🟢 93 ↑ +0.1

5 other categories are unchanged.

👍 1 group improved, 👍 2 audits improved

🗃️ Groups

🔌 Plugin 🗃️ Group ⭐ Previous score ⭐ Current score 🔄 Score change
Code coverage Code coverage metrics 🟢 92 🟢 93 ↑ +0.1

14 other groups are unchanged.

🛡️ Audits

🔌 Plugin 🛡️ Audit 📏 Previous value 📏 Current value 🔄 Value change
Code coverage Line coverage 🟩 93.6 % 🟩 94 % ↑ +0.4 %
Code coverage Branch coverage 🟨 82.8 % 🟨 83.1 % ↑ +0.4 %

443 other audits are unchanged.


13 other projects are unchanged.

@matejchalk
matejchalk merged commit 1fa12a5 into main Sep 11, 2026
25 checks passed
@matejchalk
matejchalk deleted the filter-projects-in-compare branch September 11, 2026 10:08
matejchalk added a commit to code-pushup/github-action that referenced this pull request Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants