Skip to content

Add Weekly Backlog and Review-Cost Report Script - #3400

Merged
ptr727 merged 7 commits into
developfrom
add-backlog-metrics-report
Oct 23, 2026
Merged

ptr727 merged 7 commits into
developfrom
add-backlog-metrics-report

Conversation

@ptr727

@ptr727 ptr727 commented Oct 23, 2026

Copy link
Copy Markdown
Owner

Adds scripts/backlog_metrics.py, a weekly backlog and review-cost report for the cataloged repositories. It prints markdown or minimal HTML to stdout, reads through gh only, and is wired into no workflow, so a server timer can mail it.

The test is tests/test_backlog_metrics.py (unittest, constructed fixtures, no gh call). tests/test_cli_usage.py lists the script under NO_REQUIRED_ARGUMENT, which that test requires of every tracked entry point. scripts/README.md gets a section, since it holds no table of scripts.

Review runs match the run name field, not workflowName: a live read showed name carries "Running Copilot Code Review" and workflowName reads "Copilot".

Verification

  • uvx ruff@latest format --check scripts tests -> 57 files already formatted
  • uvx ruff@latest check scripts/backlog_metrics.py tests/test_backlog_metrics.py tests/test_cli_usage.py -> All checks passed!
  • uvx mypy@latest (pyproject.toml configures mypy and pyright, the hub validator runs mypy) -> Success: no issues found in 65 source files
  • python3 tests/test_backlog_metrics.py -> Ran 27 tests ... OK
  • python3 tests/test_cli_usage.py -> Ran 16 tests ... OK
  • python3 scripts/prose_lint.py --diff <merge-base> -> exit 0, no violations
  • python3 scripts/repo_gate.py -> no failing check
  • python3 scripts/backlog_metrics.py --bogus -> error: unrecognized arguments: --bogus, exit 2
  • Live read-only smoke run against one repository confirmed the table, label line, and billing lines render.

Local strict review ran twice at the frontier tier and is recorded against the final content. Round one fixed the review-run field, the missing UTF-8 decode (now reuses gh_graphql.run_subprocess), a count in test prose, and README wording. Round two fixed the test gaps.

Pre-existing findings, not filed

  • scripts/README.md:9: a 43-word sentence in the existing utility list. The opt-in sentence-length check reports it because this change added one word to it.

Decision for the maintainer

The window is >=START with START = today minus N days, as specified. A weekly run therefore counts the boundary day in two consecutive reports. A bounded START..yesterday range would give non-overlapping full days. Not changed here because the brief fixed the >=DATE filters.

🤖 Generated with Claude Code

ptr727 and others added 5 commits October 23, 2026 07:00
Reads issues, pull requests, workflow runs, and month billing through gh and prints a table to stdout for a server timer to mail.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings October 23, 2026 14:06
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ptr727

ptr727 commented Oct 23, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head dee2c4d8433f5c35ccd4a994cc6b39849d452397, the content this pull request carries at that commit against develop, and it recorded 14 findings.

@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.77301% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.60%. Comparing base (20b9d16) to head (be1e374).

Files with missing lines Patch % Lines
scripts/backlog_metrics.py 98.77% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3400      +/-   ##
===========================================
+ Coverage    81.33%   81.60%   +0.27%     
===========================================
  Files           18       19       +1     
  Lines        10049    10212     +163     
===========================================
+ Hits          8173     8334     +161     
- Misses        1876     1878       +2     
Flag Coverage Δ
python-3.13 81.60% <98.77%> (+0.27%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Address the unresolved documentation, invalid-JSON diagnostic, and type-suppression rationale review comments.

1 open issue
What changed in this PR

Adds a standalone Python utility that collects backlog, review-run, and billing metrics for cataloged repositories through gh, then renders Markdown or minimal HTML reports.

Changes:

  • Adds scripts/backlog_metrics.py with metric collection, aggregation, billing fallback, CLI handling, and Markdown/HTML output.
  • Adds unit tests covering collection, counting, billing, rendering, errors, and CLI behavior.
  • Registers the script in CLI usage checks.
  • Documents the utility and invocation examples in scripts/README.md.
File Description
tests/​test_cli_usage.py Registers the new script as a command-line entry point.
tests/​test_backlog_metrics.py Tests counting, billing, rendering, collection, error handling, and CLI behavior.
scripts/​README.md Documents the utility, output fields, failure behavior, and usage examples.
scripts/​backlog_metrics.py Implements backlog, review-run, billing, aggregation, and report generation.

Review effort: Lite

Comment thread scripts/backlog_metrics.py Resolved
ptr727 and others added 2 commits October 23, 2026 07:19
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@ptr727

ptr727 commented Oct 23, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head be1e374efd0eaf2dba28f67a4f9c33c30beb38f7, the content this pull request carries at that commit against develop, and it recorded 1 finding.

@ptr727
ptr727 merged commit 1d0f634 into develop Oct 23, 2026
13 checks passed
@ptr727
ptr727 deleted the add-backlog-metrics-report branch October 23, 2026 14:53
ptr727 added a commit that referenced this pull request Oct 23, 2026
## Summary

Promotes these pull requests from develop. Each carries a maintainer
decision recorded on #3391.

- [#3399](#3399): states
the write guard's threat model and documents two shapes it keeps
allowed, a quoted run-time `-C` value and an `xargs` replace string run
as a shell script (item 1).
- [#3401](#3401): lists a
pre-existing review finding in the pull request body instead of filing
an issue for it (item 2).
- [#3402](#3402): keeps
one standing handoff issue per track, edited in place each round, with a
`close` subcommand that ends a track (item 5).
- [#3400](#3400): adds a
weekly backlog and review-cost report script.
- [#3403](#3403): runs the
Copilot review once per change, at the promotion, with the attested
local pass covering a pull request into develop (item 4).
- [#3404](#3404): aligns
the remaining review rule text with the promotion-only Copilot review
(items 2 and 4).
- [#3406](#3406): fixes
the findings this promotion's first Copilot round raised on the content
above, in the `wait` note for a head no local pass covers, the
pr-review-conduct heading, and the backlog report's label line and cost
scoping.

Closes #3382
Closes #3385

## Pre-existing findings, not filed

- The [Copilot round on
4a77a1d](#3405 (review))
headline says documentation does not consistently describe current-head
Copilot refusal handling. `local_cover` in `scripts/pr_review.py`
refuses an attested head whose current Copilot round is a refusal, and
`wait` exits `41` there. The `wait` entry of the `pr_review.py`
docstring ("An attested head is covered at once.") and the
`scripts/README.md` paragraph on the defined review moments ("Where it
is attested, `wait` polls the head's checks instead.") leave that
exception out. `pr-review-conduct` Merge Gate item 2 states it ("A
refusal is not that coverage"). Both omissions are on `main` already, so
this promotion did not introduce them.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants