Skip to content

feat(papercuts): recurrence per build, soak, and automatic close #33

Description

@nohat

Part of #22. Wave 4. Blocked on #30. Needs the fork version line from docs/fork/versioning.md (that work is in progress in the primary checkout; do not duplicate it).

Problem

"Validated" is measured, not my judgment: a reproduction passes and a soak window ends with no recurrence signal (docs/fork/posture.md). Nothing measures the soak or notices recurrence, and the loop has no numbers to tell whether it works.

Research basis

Chromium's ClusterFuzz verifies a fix by re-running the crash against each new build and closes the bug itself. That is verification by effect, the same idea keyed on build.

Plan

  1. Build identity is the fork version, which sorts as semver. The sha-to-version registry stays machine-local (versioning.md), and the issue comment names the version.
  2. Second pass in the triage job (feat(papercuts): a triage job that turns new reports into issues #30): for each cluster marked fixed with fixedInVersion:
    • a new report from a build at or after that version reopens the cluster (overlay back to triaged), comments on the issue ("recurred on build X, n reports"), and appears in the digest;
    • no recurrence after the soak window closes the issue as validated with a comment.
  3. Soak default: 7 days after deploy, recorded as an assumption. Closing is reversible. A rare defect can pass by absence of use; accepted for now. An iPad-only defect stays fixed-unverified until a reproduction passes (charter).
  4. t3 papercuts stats from files, no database: reports per week, time from new to issue-linked, share of clusters with a failing test, attempts per cluster, recurrence per build, reviewer catch rate where logged. One weekly digest line.

Acceptance

  • Tests with synthetic overlays and versions: reopen on a later build, no reopen on an earlier one, close after the window, reopen after close.
  • stats runs on an empty directory without error.

Surfaces

CLI and the triage job. No client, contract, or provider change.

Related

#22, #27 (build label), #30, #32.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions