Skip to content

[Findings] Calibrate the local waste detector (C1) #83

Closed

Description

@AndresL230

Summary

Measure the false-positive rate of every waste detector against the benchmark corpus. Tighten, downgrade, or remove any detector with FPR > 30%.

Why

Today there is no measured false-positive rate for any waste detector (n_plus_one, unbounded_loop, polling_no_backoff, missing_cache_guard, unbatched_parallel). User trust in findings depends on calibrated precision.

Current state

src/scanner/local-waste-detector.ts produces findings; src/scanner/python-waste-detector.ts mirrors for Python; specific detectors in src/ast/waste/*. Severity is hardcoded per detector.

What to do

  1. Take benchmark corpus (D1). Manually label every waste finding as TP or FP.
  2. Compute FPR per detector type.
  3. For each detector with FPR > 30%, inspect false positives. Likely categories:
    • "Unbounded loop": const array / Object.keys() of a literal
    • "Missing cache guard": cache check in a separate function called inside the loop
    • "Polling without backoff": single retry, not polling
    • "N+1": parallelized with Promise.all
  4. Add AST guards that distinguish each.
  5. Re-measure FPR. Iterate.

Acceptance criteria

  • Each detector has documented FPR in docs/accuracy/findings.md
  • No detector has FPR > 30%
  • FPR re-measured on every benchmark CI run; regressions fail the build
  • Documented exceptions for by-design conservative detectors

Depends on

  • D1 (benchmark corpus)

Reference

Full design: https://github.com/recost-dev/extension/blob/main/docs/accuracy/findings.md#c1-calibrate-the-local-waste-detector

Activity

  1. added 4 commits that reference this issue on May 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions