Repository navigation
feat(review): persist provider-neutral execution provenance - #182
gutierrezje wants to merge 1 commit into
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 7c68705. Configure here.
| "Cursor reasoning overrides are not supported. Omit --reasoning, select Cursor with `diffowl backend cursor`, then clear its local override with `diffowl reasoning --reset`, and remove deprecated reasoning.effort from .diffowl.yml.", | ||
| ); | ||
| } | ||
| const evidence = createUnknownReviewRuntimeEvidence(); |
There was a problem hiding this comment.
Cursor early exits skip provenance
Medium Severity
Already-aborted reviews and unsupported reasoning overrides throw before evidence is created or onProvenance runs, so the journal never records cursor-sdk identity or a cancelled/protocol failureCategory. Codex and OpenCode publish a snapshot first; Cursor only classifies failures inside the later try/catch.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 7c68705. Configure here.
| if (error instanceof ReviewTimeoutError) return "timed-out"; | ||
| if (error instanceof SchemaValidationError) return "validation"; | ||
| return "unknown"; | ||
| } |
There was a problem hiding this comment.
OpenCode misclassifies known failures
Medium Severity
classifyFailure only maps cancellation, timeout, and SchemaValidationError. OpenCode already detects quota or rate-limit exhaustion and rejects with a generic Error, so those runs persist failureCategory: "unknown" instead of quota. Other adapter-identified session failures collapse the same way.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 7c68705. Configure here.
This branch has not been deployed
1 participant
What
Closes #106.
Reviews now retain provider-neutral execution provenance in SQLite and JSON: observed runtime/model identity, native correlation IDs, policy, prompt/configuration/context hashes, validation attempts, reported usage, and failure classification. Unavailable facts remain unknown. The existing operation, reviewer role, and optional cohort ID remain the identity anchors.
How
evidence_json. Preserve published migrations 1–7 and historical execution records. Add evidence to the existing JSON projection without changing its required telemetry fields.docs/review-provenance.md.Start with
src/review/execution-evidence.ts, then the execution journal/repository and JSON mapper; adapter changes supply the observations.Why
A requested model name and a successful report cannot establish whether two reviews used comparable inputs, policies, runtimes, or retry paths. Local provenance makes those differences inspectable without storing raw prompts, credentials, account data, or provider responses in the evidence envelope. Native IDs are correlation fields, not conversation-resumption handles.
Evidence
pnpm run lintandpnpm exec tsc --noEmit: passed.GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=commit.gpgsign GIT_CONFIG_VALUE_0=false pnpm run test: 1,123 passed, 7 skipped across 89 passing test files. Includes privacy stripping, JSON projection, durable roundtrips, failure/cancellation/timeout snapshots, and migration compatibility. The prior isolated Codex process-teardown failure passed on rerun and in the final full suite.node dist/cli.js review --staged --depth shallow --format json: Codexgpt-5.6-lunaon CLI 0.154.0 and OpenCodeopencode-go/deepseek-v4-flashon runtime 1.18.29 both passed. Every evidence field matched the durable SQLite record; report creation, unchanged repository state, and child/server teardown passed. OpenCode reported zero usage counters; preservation is verified, metering accuracy is not claimed.Review status and migration risk
The automatic post-commit review failed before provider work because the installed reviewer supports schema 7 and the existing development database uses schema 8. The new binary also correctly rejects that database's earlier schema-8 checksum. Existing user state was preserved. Full-branch model-review coverage and a current-head readiness verdict remain pending; this PR does not claim merge acceptance.
Schema 8 is still unreleased (latest published package: 0.5.2, schema 7). Earlier development databases with a different schema-8 checksum require an explicit backup/recovery decision; they are not silently rewritten. Hosted checks will run on the published head.
No hosted analytics, credential/account management, conversation resumption, or model-quality ranking is added.
Note
Medium Risk
Touches all review backends, SQLite execution persistence, and the JSON export contract, though changes are additive with sanitization and broad test coverage.
Overview
Adds durable, provider-neutral execution provenance so reviews can be compared on inputs and runtime facts, not just requested model names.
A new typed evidence contract captures runtime/adapter identity, policy, prompt and pipeline hashes (rules, config, schema, context manifest), native correlation IDs, structured-output attempts, usage, and failure categories—with unknown fields left explicit and secrets/raw prompts stripped before persistence. Codex, OpenCode, and Cursor adapters stream snapshots through
onProvenance; the review journal writes them toevidence_jsonon the existing execution row (including failed, cancelled, and timed-out runs). Terminal executions promote to provenance schema v5 when evidence is present; JSON output adds an optionalexecution.evidenceblock without changing the required telemetry wire shape. Migration 8 gainsevidence_jsonalongside readiness coverage; older rows keepnullevidence. README anddocs/review-provenance.mddocument comparison semantics and per-backend limits.Reviewed by Cursor Bugbot for commit 7c68705. Bugbot is set up for automated code reviews on this repo. Configure here.