Add per-minute runner billing

[epompeii]

This changeset adds Stripe meter event based runner usage billing.

[github-actions]

🤖 Claude Code Review

PR: #707
Base: devel
Head: claude/runner-per-minute-billing-0JOha
Commit: 18cf2685284d41855213a5a27ad6afc20b7b0b90

---

PR Review: Per-Minute Runner Job Billing via Stripe

Summary

This PR implements per-minute billing for runner jobs, adds OTEL metering for billing failures, hardens billing robustness with TOCTOU race prevention, removes cargo audit from CI (replaced by cargo deny), and updates dependencies.

---

Positive Highlights

• TOCTOU prevention: The billing read+write is performed under a single write_conn! lock, preventing race conditions where concurrent heartbeats could advance last_billed_minute between read and write. Well-documented with comments.
• Best-effort billing pattern: Stripe calls happen after releasing the DB write lock, so billing failures don't block job execution. The delta is "claimed" in the DB first — correct approach.
• BillingState caching: Avoids repeated DB lookups for the metered plan on every heartbeat. Sensible for a value that won't change mid-job.
• Debounced Sentry reporting: Only the first billing failure per job is reported to Sentry, avoiding alert storms.
• Comprehensive tests: elapsed_minutes and BillingDelta have thorough unit tests covering edge cases (negative input, overflow, zero elapsed, catch-up deltas).
• Deterministic tests: Uses DateTime::TEST per CLAUDE.md guidelines.

---

Issues & Concerns

1. bill_final_minutes called before job status transition (Medium)

In handle_runner_message, for the Completed/Failed/Canceled paths, bill_final_minutes is called after handle_completed/handle_failed/handle_canceled, which transition the job to a terminal status. But in bill_final_minutes_inner, UpdateJob::final_billing uses execute() (no status filter) — this is intentionally safe per the comment. However, in the disconnect/timeout paths at the bottom of execute_job_loop, bill_final_minutes is called before handle_timeout (for timeout) but without any status transition (for disconnect). This means:

• On disconnect (lines 1343, 1365, 1405): billing runs but the job may still be in Running status — is that intentional? Who transitions the job to a terminal state on disconnect?
• On WebSocket error (line 1365): same concern.

This isn't a billing bug per se, but it could lead to billing for minutes on a job that's stuck in Running without cleanup.

2. elapsed_minutes minimum of 1 for BillingDelta on first heartbeat (Low)

elapsed_minutes always returns at least 1, even at 0 seconds elapsed. Combined with BillingDelta::new returning delta: 1 on the very first heartbeat (when last_billed_minute is None/0), this means a job is billed 1 minute immediately upon the first heartbeat — which could be seconds after starting. This is documented and intentional (minimum billing), but worth confirming this aligns with the pricing model shown to customers.

3. metered_plan_id uses auth_conn! inside BillingState (Low)

BillingState::metered_plan_id uses auth_conn!(context) for the read. This is fine per CLAUDE.md (read-only authenticated access), but note that this read happens outside the write_conn! block in handle_heartbeat. If the plan were to be updated between the write_conn! block (where billing delta is calculated) and the auth_conn! read (where plan ID is fetched), you could theoretically bill against a stale plan. In practice this is negligible since plan changes mid-job are extremely unlikely, and the plan is cached after first lookup anyway.

4. Missing #[cfg(feature = "plus")] guard on billing types? (Low)

BillingState, BillingDelta, CachedMeteredPlan, and the billing functions are defined without a #[cfg(feature = "plus")] gate. The file itself is gated by the crate's feature structure (plus/api_runners), so this is likely fine, but worth confirming these don't leak into the open-source build.

5. cargo deny set to continue-on-error: true (Low-Medium)

Making cargo deny non-blocking in CI (continue-on-error: true) while simultaneously removing cargo audit means there's no blocking supply-chain security check left. This was presumably done to unblock CI, but it weakens the security posture. Consider re-enabling as blocking once the underlying advisory is resolved.

---

Minor / Style

• clippy.toml allow-panic-in-tests = true: Reasonable cleanup — removes the need for #[expect(clippy::panic)] on many test functions. Consistent with the test attribute removals.
• Rename METRICS_METER_EVENT_NAME → METRICS_METER_NAME: Good — the old name was misleading since it's a meter name, not an event name.
• DESIGN.md / PLAN.md updates: Accurately reflect the implemented billing flow and remove the resolved TODO.

---

Verdict

The core billing logic is well-designed with proper race condition handling and defensive error management. The main area to revisit is ensuring job cleanup on disconnects so that billed jobs don't get stuck in non-terminal states. The cargo deny change should be tracked as temporary.

---

_{Model: claude-opus-4-6}

[github-actions]

Bencher Report

Branch	claude/runner-per-minute-billing-0JOha
Testbed	ubuntu-22.04

Click to view all benchmark results

Benchmark	Latency	Benchmark Result microseconds (µs) (Result Δ%)	Upper Boundary microseconds (µs) (Limit %)
Adapter::Json	📈 view plot 🚷 view threshold	3.80 µs (+9.47%) Baseline: 3.48 µs	4.62 µs (82.31%)
Adapter::Magic (JSON)	📈 view plot 🚷 view threshold	3.75 µs (+8.56%) Baseline: 3.46 µs	4.52 µs (82.97%)
Adapter::Magic (Rust)	📈 view plot 🚷 view threshold	25.24 µs (-1.59%) Baseline: 25.65 µs	31.05 µs (81.30%)
Adapter::Rust	📈 view plot 🚷 view threshold	2.84 µs (-0.34%) Baseline: 2.85 µs	3.34 µs (85.15%)
Adapter::RustBench	📈 view plot 🚷 view threshold	2.82 µs (-1.07%) Baseline: 2.85 µs	3.32 µs (84.92%)
head_version_insert/batch/10	📈 view plot 🚷 view threshold	99.39 µs (-0.76%) Baseline: 100.15 µs	120.45 µs (82.51%)
head_version_insert/batch/100	📈 view plot 🚷 view threshold	235.28 µs (-0.87%) Baseline: 237.33 µs	266.98 µs (88.12%)
head_version_insert/batch/255	📈 view plot 🚷 view threshold	457.70 µs (-0.76%) Baseline: 461.21 µs	492.34 µs (92.96%)
head_version_insert/batch/50	📈 view plot 🚷 view threshold	160.98 µs (+0.35%) Baseline: 160.42 µs	182.51 µs (88.20%)
threshold_query/join/10	📈 view plot 🚷 view threshold	141.98 µs (-1.69%) Baseline: 144.42 µs	170.07 µs (83.48%)
threshold_query/join/20	📈 view plot 🚷 view threshold	157.91 µs (-0.42%) Baseline: 158.57 µs	186.03 µs (84.88%)
threshold_query/join/5	📈 view plot 🚷 view threshold	134.66 µs (-1.29%) Baseline: 136.42 µs	159.81 µs (84.26%)
threshold_query/join/50	📈 view plot 🚷 view threshold	197.76 µs (-1.11%) Baseline: 199.99 µs	232.82 µs (84.94%)

🐰 View full continuous benchmarking report in Bencher