docs(plan): page-free reads wave - goal, predict, scenario, plan

docs/superpowers/plans/2026-11-10-flow-reader/GOAL.md

@@ -0,0 +1,78 @@
+# Goal: page-free reads wave (FlowReader)
+
+**Milestone:** Wire resilience (#1229) · **Evidence base:** #1192 (PR #1205), #1217 (PR #1219),
+#1221 (PR #1231), #1225 (PR #1232), #1218 (PR #1230) · **Bug in scope:** #1233
+
+## Outcome
+
+gflow reads a Flow balance, a clip's status and result, and a project's listing **without
+starting Chrome** whenever the profile's cookies are accepted, and falls back to today's page
+path whenever they are not, so no read command ever fails because the direct path failed.
+Submits, uploads and anything that needs reCAPTCHA stay on the browser.
+
+## Amended by predict (CAUTION, [PREDICT.md](PREDICT.md))
+
+- **Shape:** no `FlowReader` Protocol yet. Instead, `api/flow_wire.py` (plain functions plus one
+  `WireReadUnavailable(reason)`), called from `services/credits.py::_fetch_once` **before**
+  `FlowApiClient` is built. The Protocol arrives with its second consumer.
+- **Scope:** the wire reads **credits only** in this wave. `record` (as29s) lands with #1194, and
+  `listing` (Zzl0ze) with #1195/#1199: no command reads them without a page today.
+- **G2** applies to credits (CLI and MCP).
+- **G6** is satisfied by CLI `doctor` plus the MCP `gflow_auth_status` health field; `doctor` has
+  no MCP twin (#542).
+- **New G9:** a run of wire reads leaves the browser session valid (e2e). A session loss in
+  experiment E2 is a STOP for the wave.
+- **Order:** #1140 (duplicate #1233) lands first, so G1 is measured against the fixed baseline.
+
+## Shape as first proposed (superseded where the section above differs)
+
+- A new narrow port `FlowReader` (`credits()`, `listing(project_id)`, `record(workflow_id)`),
+  separate from `FlowTransportStrategy`, which stays the generation port.
+- Two adapters: `PageReader` (today's code moved behind the port, no behaviour change) and
+  `WireReader` (httpx + profile cookies + `at` from a cookie GET of the shell, reusing
+  `batchexecute.py` parsers and `CreditsInfo.from_nzlxg`).
+- A fallback composer: wire first, page on any classified wire failure, with the reason logged.
+
+## Exit criteria (each is measured, not asserted)
+
+| # | Criterion | How it is proven |
+|---|---|---|
+| G1 | `gflow credits user` on `denon82`, no Chrome open: p50 ≤ 3 s over 5 runs (baseline ~40 s) | timed live run, pasted in the PR |
+| G2 | credits, clip status/result and listing launch **zero** Chrome processes when the wire is healthy | e2e asserts the page factory was never called |
+| G3 | every wire failure class falls back and the command still succeeds: signed-out shell (no `SNlM0e`), 401, 400, unparseable reply, cookie store unreadable/locked, network error | one offline test per class + one e2e that forces the fallback (cookie-free arm) |
+| G4 | browser close no longer waits out its 30 s timeout (#1233) | Bug Lane: failing test first, then timed live run |
+| G5 | a wire read sends no more requests than the page read it replaces (≤ 1 GET + 1 POST per read, `at` reused within a process, no retry on 4xx, poll interval ≥ the app's own 5 s) | request count asserted in the e2e |
+| G6 | `gflow doctor` reports wire health (cookies accepted / signed-out shell / build label) and the CLI and MCP surfaces agree | doctor e2e + parity tests |
+| G7 | an operator kill switch turns the wire path off | offline test |
+| G8 | docs: ARCHITECTURE.md read-port section, KNOWN_ISSUES account-risk and fallback note, FLOW_WIRE_RPCS adapter column, `.env.template` | doc gates green |
+
+## Process (every step is a gate)
+
+`/gflow:predict` → `/gflow:scenario` → `/gflow:plan` (this directory) → worktree per ticket from
+`develop` → TDD → `/gflow:check` (ten gates) → two fresh-agent xhigh reviews (`/code-review`,
+`ponytail-review`) + one fix round → `/gflow:sonar` → `pytest -m e2e` on `denon82` → PR left
+open. A predict STOP halts the wave and reports why.
+
+## Close-out (after G1–G8 hold and the owner has merged the wave)
+
+| # | Step | Gate |
+|---|---|---|
+| R1 | Cut a release with `/gflow:release` (changelog → check → live-verify → doc-review, signed tag). The owner merges and approves the publish; the agent never merges | each internal gate is a STOP |
+| R2 | Draft a promotion post at the business level: what users get (faster answers, no browser for checks, keeps working when the direct path doesn't, fewer requests), not RPC ids or internals. Every number in it comes from the release's live-verify ledger or a spike doc, with before/after measured the same way | numbers traced to a file; `humanizer`/`stop-slop` pass; forbidden-word list clean |
+| R3 | Publish nothing. The post is written locally (Markdown + a local HTML preview opened in the browser) for the owner to edit and post | owner posts it |
+
+The post says honestly what stays the same: generation still uses a real browser, and the
+direct path is the account's own session, not a public API.
+
+## Out of scope
+
+Caching `at` across processes (waits for S2 #1222), project create/rename over the wire (S7
+#1227), any submit, any reCAPTCHA emulation, any new traffic pattern the app does not send.
+
+## Known facts the plan must honour
+
+- `get_chrome_cookie_snapshot` filters to `labs.google` cookies (`auth/cookies.py:26`); the
+  wire needs the Google-domain cookies, so the read is new, not reused as-is.
+- Its fallback for an unreadable store launches Playwright on the profile. The wire path must
+  not use that fallback (it would start Chrome to avoid starting Chrome): unreadable store →
+  page path, under the profile lease.

docs/superpowers/plans/2026-11-10-flow-reader/PLAN.md

@@ -0,0 +1,266 @@
+# Page-free Credits Read Implementation Plan
+
+> **For agentic workers:** Run `/gflow:status --feature flow-reader` to find the next unchecked
+> task. One ticket = one worktree from `origin/develop` = one PR. TDD: red first, watched red for
+> the right reason. Run `/gflow:check` before every commit. Never merge. No AI attribution.
+
+**Goal:** `gflow credits user` / `credits list` and MCP `gflow_get_credits` read the balance
+without starting Chrome whenever the profile's cookies are accepted, and fall back to today's
+browser read whenever they are not.
+
+**Architecture:**
+- A new `api/flow_wire.py` (plain functions, one `WireReadUnavailable(reason)` exception) holds
+  the direct path: a cookie GET of the shell for `at`/`f.sid`/`bl`, then one batchexecute POST for
+  `nzlxg`, decoded by the existing `CreditsInfo.from_nzlxg`.
+- `services/credits.py::_fetch_once` calls it **before** any `FlowApiClient` exists. On
+  `WireReadUnavailable` it runs today's block unchanged.
+- The cookie read reuses `auth/cookies.py`'s fast path with a domain parameter and **no**
+  Playwright fallback.
+- Nothing in `api/transports/` or the generation port changes.
+
+**Predict verdict:** CAUTION, 4.6/10 ([PREDICT.md](PREDICT.md)). Mitigations are folded into the
+tasks below. Scenarios: [SCENARIO.md](SCENARIO.md).
+
+**Gate before Ticket A starts:** the cookie spike's E2 (rotation conflict) shows **no session
+loss**, and #1140 is in review. If E1 shows the jar is unreadable while Chrome runs, that is
+expected (fallback row 5), not a blocker.
+
+**Risk register:**
+
+| Severity | Risk | Mitigation |
+|---|---|---|
+| Critical | The reader is placed inside `FlowApiClient`, so Chrome still launches (G2 fails) | the service-level call, plus a sentinel test that `FlowApiClient` is never constructed on the healthy path |
+| Critical | A secret leaks via logs, errors or problem details | redaction widened and tested with fake values; the reason is an enum; no body text or `str(exc)` |
+| Critical | Copied cookies vs Chrome's rotation logs the account out | spike E2 gate; e2e G9 (session valid after N wire reads); kill switch |
+| High | Cookie store locked or undecryptable → every read pays wire + page | bounded wire attempt; immediate fallback; the fallback rate is logged and visible in doctor |
+| High | macOS Keychain prompt on every read | the wire defaults off on darwin |
+| High | Docs and the MCP description keep saying "opens the browser" | the claim sweep in Ticket C |
+| Medium | Flow renames `nzlxg` or changes its shape | fallback `unparseable`; the nightly RPC-DRIFT canary (#1218, PR #1230) |
+
+---
+
+## File structure
+
+### New files
+```
+src/gflow_cli/api/flow_wire.py
+  direct read: shell GET (at, f.sid, bl) + nzlxg POST; reasons enum; allow-list; budget; single-flight at
+tests/api/test_flow_wire.py
+  one test per SCENARIO row 1–8, 13, 16, 20, 23, 26 (httpx.MockTransport, fake cookies)
+tests/services/test_credits_wire_fallback.py
+  rows 10, 11, 14, 15, 17, 18, 28 (sentinel FlowApiClient)
+tests/features/credits_wire.feature + tests/features/test_credits_wire_steps.py
+  offline Scenario Outline over the G3 reasons
+tests/features/credits_wire_live.feature + tests/e2e/test_credits_wire_bdd.py
+  @e2e @e2e_auth: healthy read (no Chrome, request count), cookie-free fallback, session survives
+```
+
+### Modified files
+```
+src/gflow_cli/auth/cookies.py      domain-parameterised fast read; CookieStoreUnreadable; no Playwright fallback on this path
+src/gflow_cli/data/redaction.py    +__Secure-1PAPISID/3PAPISID, SIDCC, NID, AEC, at=, f.sid=
+src/gflow_cli/config.py            wire_reads: bool (GFLOW_CLI_WIRE_READS), default on except darwin
+src/gflow_cli/services/credits.py  wire first in _fetch_once; read_source / wire_fallback log events
+src/gflow_cli/mcp/tools.py         gflow_get_credits description; gflow_auth_status wire-health field (Ticket B)
+src/gflow_cli/services/doctor.py   wire-health finding (Ticket B)
+src/gflow_cli/cli_credits.py       help text
+.env.template, docs/CONFIGURATION.md, docs/MCP.md, docs/USAGE.md, docs/USER_GUIDE.md,
+docs/ARCHITECTURE.md, KNOWN_ISSUES.md, skills/gflow-cli/SKILL.md (+ plugin copy), llms.txt,
+docs/FLOW_WIRE_RPCS.md (adapter column, once #1219 lands), CHANGELOG.md, website/docs mirror
+```
+
+---
+
+## Ticket 0: #1140, Chrome close no longer waits out the 30 s timeout (G4)
+
+In flight (Bug Lane agent, branch `bugfix/chrome-close-hang-1140`).
+- [ ] PR open with before/after `credits user` timings and the e2e output
+- [ ] two xhigh reviews + one fix round, Sonar 0
+- [ ] #1233 comment updated with the re-time
+
+---
+
+## Ticket A: wire credits read with page fallback (G1, G2, G3, G5, G7, G9)
+
+Branch `feature/wire-credits-read`, one PR.
+
+### A1. Red tests (no production code)
+- [ ] `tests/api/test_flow_wire.py`, one test per G3 reason:
+  - `signed_out_shell` (shell without `SNlM0e`)
+  - `http_401`
+  - `http_400`
+  - `unparseable` (empty envelope, an `er` frame, `from_nzlxg` rejects)
+  - `store_unreadable` (locked, DPAPI, v20, `KeyError`, missing file)
+  - `network` (connect timeout, read timeout)
+  - `redirect` (3xx is not followed)
+- [ ] back-pressure (SCENARIO rows 29–30):
+  - 429/503 → typed `RateLimitError` / `NetworkError` with `retry_after` via
+    `_retry.parse_retry_after`, **no page fallback**, wire off for the process
+  - an `er` frame or a null-payload status → `unparseable`, with the numeric code logged
+- [ ] budget and shape:
+  - exactly 1 GET + 1 POST
+  - the second call in the same process reuses `at` (0 GETs)
+  - the rpcid allow-list refuses anything but `nzlxg`
+  - headers include `x-same-domain: 1`, origin/referer, and the profile's UA
+- [ ] the wire stays off for the process after `signed_out_shell` / `http_401`
+- [ ] `Set-Cookie` in a reply is never written anywhere
+- [ ] the cookie read is always called with an explicit `cookie_file`, and its domain scope keeps
+  only `flow.google.com` / `.google.com` cookies
+- [ ] `tests/services/test_credits_wire_fallback.py`:
+  - healthy wire → `FlowApiClient` never constructed (sentinel)
+  - each reason → the page path ran, `credits.wire_fallback reason=<enum>` logged
+  - both fail → the page's typed error surfaces
+  - `--refresh` path
+  - kill switch → no cookie read
+  - darwin default off
+  - `SecurityError` for a profile outside GFLOW_CLI_HOME before any read
+- [ ] 5 concurrent reads → 1 shell GET (single flight)
+- [ ] redaction: fake `__Secure-3PAPISID=`, `SIDCC=`, `NID=`, `AEC=`, `at=…&`, `f.sid=` all
+  redacted by `redact_error_detail`
+- [ ] watch each red for the right reason; paste the red summary in the PR
+
+### A2. BDD scaffold
+- [ ] the offline `credits_wire.feature` Scenario Outline over the 7 reasons, bound in
+  `tests/features/`
+- [ ] the live `credits_wire_live.feature` (3 scenarios from SCENARIO.md), bound from
+  `tests/e2e/test_credits_wire_bdd.py`; `test_e2e_binding_guard.py` stays green
+
+### Spike E1–E3 results (PR #1234, [doc](../../spikes/2026-11-10-cookie-jar-and-wire-cookies.md) once merged)
+- **No Chrome running:** the jar decrypts fully. A browser sends **26** cookies to flow.google.com
+  (`.google.com` domain cookies + host `flow.google.com`). With those, the shell has `SNlM0e` and
+  `nzlxg` returns 200. A minimal set exists (`OSID`, `SID`, `__Secure-1PSIDTS`, `__Secure-3PSID`),
+  but the wire sends **the browser-equivalent set**, never a hand-picked subset (G5: no traffic
+  the app does not send).
+- **Chrome running on the profile:** the jar is unreadable (errno 13, sqlite `immutable=1` fails,
+  `copy2` WinError 32). browser_cookie3 then raises **`RequiresAdminError` from its shadow-copy
+  (VSS) fallback**. The wire must **never** reach that fallback: an elevated gflow would otherwise
+  read a locked jar by snapshotting the volume, which is the lock-bypass the Security persona
+  ruled out. Classify it as `store_unreadable` before browser_cookie3 can try it (pre-check that
+  the file opens, or call it in a way that disables the fallback). A test asserts no shadow-copy
+  path runs.
+- **Rotation:** Chrome rotates `*SIDCC`, `*PSIDRTS` and `*PSIDTS`. A stale snapshot is still
+  accepted and the browser session survived (run 2 + replay; run 1 contaminated). Wire replies
+  `Set-Cookie` `SIDCC`/`__Secure-1PSIDCC`/`__Secure-3PSIDCC`; these are dropped (row 13).
+- **Encryption:** all 62 Google cookies are `v10`; none are `v20` app-bound.
+- **Open lead:** 2 of 9 browser reads after the burst exited 1 with `UnexpectedError`, with no
+  stderr and no record; 0/3 on an exact replay. The G9 e2e keeps watching. Capture stderr on every
+  live run in this wave.
+
+### A3. Cookie read (`auth/cookies.py`)
+- [ ] `read_cookies_for(profile_dir, *, host="flow.google.com") -> dict[str, str]`:
+  - the `browser_cookie3` fast path with an explicit `cookie_file`
+  - RFC 6265 domain match for the host
+  - every failure raises `CookieStoreUnreadable`
+  - never reaches `_get_chrome_cookies_playwright`
+- [ ] the existing `get_chrome_cookie_snapshot` behaviour is unchanged (its tests stay green)
+
+### A4. `api/flow_wire.py`
+- [ ] `class WireFallbackReason(StrEnum)` and `class WireReadUnavailable(Exception)` carrying
+  only the reason
+- [ ] `async def read_credits(profile_dir) -> CreditsInfo`:
+  - `to_thread` for the cookie read
+  - one `httpx.AsyncClient` (`follow_redirects=False`, connect 3 s / total 5 s)
+  - shell GET → scrape `SNlM0e` / `FdrFJe` / `cfb2h`
+  - POST `nzlxg`
+  - `batchexecute` envelope parse → `CreditsInfo.from_nzlxg`
+- [ ] the per-process `at` cache under an `asyncio.Lock`; process-wide disable after
+  signed-out/401
+- [ ] the UA: the profile's Chrome version string; read how the browser path derives it and reuse
+  that, don't hard-code one
+- [ ] import time: does not import Playwright (`python -X importtime` check in the PR)
+
+### A5. Redaction (`data/redaction.py`)
+- [ ] widen the cookie-name pattern
+- [ ] add form-encoded `at=` and `f.sid=` patterns
+
+### A6. Config (`config.py`, `.env.template`, `docs/CONFIGURATION.md`)
+- [ ] `wire_reads: bool` ← `GFLOW_CLI_WIRE_READS`, default `True` on win32/linux and `False` on
+  darwin
+- [ ] the `.env.template` entry next to `GFLOW_CLI_FLOW_HOST`; the website mirror regenerated
+
+### A7. Service (`services/credits.py`)
+- [ ] `_fetch_once`: if `wire_reads`, try `flow_wire.read_credits` → `_success`; on
+  `WireReadUnavailable` log `credits.wire_fallback` and run the unchanged `FlowApiClient` block
+- [ ] log `credits.read source=wire|page`
+- [ ] remove the stale "#795 … no fast path left" comment
+
+### A8. CLI + MCP surface
+- [ ] `cli_credits.py` help: "reads the balance directly when the saved session is accepted;
+  otherwise opens the profile's browser"
+- [ ] the `mcp/tools.py` `gflow_get_credits` `description=`, the field agents read (memory
+  `mcp-agents-read-description-not-args`)
+- [ ] the six mirror axes (`skills/check/SKILL.md` step 1b): no new option, no queued path; record
+  that in the PR
+
+### A9. Live proof (Iron Law) on `denon82`, $0
+- [ ] `pytest -m e2e_auth tests/e2e/test_credits_wire_bdd.py tests/e2e/test_credits_mcp_e2e.py
+  tests/e2e/test_credits_on_flow_bdd.py`, with the output pasted
+- [ ] G1: cold `gflow credits user` × 5 (separate processes), p50 and the decrypt / GET / POST /
+  parse breakdown pasted
+- [ ] G5: the request count from the e2e pasted
+- [ ] G9: `gflow auth status` valid after the run, pasted
+- [ ] G7: one run with `GFLOW_CLI_WIRE_READS=0` showing `source=page`
+
+### A10. Review
+- [ ] `/gflow:check` (ten gates)
+- [ ] fresh-agent `/code-review` xhigh + `ponytail-review` xhigh, then one fix round
+- [ ] `/gflow:sonar` to zero
+
+---
+
+## Ticket B: wire health in `doctor` and `gflow_auth_status` (G6)
+
+Branch `feature/wire-health`, after A. It reuses `flow_wire`, with no second implementation.
+- [ ] red tests:
+  - a `doctor` finding with outcomes `accepted` / `signed_out_shell` / `store_unreadable` /
+    `disabled`, plus the build label
+  - names and counts only (asserted against fake secrets)
+- [ ] `services/doctor.py` finding
+- [ ] the `cli_doctor.py` docstring becomes "… and, unless disabled, one read-only request to
+  flow.google.com"
+- [ ] the `gflow_auth_status` result gains `wire: {status, build}`, with the description updated
+- [ ] e2e: `gflow doctor --json` and `gflow_auth_status` on `denon82` agree; output pasted
+- [ ] `/gflow:check`, two xhigh reviews + fix round, Sonar 0
+
+## Ticket C: docs (G8)
+
+Branch `docs/wire-reads`, after A (it can ride with B if small).
+- [ ] the claim sweep (PREDICT.md, UX persona list):
+  - `cli_credits.py:73-75`
+  - `docs/MCP.md:130`
+  - `docs/USAGE.md:135`
+  - `docs/USER_GUIDE.md:425`
+  - `skills/gflow-cli/SKILL.md:64` + the plugin copy
+  - `llms.txt:6`
+  - KNOWN_ISSUES #795
+  - `docs/ARCHITECTURE.md:100`
+- [ ] the ARCHITECTURE.md "Reads without a browser" section: placement, fallback, the reasons
+  enum, when a Protocol will be introduced (with #1194)
+- [ ] a new KNOWN_ISSUES entry: the risk of copied cookies vs Chrome rotation, the fallback, the
+  kill switch, measured status (E2, S2 #1222)
+- [ ] `docs/FLOW_WIRE_RPCS.md`: a "Read directly by" column (`nzlxg` → `flow_wire`), once #1219
+  is on develop
+- [ ] CHANGELOG `[Unreleased]`; regenerate the website docs and plugin skills; doc gates green
+
+---
+
+## Cross-ticket follow-ups
+- [x] Ticket B clears a profile's signed-out mark after a successful browser read or renew. Once B
+  and C (#1237) are both on develop, update the KNOWN_ISSUES "direct credits read" entry's
+  sentence "stops trying the direct read for that profile until the process exits" to match.
+  Done in #1237 (37c39a0c); B (#1238) also lifts the mark after a `labs_unverified` renewal.
+- [x] Once #1219 is on develop, add `nzlxg`'s "read directly by `flow_wire`" note to
+  `docs/FLOW_WIRE_RPCS.md`. Done in #1237.
+
+## Close-out (GOAL R1–R3, after the owner merges A–C and #1140)
+- [ ] `/gflow:release`, including its live-verify ledger with the G1 numbers
+- [ ] a business-level post draft (local Markdown + HTML, opened locally), with every number
+  traced to the ledger or a spike doc
+- [ ] session-hardening and the reactivation prompt
+
+## Definition of done
+- [ ] G1–G9 each proven by pasted run output in a PR (G2 and G6 as amended in GOAL.md)
+- [ ] `/gflow:check` green; Sonar 0 on every PR
+- [ ] BDD covers every Critical and High row in SCENARIO.md
+- [ ] CHANGELOG and docs updated; no stale "opens the browser" claim
+- [ ] no `# TODO` without an issue link