Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ the five other mirror axes), which no command here can check and no CI gate can
- Documentation is a first-class deliverable. Every behavior, workflow, config, or operator-facing change must update the relevant docs or state why no docs changed in the PR/checklist. `scripts/ci/check_doc_links.py` is a merge gate.
- **A PR is not done until its SonarCloud gate is green (zero new issues).** The nine gates above are local/pre-commit; SonarCloud is server-side and runs in CI (`sonar.qualitygate.wait=true` → a red gate turns the `SonarCloud analysis` check red). Before calling a PR merge-ready, verify it with `/gflow:sonar <N>` (it is skipped on fork PRs — maintainer-checked there).
- **E2E is the decisive layer, and it is required for every behavior change that touches a Flow surface.** Run the `tests/e2e/` test that covers the change, or write one — offline-green proves only that *our* code does what we think, never that Flow still behaves as captured. `/gflow:live-verify` does **not** satisfy this: it drives CLI commands by hand into a gitignored note and never runs `pytest -m e2e`. The two are complementary — an e2e test is a re-runnable regression, a live-verify ledger is a narrative record. A change touching no Flow surface at all (docs, help text, exit-code plumbing) is out of scope — say so explicitly rather than leaving it blank. Anything else follows the **Iron Law** above: run it, or name the external blocker that stops you. "A maintainer will run it" is a handoff, not a verification, and is only available once the blocker is named.
- E2E tests require `GFLOW_CLI_E2E_PROFILE` and `-m e2e` (excluded from the default `addopts`, so they never run by accident). Cost sub-markers: `e2e_auth` (zero credits, browser only), `e2e_image` (zero credits, daily cap), `e2e_scene`, `e2e_data`, `e2e_batch`, `e2e_character` (opt in with `GFLOW_CLI_E2E_RUN_CHARACTER=1`), `e2e_video` (spends Veo credits — opt in with `GFLOW_CLI_E2E_RUN_VIDEO=1`). Redact account identifiers and any token or cookie value before pasting output into a PR.
- E2E tests require `GFLOW_CLI_E2E_PROFILE` and `-m e2e` (excluded from the default `addopts`, so they never run by accident). Cost sub-markers: `e2e_auth` (zero credits, browser only), `e2e_image` (zero credits, daily cap), `e2e_scene`, `e2e_data`, `e2e_batch`, `e2e_character` (opt in with `GFLOW_CLI_E2E_RUN_CHARACTER=1`), the migrated refusal e2e (`e2e_image`, opt in with `GFLOW_CLI_E2E_RUN_REFUSAL=1` — it sends one corrupted reCAPTCHA token, so run it on a probe profile), `e2e_video` (spends Veo credits — opt in with `GFLOW_CLI_E2E_RUN_VIDEO=1`). Redact account identifiers and any token or cookie value before pasting output into a PR.
- Live tests (`@pytest.mark.live`) opt in via `GFLOW_LIVE=1`.

## Code style
Expand Down
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- **A generation flow.google.com refuses is now reported as that refusal, by name (#906, #873).**
Flow answers a refused submit with HTTP 200 and a `batchexecute` error envelope carrying
Google's reason (`PUBLIC_ERROR_UNUSUAL_ACTIVITY`), but gflow's parser discarded any frame
without a payload. So a refusal ended as a retryable 60 s video timeout (exit 9) or an image
"no ogiZ0b frame" wire fault (exit 7), whose remediation tells you to file a frontend bug.
It is now `WafRejectionError` (exit 10) — the class the labs path has always used for the same
reason — with a remediation that says the prompt is not the cause and nothing was charged. The
content-safety reasons map to `ContentPolicyError` (exit 5) as on the REST path. Read from the
wire, not from the grid's failure card, which looks identical for a refusal and an aborted
submit ([spike](docs/superpowers/spikes/2026-09-27-migrated-refusal-is-on-the-wire.md)).
The refusal was observed live on the image submit; the video submit reads the same framing
and is covered offline, but a live video refusal has not been captured yet.
Diagnosed by [@stgmt](https://github.com/stgmt) in #873 and #906; this change supersedes
#873 and the refusal-card half of #907.

- **`gflow auth login` no longer reports success while Google is waiting for the account to
"Confirm it's you" (#902).** In that state the cookies stay valid, so the session probe
answered in 0.4 s and the login closed Chrome before Flow's client-side hop to
Expand Down
5 changes: 5 additions & 0 deletions KNOWN_ISSUES.md
Original file line number Diff line number Diff line change
Expand Up @@ -966,6 +966,11 @@ find "$HOME/Downloads/gflow-cli" -type f -mtime +30 -delete
- **Status:** Open · **Severity:** High (blocks affected profile until WAF score decays or profile is replaced)
- **First observed:** 2026-05-23 on profile `denon82` during `gflow image batch` runs
- **Surfaces as:** `gflow_cli.errors.WafRejectionError: WAF rejection (HTTP 403): batchGenerateImages HTTP 403 — reCAPTCHA score too low or WAF fingerprint mismatch`
- **On flow.google.com** (measured 2026-09-27): the same reason arrives as HTTP 200 with a
`batchexecute` error envelope (`[7,null,[["type.googleapis.com/google.rpc.ErrorInfo",["PUBLIC_ERROR_UNUSUAL_ACTIVITY"]]]]`)
and surfaces as `WafRejectionError: … Flow refused the submit: PUBLIC_ERROR_UNUSUAL_ACTIVITY`.
Flow's grid shows *"We noticed some unusual activity… You have not been charged"*
([spike](docs/superpowers/spikes/2026-09-27-migrated-refusal-is-on-the-wire.md)).
- **structlog signature:** `ui_automation.batch_response_seen` with `status=403` followed by `ui_automation.batch_403_body` containing `'message': 'reCAPTCHA evaluation failed', 'status': 'PERMISSION_DENIED', 'reason': 'PUBLIC_ERROR_UNUSUAL_ACTIVITY'`

Distinct from the historical `aisandbox-pa` 401 (resolved in v0.7.0). The 403
Expand Down
2 changes: 1 addition & 1 deletion docs/DATA_LAYER.md
Original file line number Diff line number Diff line change
Expand Up @@ -535,7 +535,7 @@ See [`errors.py::EXIT_CODE_MAP`](../src/gflow_cli/errors.py) for the complete ex
Distinct from the above (which are errors OF the data layer), the
`operations.error_type` column stores the taxonomy of RECORDED generation
failures — the last segment of each exception's `problem_type` URI. Common
values: `waf-rejection` (HTTP 403 / WAF), `content-policy`, `auth-expired`,
values: `waf-rejection` (reCAPTCHA/WAF refusal — labs HTTP 403, or a flow.google.com `PUBLIC_ERROR_UNUSUAL_ACTIVITY` envelope), `content-policy`, `auth-expired`,
`transport-timeout`, `wire-format`, `rate-limit`, `ui-mode-unavailable`
(cohort pin), `media-attribution`. Query them with `gflow data list errors`.

Expand Down
2 changes: 1 addition & 1 deletion docs/DEBUGGING.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ First visible-and-clickable wins. Log:
| `AuthLoginTimeoutError` | 12 | User did not finish the OAuth flow in time | Run `gflow auth login` again; raise `GFLOW_CLI_AUTH_LOGIN_TIMEOUT` |
| `IdentityRecheckPendingError` | 12 | Login ended while Flow was still sending the account to `/about` because Google wants it to "Confirm it's you" (#902) | Run `gflow auth login` again, press the page's main button, and finish Google's check until the Flow app loads |
| `TransportTimeoutError` | 9 | A single API call exceeded its timeout | Retry; check Flow status |
| `WafRejectionError` | 10 | reCAPTCHA / WAF blocked the request | Wait + retry; verify session is healthy |
| `WafRejectionError` | 10 | reCAPTCHA / WAF blocked the request (`PUBLIC_ERROR_UNUSUAL_ACTIVITY`) | Stop on that profile — each rejected request can raise its score. Wait hours or switch profile; see [ACCOUNT_SAFETY § When Google does push back](ACCOUNT_SAFETY.md#when-google-does-push-back) |

For health probes from a worker process, use
`FlowApiClient.health_check()` — it returns `bool` and never raises.
Expand Down
1 change: 1 addition & 0 deletions docs/E2E_TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,7 @@ test, not a mocked proxy — the Bug Lane's step 5.
| `GFLOW_CLI_E2E_VIDEO_ASPECT` | `"landscape"` | Aspect ratio for video e2e: `landscape` or `portrait`. |
| `GFLOW_CLI_E2E_VIDEO_MODEL` | `"omni-flash"` | Veo model for i2v tests: `omni-flash` or `veo-fast`. |
| `GFLOW_CLI_E2E_VIDEO_DURATION` | `"4"` | Seconds of video to generate in i2v tests (minimum credit unit). |
| `GFLOW_CLI_E2E_RUN_REFUSAL` | `"0"` | Set to `"1"` to run the migrated refusal e2e (`e2e_image`): one image submit goes out with its reCAPTCHA token corrupted so Flow refuses it, proving the refusal is reported as `WafRejectionError`. Zero credits, but a refused token can raise the profile's WAF score for hours, so run it **alone** and on a probe profile, never before other image tests in the same session. |
| `GFLOW_CLI_E2E_RUN_ENTITY_PROV` | `"0"` | Set to `"1"` to run the entity-provenance e2e (#402) — asserts a generation Flow actually accepted records its `entity_ids` / `entity_names` in `operations.metadata_json`. Opt-in because it drives a real browser generation (no credit cost — images are free). |
| `GFLOW_CLI_E2E_BATCH_MANIFEST` | `test_assets/sample_batch.tsv` | TSV manifest for the batch image e2e. |
| `GFLOW_CLI_E2E_BATCH_JITTER` | `"1"` | Set to `"0"` to disable inter-request jitter in batch tests. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,12 @@ Two phrases are specifically **not** refusal evidence:
[[flow-credits-videos-only]] and the #721/#719 pair where a drained wallet
replaced the submit control entirely.

**On flow.google.com, read the wire, not the page.** A refused submit carries its reason in
the `batchexecute` error envelope (`rpc_errors`), and a refusal tile is structurally
identical to an aborted submit's — measured 2026-09-27, see
[[migrated-refusal-is-a-dom-card-not-a-wire-record]]. The rule below still governs any
DOM signal where no wire reason exists.

**How to apply:** a new refusal/blocked detector anchors on **structure** and reads
text only from the node it already matched — detection structural, detail textual.
`CREDITS_WARNING` (`src/gflow_cli/api/transports/migrated_composer.py:152`) is the
Expand Down
Original file line number Diff line number Diff line change
@@ -1,50 +1,48 @@
---
name: migrated-refusal-is-a-dom-card-not-a-wire-record
description: On flow.google.com a policy refusal renders as a media-grid card; the batchexecute record is a bare status-4 or never parses, so gflow reports a retryable timeout for something that will never succeed.
description: CORRECTED 2026-09-27 — a flow.google.com refusal IS on the wire (HTTP 200, a null-payload batchexecute envelope carrying gRPC status + ErrorInfo reason); gflow used to drop it in parse_frames. Read the wire, never the grid card.
---

On `flow.google.com` a content/abuse refusal is delivered to the **page**, not to
the wire. The grid renders a card —

> Failed · We noticed some unusual activity… · You have not been charged for this
> generation

— while the batchexecute side gives gflow one of three useless shapes: silence
(the submit reply never parses), a bare `status 4` record with no reason field, or
a parsed-but-empty reply.

So the driver names it wrong, in a way that matters:

| What Flow means | What gflow says today | Consequence |
|---|---|---|
| refused, never retry | `TransportTimeoutError` (exit 9, **retryable**) | the CLI, MCP and worker envelopes all invite a retry of a refusal |
| refused, rewrite the prompt | `migrated host reported status 4` | true and unactionable |
| refused | `WireFormatError` (exit 7) | the remediation string tells the user to file a frontend bug |

`ContentPolicyError` (exit 5, remediation *"rewrite and retry"*) exists and is
wired through `--json`, MCP and the worker — but its raise sites are all on the
**old REST path** (`errors.py`, docstring: "two known raise sites"). On the
migrated host a policy refusal is therefore **unreportable**, and `retryable` is
`true` when it should be `false` (`ContentPolicyError` is absent from
`RETRYABLE_ERRORS`).

**Why it is worth knowing even before it is fixed:** a refusal presenting as a
transport fault is what sends a session spiking selectors and filing frontend bugs
for Google simply saying no. Recognise the shape first — see
[[feedback-intermediate-signal-is-not-terminal]]: a null or unparsed submit reply
is an intermediate signal, and the outcome lives in the project, not in the
initiating call.

**How to apply:** reading the card is the right instinct; how it is read decides
whether the fix is an improvement or a regression. Anchor on the failed tile's
**structure** and read `inner_text()` of the node you already matched — never
scan the page body for English phrases, and never treat `"not been charged"` as
refusal evidence (it is the credit-free abort signature). The full set of traps,
with the live incident behind each, is
[[content-policy-text-scan-false-positives-on-page-chrome]].

Before implementing, spike the failed-card DOM: as of 2026-09-18 nothing in
`docs/superpowers/spikes/` describes it, so any structural anchor is currently a
guess — see [[migrated-host-driver-wire-lessons]] for what *is* measured on this
host. Diagnosed by an external contributor in PR #873; the diagnosis is sound and
the PR was blocked on its detection mechanism, not on its premise.
**The slug is historical and now wrong; it is kept because the council routing table
and `check_council_memory.py` cite it.** Until 2026-09-27 this note said a refusal on
`flow.google.com` reaches the page but not the wire. Measured that day
([spike](../spikes/2026-09-27-migrated-refusal-is-on-the-wire.md), a submit with its
reCAPTCHA token corrupted in flight), it is on the wire:

```
HTTP 200
[["wrb.fr","ogiZ0b",null,null,null,
[7,null,[["type.googleapis.com/google.rpc.ErrorInfo",["PUBLIC_ERROR_UNUSUAL_ACTIVITY"]]]],
"generic"], …]
```

A null payload, the gRPC status in slot 5, Google's `ErrorInfo` reason inside it.
`batchexecute.parse_frames` kept only frames whose payload is a string, so it silently
discarded this one — the "silence / never parses" shape the old note recorded was **our
parser**, not Flow. That is why the video path waited out a retryable 60 s
`TransportTimeoutError` and the image path raised `WireFormatError("no ogiZ0b frame")`
while the grid showed *"We noticed some unusual activity… You have not been charged"*.

**The fix reads the wire:** `batchexecute.rpc_errors` returns the envelopes
`parse_frames` skips, and `migrated_composer._submit_refusal` maps a REASON to a class —
`PUBLIC_ERROR_UNUSUAL_ACTIVITY` → `WafRejectionError` (the labs path's class for the same
reason), `CONTENT_SAFETY_REASONS` → `ContentPolicyError`. A bare status with no reason is
not a refusal: the #723 entity submit Flow queued and ran replies `[5]`.

**Why the grid card is the wrong source, measured the same day:** a submit aborted in
the browser (`route.abort()`, no reply, `net::ERR_FAILED`) renders the identical
`<flow-error-tile>` — same `warning`/`refresh`/`undo`/`delete_forever` ligatures, only the
copy differs. Structure cannot tell a refusal from an abort; the wire can (a reply with a
reason vs. no reply at all). This is the finding that blocked PR #873 and PR #907, both of
which scraped the tile — see [[content-policy-text-scan-false-positives-on-page-chrome]].

**"Unusual activity" is not a content verdict.** It is reCAPTCHA Enterprise scoring the
browser profile. Never route it to `ContentPolicyError`'s "rewrite the prompt"
remediation. #906 reported it on every automated submit on one account; the stealth
spike (PR #908, `2026-09-27-stealth-fingerprint-delta.md`) found
`playwright-stealth` changes no automation tell on gflow's context, so do not add a
stealth dependency on the strength of #906.

Diagnosed by an external contributor (stgmt) in #873 and #906; the diagnosis was right
from the start. See [[migrated-host-driver-wire-lessons]] for the rest of this host's
wire.
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# A flow.google.com refusal is on the wire, and gflow was discarding it (2026-09-27)

**Question (#873, #906, #907).** On the migrated host a refused generation surfaces as a
60 s video `TransportTimeoutError` or an image `WireFormatError("returned no ogiZ0b
frame")`, while Flow's grid shows *"We noticed some unusual activity"*. Two PRs tried to
scrape that card. Is the refusal's reason in the batchexecute reply, and do we drop it?

**Answer: yes and yes.** Google replies HTTP 200 with an error envelope naming the reason;
`batchexecute.parse_frames` discards any frame whose payload is not a string.

**Instrument.** `scripts/dev/spike_migrated_refusal_envelope.py`, through
`FlowApiClient.generate_image` on profile `ci-probe` (a probe account, chosen because a
refused token can raise a profile's WAF score). One image, benign prompt. Readings were
pre-registered in the script docstring. Zero credits. Evidence:
`scripts/dev/_spike_out/spike_migrated_refusal_envelope_20260927_21{2648,2825}.json`
(gitignored).

| Arm | What left the browser | Reply | `parse_frames` | gflow raised | Grid tile |
|---|---|---|---|---|---|
| `tamper` | the `ogiZ0b` submit with its 2 425-char reCAPTCHA token corrupted (tail reversed) | **HTTP 200** at 5.2 s: `["wrb.fr","ogiZ0b",null,null,null,[7,null,[["type.googleapis.com/google.rpc.ErrorInfo",["PUBLIC_ERROR_UNUSUAL_ACTIVITY"]]]],"generic"]` | `[]` | `WireFormatError: migrated image submit returned no ogiZ0b frame` | `<flow-error-tile>` · *"We noticed some unusual activity…"* |
| `abort` | nothing — `route.abort()` | none; `requestfailed net::ERR_FAILED` | — | `TransportTimeoutError` after 180 s | `<flow-error-tile>` · *"Sorry, this image failed to generate."* |

Both tiles carry the same ligatures (`warning`, `refresh`, `undo`, `delete_forever`) and
the same custom-element structure; only the copy differs.

## Reading

1. **The refusal is on the wire.** Status 7 (`PERMISSION_DENIED`) with `ErrorInfo` reason
`PUBLIC_ERROR_UNUSUAL_ACTIVITY` — the same reason the labs REST path returned as an
HTTP 403 body (KNOWN_ISSUES, `WafRejectionError`).
2. **We dropped it.** `parse_frames` requires `item[2]` to be a string; the envelope's is
`null`. Every caller therefore saw an empty reply. Root cause:
`src/gflow_cli/api/transports/batchexecute.py` `parse_frames`, reached from both submit
observers in `migrated_composer.py`.
3. **The DOM cannot discriminate; the wire can.** A refusal and an abort render the same
tile. A refusal has a reply with a reason; an abort has no reply. This confirms the
#873 re-review finding that blocked tile-scraping.

## Not measured

- **The video submit's envelope.** Measured on `ogiZ0b` (image) only, to spend no
credits. The video observer reads the same `batchexecute` framing and is covered offline;
a live video refusal would cost credits if the token were accepted.
- **Content-safety reasons on this host.** No `PUBLIC_ERROR_UNSAFE_*` refusal was
provoked; the mapping to `ContentPolicyError` reuses the REST path's reason set.
- **What #906's account sees.** This provokes the refusal; it does not explain why one
account gets it on every automated submit.
Loading
Loading