Skip to content

submit io.pilot.firecrawl v0.1.0 - #95

Merged
Alexgodoroja merged 1 commit into
mainfrom
submit/io.pilot.firecrawl
Aug 3, 2026
Merged

submit io.pilot.firecrawl v0.1.0#95
Alexgodoroja merged 1 commit into
mainfrom
submit/io.pilot.firecrawl

Conversation

@Alexgodoroja

Copy link
Copy Markdown
Collaborator

io.pilot.firecrawl v0.1.0

The complete Firecrawl v2 API — all 50 operations, generated 1:1 from Firecrawl's published OpenAPI 3.0 spec (docs.firecrawl.dev/api-reference/v2-openapi.json) — as a keyless Pilot app. 51 exposed methods (50 + firecrawl.help, plus the auto firecrawl.balance).

Auth model — managed, and why

Firecrawl has no headless signup. Verified, not assumed:

  • The official firecrawl-cli@1.19.28 implements exactly one auth flow: open /cli-auth in a browser, poll /api/auth/cli/status. There is no server-to-server account-creation endpoint.
  • /agent/auth (their agent-native door) requires a WorkOS ID-JAG. That is not self-serve: an anonymous WorkOS one-shot environment offers only authorization_code/refresh_token/device_code — no token-exchange grant, no auth.workos.bot issuance.

So this ships managed: the broker holds one partner key, users bring nothing. A partner key-mint endpoint is being discussed with Firecrawl; when it lands, backend.auth flips to provisioned and the key moves to per-user local injection in $APP/secrets.jsonno method signatures change.

Isolation (the part that needed care)

One shared Firecrawl team means Firecrawl cannot isolate Pilot users from each other — job ids, sessions, monitors and the activity feed are all team-wide. The broker enforces it instead, via internal/broker/tenancy.go:

  • creating a crawl/batch/extract/agent/scrape/session/monitor claims it for the caller;
  • referencing one is ownership-checked before forwarding;
  • team-wide lists (crawl/active, interact, monitor, team/activity) are filtered to the caller's own rows;
  • team/credit-usage and team/token-usage have the shared pool's totals redacted;
  • every allow entry is method-scoped — notably PUT /team/threat-protection is refused, because that policy is team-wide and one caller's update would change it for every Pilot user.

Verified live: the owner reads its own scrape job; a second Pilot identity gets 404 not found on the same id, and on a cancel attempt.

Testing

  • pilot-app verify — 10/10 catalogue gate checks pass.
  • firecrawl.help in socket mode — 51 methods with full parameter sets.
  • Real calls through the deployed broker: scrape, map, search, research_papers, docs_search, credit_usage, queue_status, balance.
  • Credit ledger exact: 41500 → 37350 micro-USD after 5 Firecrawl credits.
  • Cross-tenant isolation as described above.
  • demo-score: 95.8 (gate 60). TestAllSubmissionDemosValid + TestAllSubmissionNextStepsValid green.

Not tested: install via a signed catalogue. The catalogue is fail-closed on signature and the release key is a CI secret; sideload correctly refuses key.sign/net.dial. That step runs post-merge.

Known limit

The current partner key is on Firecrawl's Free plan: maxConcurrency: 2, 1000 credits/month, shared across all Pilot users. Per-user budget is sized to that (~50 credits each). A concurrency and credit increase is being requested from Firecrawl before wide launch.

POST /v2/parse is not exposed — it needs multipart/form-data, which the JSON IPC contract cannot express. firecrawl.scrape on a public document URL covers the case.

🤖 Generated with Claude Code

Full Firecrawl v2 API (50 operations from their published OpenAPI spec) as a
keyless managed app: the Pilot broker holds one partner key, verifies the
caller's ed25519 identity, meters per user, and enforces per-user resource
ownership so job ids, sessions and monitors are not visible across users.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant