submit io.pilot.firecrawl v0.1.0 - #95
Merged
Merged
Conversation
Full Firecrawl v2 API (50 operations from their published OpenAPI spec) as a keyless managed app: the Pilot broker holds one partner key, verifies the caller's ed25519 identity, meters per user, and enforces per-user resource ownership so job ids, sessions and monitors are not visible across users. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
io.pilot.firecrawl v0.1.0
The complete Firecrawl v2 API — all 50 operations, generated 1:1 from Firecrawl's published OpenAPI 3.0 spec (
docs.firecrawl.dev/api-reference/v2-openapi.json) — as a keyless Pilot app. 51 exposed methods (50 +firecrawl.help, plus the autofirecrawl.balance).Auth model —
managed, and whyFirecrawl has no headless signup. Verified, not assumed:
firecrawl-cli@1.19.28implements exactly one auth flow: open/cli-authin a browser, poll/api/auth/cli/status. There is no server-to-server account-creation endpoint./agent/auth(their agent-native door) requires a WorkOS ID-JAG. That is not self-serve: an anonymous WorkOS one-shot environment offers onlyauthorization_code/refresh_token/device_code— no token-exchange grant, noauth.workos.botissuance.So this ships
managed: the broker holds one partner key, users bring nothing. A partner key-mint endpoint is being discussed with Firecrawl; when it lands,backend.authflips toprovisionedand the key moves to per-user local injection in$APP/secrets.json— no method signatures change.Isolation (the part that needed care)
One shared Firecrawl team means Firecrawl cannot isolate Pilot users from each other — job ids, sessions, monitors and the activity feed are all team-wide. The broker enforces it instead, via
internal/broker/tenancy.go:crawl/active,interact,monitor,team/activity) are filtered to the caller's own rows;team/credit-usageandteam/token-usagehave the shared pool's totals redacted;PUT /team/threat-protectionis refused, because that policy is team-wide and one caller's update would change it for every Pilot user.Verified live: the owner reads its own scrape job; a second Pilot identity gets
404 not foundon the same id, and on a cancel attempt.Testing
pilot-app verify— 10/10 catalogue gate checks pass.firecrawl.helpin socket mode — 51 methods with full parameter sets.scrape,map,search,research_papers,docs_search,credit_usage,queue_status,balance.demo-score: 95.8 (gate 60).TestAllSubmissionDemosValid+TestAllSubmissionNextStepsValidgreen.Not tested: install via a signed catalogue. The catalogue is fail-closed on signature and the release key is a CI secret; sideload correctly refuses
key.sign/net.dial. That step runs post-merge.Known limit
The current partner key is on Firecrawl's Free plan:
maxConcurrency: 2, 1000 credits/month, shared across all Pilot users. Per-user budget is sized to that (~50 credits each). A concurrency and credit increase is being requested from Firecrawl before wide launch.POST /v2/parseis not exposed — it needsmultipart/form-data, which the JSON IPC contract cannot express.firecrawl.scrapeon a public document URL covers the case.🤖 Generated with Claude Code