feat(rate-limit): bypass per-IP throttle for known AI crawler UAs (cherry-pick)

[Flotapponnier]

Cherry-pick of #615 onto main. SEO-critical, hot path to ship before the
next AI crawl cycle.

AI crawlers (PerplexityBot, GPTBot, ClaudeBot, etc.) often share a small
pool of egress IPs across many user queries. The per-IP rate limit on
/api/citable, /api/llm-context, /llms-full.txt (which re-exports
llm-context), /api/stat/[slug], /api/mcp/[transport] and adjacent
read-only endpoints was triggering 429 responses to legitimate AI
citation fetches, lowering OCB visibility in Perplexity, ChatGPT search,
Claude, and Google AI Overviews.

This change adds a User-Agent allowlist in rate-limit.ts. Requests with
a known AI crawler UA bypass the throttle without consuming a token.
Human traffic and unknown UAs remain throttled. UA spoofing is possible
but the read-only nature of these endpoints makes it a non-concern: we
WANT to be cited.

Allowlist patterns (case-insensitive substring match): GPTBot,
ChatGPT-User, OAI-SearchBot, ClaudeBot, anthropic-ai, Claude-Web,
PerplexityBot, Google-Extended, GoogleOther, CCBot, Bytespider,
Applebot-Extended, Amazonbot, Diffbot, YouBot, DuckAssistBot.

Optional AI_BYPASS_DEBUG=true env var enables a one-line console log
per bypass for debugging.

Adds tests covering: known AI UA bypasses a drained bucket, default
Chrome UA still throttled after capacity, missing UA defaults to
throttled, omitting the optional req argument keeps original behaviour.

Validation:

• pnpm typecheck: clean
• pnpm lint: clean (only preexisting warnings)
• bun test src/: 97 pass / 0 fail (6 new tests)
• pnpm build: green

[Flotapponnier]

Superseded. Will cherry pick the merged dev commit dea277d in a fresh PR with the expanded bot list.