diff --git a/docs/methodology/solana-tx-landing-active.md b/docs/methodology/solana-tx-landing-active.md index d367b35c..e014a28b 100644 --- a/docs/methodology/solana-tx-landing-active.md +++ b/docs/methodology/solana-tx-landing-active.md @@ -52,14 +52,15 @@ We do **not** vary tip amount across cycles. A "tip elasticity" experiment is a 1. Fetch a recent blockhash via the public mainnet RPC (`api.mainnet-beta.solana.com`) with `commitment = "processed"`. The same blockhash is used for all services in the same cycle so they share a chain-state reference point. `processed` is preferred over `confirmed` because the resulting blockhash is fresher (~400 ms vs ~6 s); the marginal fork risk is acceptable since landing services dedup on signature, not blockhash. 2. Build the transaction described in §3 for that service (the tip-transfer differs per service). -3. Sign with the region's persistent keypair. -4. Capture `submit_slot = getSlot(commitment="processed")` and `submit_wallclock = time.Now()`. -5. POST the base64-encoded signed transaction to the service's documented submission endpoint (exact URLs published in the harness source) with `skipPreflight = true`, `maxRetries = 0`, `encoding = "base64"`. Per-service auth headers / query params are applied as documented. -6. Capture the returned signature (or fail-fast on RPC error). -7. Poll `getSignatureStatuses` on the public mainnet RPC every 200 ms. The poll cadence is the measurement floor for `latency_ms`. We chose 200 ms because Solana's slot duration is ~400 ms and "confirmed" lands ~1-2 slots after inclusion (~400 ms-1 s realistic), so a 1 s poll quantizes everything into 1 s steps and erases differentiation between services. 200 ms gives 5x the resolution while staying within public RPC budget. -8. On status reaching `confirmationStatus = "confirmed"` or `"finalized"`, record `land_slot` from the response context and `land_wallclock = time.Now()`. Classify as **landed**. -9. If 60 seconds elapse without a non-null `confirmationStatus`, abandon the poll. Classify as **dropped** with reason `timeout`. -10. If the original submission returned a transport error (HTTP timeout, DNS, EOF, connection refused), classify as **dropped** with reason `network_error`. If the upstream returned HTTP 419 / 429 or a JSON-RPC error containing "rate limit" / "too many requests", classify as `rate_limited`. If the submission was rejected with a structured RPC error (`InstructionError`, `BlockhashNotFound`, etc.) or the on-chain status comes back with an `Err`, classify as `invalid`. +3. Sign with the region's persistent keypair. The signature is known at this point, before any network call. +4. **Subscribe to the signature via `signatureSubscribe` on the public mainnet WebSocket** (`wss://api.mainnet-beta.solana.com`) at `commitment = "confirmed"`. The subscription is registered **before submission** so a fast-confirming tx cannot complete before we are listening (otherwise we would miss the notification and incorrectly timeout). +5. Capture `submit_slot = getSlot(commitment="processed")` and `submit_wallclock = time.Now()`. +6. POST the base64-encoded signed transaction to the service's documented submission endpoint (exact URLs published in the harness source) with `skipPreflight = true`, `maxRetries = 0`, `encoding = "base64"`. Per-service auth headers / query params are applied as documented. +7. Capture the returned signature (or fail-fast on RPC error). +8. Block on the `signatureNotification` push from the WebSocket. On notification, record `land_slot` from the notification context and `land_wallclock = time.Now()`. Classify as **landed**. Resolution is RTT-bounded (~30-50 ms us-east → mainnet-beta) since the RPC pushes the notification the instant the commitment level is reached, with no client polling cadence floor. +9. **Fallback:** if the WebSocket connection fails to establish at the start of the cycle (transient network issue, RPC overload), every probe in that cycle falls back to HTTP polling of `getSignatureStatuses` every 200 ms. This preserves bench continuity but adds a ~200 ms quantization penalty for the affected cycle. The fallback path is logged. +10. If 60 seconds elapse without a notification (or without a non-null `confirmationStatus` on the polling fallback), abandon the wait. Classify as **dropped** with reason `timeout`. +11. If the original submission returned a transport error (HTTP timeout, DNS, EOF, connection refused), classify as **dropped** with reason `network_error`. If the upstream returned HTTP 419 / 429 or a JSON-RPC error containing "rate limit" / "too many requests", classify as `rate_limited`. If the submission was rejected with a structured RPC error (`InstructionError`, `BlockhashNotFound`, etc.) or the on-chain status comes back with an `Err`, classify as `invalid`. The 5 services for a given cycle are submitted **in parallel** (Go goroutines) so they sample the same congestion window. The order of `submit_slot` reads is arbitrary but all reads happen within 200 ms. diff --git a/src/lib/spec.ts b/src/lib/spec.ts index a8859101..2b8a07e6 100644 --- a/src/lib/spec.ts +++ b/src/lib/spec.ts @@ -180,35 +180,45 @@ async function specToBenchmark( }; const activeLabels = activeFilterLabels(options); - const filteredSpec = - Object.keys(activeLabels).length > 0 ? applyDimensionsToSpec(spec, activeLabels) : spec; + const isFiltered = Object.keys(activeLabels).length > 0; + const filteredSpec = isFiltered ? applyDimensionsToSpec(spec, activeLabels) : spec; const live = await tryLoadLive(filteredSpec); if (live) { - // Augment live results with any spec-declared providers that didn't - // return data this cycle. Without this, providers with transiently - // missing Prom data fall out of `getProviders()` entirely → their - // /products/ page 404s and they disappear from the sitemap. - // - // These entries are tagged `availability: "unavailable"` so the - // leaderboard renders a soft offline pill ("Currently unavailable") - // instead of a row of 0 ms / 0% that misleads readers into thinking - // the provider is genuinely the fastest. Mark live entries explicitly - // too so a missing `availability` field always reads as "unknown". - const liveSlugs = new Set(live.results.map((r) => r.slug.toLowerCase())); + // Mark live entries explicitly so a missing `availability` reads as + // "unknown" everywhere else in the code. for (const r of live.results) r.availability = "live"; - for (const p of spec.providers) { - if (liveSlugs.has(p.slug.toLowerCase())) continue; - live.results.push({ - name: p.name, - slug: p.slug, - tag: p.tag, - type: p.type, - ms: { p50: 0, p90: 0, p99: 0, mean: 0 }, - successRate: 0, - secondary: p.secondary, - availability: "unavailable", - }); + + // Augment with spec-declared providers that didn't return data this + // cycle, but only on the *unfiltered* view. When the reader has + // applied a dimension filter (e.g. chain=bnb on rpc-capabilities) + // a no-data result almost always means the provider doesn't cover + // that dimension at all (rpc-capabilities ships ~15 providers but + // only 5 of them serve BNB; the other 10 are by-design absent on + // that tab). Surfacing those as "Currently unavailable" rows would + // pollute the leaderboard with 10 fake-offline entries and confuse + // the reader about which providers are actually broken vs which + // simply don't compete on this chain. + // + // On the unfiltered "All" tab we still augment because then a no-data + // result really does mean "harness lost this provider"; product + // pages also rely on the augmentation to stay reachable when the + // upstream is briefly down. + if (!isFiltered) { + const liveSlugs = new Set(live.results.map((r) => r.slug.toLowerCase())); + for (const p of spec.providers) { + if (liveSlugs.has(p.slug.toLowerCase())) continue; + live.results.push({ + name: p.name, + slug: p.slug, + tag: p.tag, + type: p.type, + ms: { p50: 0, p90: 0, p99: 0, mean: 0 }, + successRate: 0, + secondary: p.secondary, + availability: "unavailable", + }); + } } // Resolve {{p50:slug}} / {{best_name}} / {{count}} etc. placeholders // against the freshly loaded numbers so editorial text (findings,