Skip to content

fix: hide 'currently unavailable' rows when a dimension filter is active #112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Flotapponnier merged 1 commit into from
May 23, 2026
Merged

fix: hide 'currently unavailable' rows when a dimension filter is active #112

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions docs/methodology/solana-tx-landing-active.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,15 @@ We do **not** vary tip amount across cycles. A "tip elasticity" experiment is a

1. Fetch a recent blockhash via the public mainnet RPC (`api.mainnet-beta.solana.com`) with `commitment = "processed"`. The same blockhash is used for all services in the same cycle so they share a chain-state reference point. `processed` is preferred over `confirmed` because the resulting blockhash is fresher (~400 ms vs ~6 s); the marginal fork risk is acceptable since landing services dedup on signature, not blockhash.
2. Build the transaction described in §3 for that service (the tip-transfer differs per service).
3. Sign with the region's persistent keypair.
4. Capture `submit_slot = getSlot(commitment="processed")` and `submit_wallclock = time.Now()`.
5. POST the base64-encoded signed transaction to the service's documented submission endpoint (exact URLs published in the harness source) with `skipPreflight = true`, `maxRetries = 0`, `encoding = "base64"`. Per-service auth headers / query params are applied as documented.
6. Capture the returned signature (or fail-fast on RPC error).
7. Poll `getSignatureStatuses` on the public mainnet RPC every 200 ms. The poll cadence is the measurement floor for `latency_ms`. We chose 200 ms because Solana's slot duration is ~400 ms and "confirmed" lands ~1-2 slots after inclusion (~400 ms-1 s realistic), so a 1 s poll quantizes everything into 1 s steps and erases differentiation between services. 200 ms gives 5x the resolution while staying within public RPC budget.
8. On status reaching `confirmationStatus = "confirmed"` or `"finalized"`, record `land_slot` from the response context and `land_wallclock = time.Now()`. Classify as **landed**.
9. If 60 seconds elapse without a non-null `confirmationStatus`, abandon the poll. Classify as **dropped** with reason `timeout`.
10. If the original submission returned a transport error (HTTP timeout, DNS, EOF, connection refused), classify as **dropped** with reason `network_error`. If the upstream returned HTTP 419 / 429 or a JSON-RPC error containing "rate limit" / "too many requests", classify as `rate_limited`. If the submission was rejected with a structured RPC error (`InstructionError`, `BlockhashNotFound`, etc.) or the on-chain status comes back with an `Err`, classify as `invalid`.
3. Sign with the region's persistent keypair. The signature is known at this point, before any network call.
4. **Subscribe to the signature via `signatureSubscribe` on the public mainnet WebSocket** (`wss://api.mainnet-beta.solana.com`) at `commitment = "confirmed"`. The subscription is registered **before submission** so a fast-confirming tx cannot complete before we are listening (otherwise we would miss the notification and incorrectly timeout).
5. Capture `submit_slot = getSlot(commitment="processed")` and `submit_wallclock = time.Now()`.
6. POST the base64-encoded signed transaction to the service's documented submission endpoint (exact URLs published in the harness source) with `skipPreflight = true`, `maxRetries = 0`, `encoding = "base64"`. Per-service auth headers / query params are applied as documented.
7. Capture the returned signature (or fail-fast on RPC error).
8. Block on the `signatureNotification` push from the WebSocket. On notification, record `land_slot` from the notification context and `land_wallclock = time.Now()`. Classify as **landed**. Resolution is RTT-bounded (~30-50 ms us-east → mainnet-beta) since the RPC pushes the notification the instant the commitment level is reached, with no client polling cadence floor.
9. **Fallback:** if the WebSocket connection fails to establish at the start of the cycle (transient network issue, RPC overload), every probe in that cycle falls back to HTTP polling of `getSignatureStatuses` every 200 ms. This preserves bench continuity but adds a ~200 ms quantization penalty for the affected cycle. The fallback path is logged.
10. If 60 seconds elapse without a notification (or without a non-null `confirmationStatus` on the polling fallback), abandon the wait. Classify as **dropped** with reason `timeout`.
11. If the original submission returned a transport error (HTTP timeout, DNS, EOF, connection refused), classify as **dropped** with reason `network_error`. If the upstream returned HTTP 419 / 429 or a JSON-RPC error containing "rate limit" / "too many requests", classify as `rate_limited`. If the submission was rejected with a structured RPC error (`InstructionError`, `BlockhashNotFound`, etc.) or the on-chain status comes back with an `Err`, classify as `invalid`.

The 5 services for a given cycle are submitted **in parallel** (Go goroutines) so they sample the same congestion window. The order of `submit_slot` reads is arbitrary but all reads happen within 200 ms.

Expand Down
60 changes: 35 additions & 25 deletions src/lib/spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,35 +180,45 @@ async function specToBenchmark(
};

const activeLabels = activeFilterLabels(options);
const filteredSpec =
Object.keys(activeLabels).length > 0 ? applyDimensionsToSpec(spec, activeLabels) : spec;
const isFiltered = Object.keys(activeLabels).length > 0;
const filteredSpec = isFiltered ? applyDimensionsToSpec(spec, activeLabels) : spec;

const live = await tryLoadLive(filteredSpec);
if (live) {
// Augment live results with any spec-declared providers that didn't
// return data this cycle. Without this, providers with transiently
// missing Prom data fall out of `getProviders()` entirely → their
// /products/<slug> page 404s and they disappear from the sitemap.
//
// These entries are tagged `availability: "unavailable"` so the
// leaderboard renders a soft offline pill ("Currently unavailable")
// instead of a row of 0 ms / 0% that misleads readers into thinking
// the provider is genuinely the fastest. Mark live entries explicitly
// too so a missing `availability` field always reads as "unknown".
const liveSlugs = new Set(live.results.map((r) => r.slug.toLowerCase()));
// Mark live entries explicitly so a missing `availability` reads as
// "unknown" everywhere else in the code.
for (const r of live.results) r.availability = "live";
for (const p of spec.providers) {
if (liveSlugs.has(p.slug.toLowerCase())) continue;
live.results.push({
name: p.name,
slug: p.slug,
tag: p.tag,
type: p.type,
ms: { p50: 0, p90: 0, p99: 0, mean: 0 },
successRate: 0,
secondary: p.secondary,
availability: "unavailable",
});

// Augment with spec-declared providers that didn't return data this
// cycle, but only on the *unfiltered* view. When the reader has
// applied a dimension filter (e.g. chain=bnb on rpc-capabilities)
// a no-data result almost always means the provider doesn't cover
// that dimension at all (rpc-capabilities ships ~15 providers but
// only 5 of them serve BNB; the other 10 are by-design absent on
// that tab). Surfacing those as "Currently unavailable" rows would
// pollute the leaderboard with 10 fake-offline entries and confuse
// the reader about which providers are actually broken vs which
// simply don't compete on this chain.
//
// On the unfiltered "All" tab we still augment because then a no-data
// result really does mean "harness lost this provider"; product
// pages also rely on the augmentation to stay reachable when the
// upstream is briefly down.
if (!isFiltered) {
const liveSlugs = new Set(live.results.map((r) => r.slug.toLowerCase()));
for (const p of spec.providers) {
if (liveSlugs.has(p.slug.toLowerCase())) continue;
live.results.push({
name: p.name,
slug: p.slug,
tag: p.tag,
type: p.type,
ms: { p50: 0, p90: 0, p99: 0, mean: 0 },
successRate: 0,
secondary: p.secondary,
availability: "unavailable",
});
}
}
// Resolve {{p50:slug}} / {{best_name}} / {{count}} etc. placeholders
// against the freshly loaded numbers so editorial text (findings,
Expand Down
Loading