Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 5 additions & 19 deletions benchmarks/token-quote-coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ per_chain_explainer:
- slug: bnb
h2: "Best DEX aggregator for BNB Chain new token coverage"
body: |
{{best_name:chain:bnb}} currently leads on BNB Chain at {{best_p50:chain:bnb}} coverage (24h) across Four.meme. BNB is the hardest environment in this benchmark for non-native aggregators: Four.meme graduates to PancakeSwap V2, and routing engines that do not maintain a live BNB Chain pool-creation index miss newly-minted pairs entirely. Providers that derive their BNB token list from a shared coingecko-style database rather than on-chain event tracking typically lag by several hours. The probe fires a 1 USDC (via USDC.e bridged) to token quote; BNB-native aggregators (OKX DEX, KyberSwap) have structural coverage advantages over Solana-first providers here.
{{best_name:chain:bnb}} currently leads on BNB Chain at {{best_p50:chain:bnb}} coverage (24h) across Four.meme. BNB is the hardest environment in this benchmark for non-native aggregators: Four.meme graduates to PancakeSwap V2, and routing engines that do not maintain a live BNB Chain pool-creation index miss newly-minted pairs entirely. Providers that derive their BNB token list from a shared coingecko-style database rather than on-chain event tracking typically lag by several hours. The probe fires a 1 USDC (via USDC.e bridged) to token quote; KyberSwap has structural coverage advantages over Solana-first providers here due to its live BNB pool-creation event index.

category: Aggregators
status: draft
Expand All @@ -45,7 +45,6 @@ seo_intro: |
a live Base pool graph. Four.meme graduates need PancakeSwap v2
event indexing. An aggregator that dominates one launchpad can
have gaps on another depending on how it discovers new pools.
Current leader overall is {{best_name}} at {{best_p50}}.

abstract: |
We benchmark DEX aggregators on their ability to quote tokens that
Expand All @@ -65,15 +64,15 @@ abstract: |
structural (it does not index Meteora DBC pool types) or temporary
(it lags new Raydium pairs by a few minutes). The chain dimension
narrows to a single chain for providers that are chain-specific
(Jupiter is Solana-only; 1inch and KyberSwap are EVM-only on the
(Jupiter is Solana-only; KyberSwap and Odos are EVM-only on the
launchpads they cover).

methodology:
- "Token discovery: Dexscreener /token-boosts/latest/v1 returns recently-active tokens sorted by on-chain boost activity. Up to 20 tokens per 30-minute batch are selected from each supported chain (Solana, Base, BNB). A follow-up batch call to /latest/dex/tokens/{addresses} resolves the originating dexId (pool factory), which maps to a venue label: pumpfun/moonshot/meteora-dlmm→Solana venues, virtual-fun/clanker/wow→Base venues, any BNB pair→four-meme."
- "Quote probe: 1 USDC → token, fired in parallel to all in-scope providers with a 10s timeout. Input token is always USDC (native USDC on Solana and Base, bridged USDC.e on BNB). A probe is a coverage hit when the HTTP response is 2xx and the parsed outAmount field is a non-zero positive integer or decimal. All other outcomes (4xx, 5xx, no-route JSON, empty outAmount, timeout) record as misses."
- "Aggregator endpoints: Mobula /api/1/trade/swap/quote (Bearer auth, multi-chain), Jupiter quote-api.jup.ag/v6/quote?strictMode=false (Solana), KyberSwap aggregator-api.kyberswap.com/{chain}/api/v1/routes (EVM), OKX DEX /api/v5/dex/aggregator/quote (multi-chain), Odos /sor/quote/v2 (EVM). Each adapter normalises chain identifiers, input/output decimals, and error response shapes before recording outcome."
- "Aggregator endpoints: Mobula /api/2/swap/quoting (Bearer auth, EVM), Jupiter lite-api.jup.ag/swap/v1/quote (Solana), KyberSwap aggregator-api.kyberswap.com/{chain}/api/v1/routes (EVM), Odos /sor/quote/v2 (EVM). Each adapter normalises chain identifiers, input/output decimals, and error response shapes before recording outcome."
- "Jupiter strictMode=false: queried without the curated-token filter so Jupiter will attempt routing for any token with a discoverable on-chain pool, matching how most wallets call the API. strictMode=true would trivially exclude new tokens and defeat the purpose of the benchmark."
- "Chain-scope enforcement: Jupiter probes Solana-only; KyberSwap and Odos probe EVM chains only (Base, BNB). OKX DEX and Mobula probe all three chains. Providers are never penalised for chains they do not support: Solana cells for KyberSwap and Odos are null, not zero, and are excluded from cross-chain aggregates."
- "Chain-scope enforcement: Jupiter probes Solana-only; KyberSwap, Odos, and Mobula probe EVM chains only (Base, BNB). Providers are never penalised for chains they do not support: Solana cells for KyberSwap, Odos, and Mobula are null, not zero, and are excluded from cross-chain aggregates."
- "Coverage metric: Prometheus counters token_quote_coverage_success_total and token_quote_coverage_attempts_total carry labels {provider, venue, chain}. The bench queries compute increase(success[24h]) / increase(attempts[24h]) * 100 live at read time. No pre-aggregation; the ratio reflects exactly the tokens probed in the rolling window."
- "Probe cadence and deduplication: each batch cycles through the current Dexscreener token-boosts feed. Tokens are not deduplicated across batches; the same token may appear in multiple 30-minute windows if it remains boosted. The score therefore reflects freshness of the aggregator's current routing graph, not historical coverage depth."
- "Single origin: all probes originate from the eu-west OCB VPS. Regional latency is not a factor in this bench (the outcome is binary hit/miss, not latency). No region dimension in v1."
Expand All @@ -100,7 +99,7 @@ faq:
- q: "Why is Jupiter queried with strictMode=false?"
a: "Jupiter's strictMode=true restricts quotes to verified tokens on its curated list, which defeats the purpose of a coverage bench targeting newly-launched tokens. strictMode=false instructs Jupiter to attempt routing for any token with an existing pool, matching how wallets typically call the API when displaying all tradeable tokens."
- q: "What happens when a provider is chain-specific?"
a: "Jupiter is Solana-only and appears with null values on Base and BNB launchpad rows. 1inch and KyberSwap are EVM-only and appear as null on Solana launchpad rows. OKX DEX and Mobula cover multiple chains and are probed on all launchpads they support. Null-chain cells are excluded from cross-launchpad aggregates so chain-specific providers are not penalized."
a: "Jupiter is Solana-only and appears with null values on Base and BNB launchpad rows. KyberSwap, Odos, and Mobula are EVM-only and appear as null on Solana launchpad rows. Null-chain cells are excluded from cross-launchpad aggregates so chain-specific providers are not penalized."

source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/token-quote-coverage

Expand Down Expand Up @@ -175,19 +174,6 @@ providers:
sample_size: sum(increase(token_quote_coverage_attempts_total{provider="kyberswap"}[24h]))
series: sum(rate(token_quote_coverage_success_total{provider="kyberswap"}[1h])) / sum(rate(token_quote_coverage_attempts_total{provider="kyberswap"}[1h])) * 100

- slug: okx-dex
name: OKX DEX
tag: Multi-chain, Solana + EVM
formula: "Rolling 24h ratio for provider=okx-dex. Covers Solana and EVM chains."
queries:
p50: sum(increase(token_quote_coverage_success_total{provider="okx-dex"}[24h])) / sum(increase(token_quote_coverage_attempts_total{provider="okx-dex"}[24h])) * 100
p90: sum(increase(token_quote_coverage_attempts_total{provider="okx-dex"}[24h]))
p99: sum(increase(token_quote_coverage_success_total{provider="okx-dex"}[24h])) / sum(increase(token_quote_coverage_attempts_total{provider="okx-dex"}[24h])) * 100
mean: sum(increase(token_quote_coverage_success_total{provider="okx-dex"}[24h])) / sum(increase(token_quote_coverage_attempts_total{provider="okx-dex"}[24h])) * 100
success: clamp_max(sum(increase(token_quote_coverage_success_total{provider="okx-dex"}[24h])) / sum(increase(token_quote_coverage_attempts_total{provider="okx-dex"}[24h])), 1)
sample_size: sum(increase(token_quote_coverage_attempts_total{provider="okx-dex"}[24h]))
series: sum(rate(token_quote_coverage_success_total{provider="okx-dex"}[1h])) / sum(rate(token_quote_coverage_attempts_total{provider="okx-dex"}[1h])) * 100

- slug: odos
name: Odos
tag: EVM multi-chain, Base + BNB + Ethereum
Expand Down
1 change: 0 additions & 1 deletion harnesses/token-quote-coverage/.env.example
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
MONITOR_REGION=eu-west
MOBULA_API_KEY=your_key
OKX_DEX_API_KEY=optional
LOGS_TOKEN=optional_for_logs_endpoint
6 changes: 2 additions & 4 deletions harnesses/token-quote-coverage/cmd/monitor/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ import (
type Config struct {
MonitorRegion string
LogsToken string
MobulaAPIKey string
OKXDEXAPIKey string // optional
MobulaAPIKey string
}

func loadEnv() (*Config, error) {
Expand All @@ -21,8 +20,7 @@ func loadEnv() (*Config, error) {
c := &Config{
MonitorRegion: strings.TrimSpace(os.Getenv("MONITOR_REGION")),
LogsToken: strings.TrimSpace(os.Getenv("LOGS_TOKEN")),
MobulaAPIKey: strings.TrimSpace(os.Getenv("MOBULA_API_KEY")),
OKXDEXAPIKey: strings.TrimSpace(os.Getenv("OKX_DEX_API_KEY")),
MobulaAPIKey: strings.TrimSpace(os.Getenv("MOBULA_API_KEY")),
}
if c.MonitorRegion == "" {
c.MonitorRegion = "eu-west"
Expand Down
1 change: 0 additions & 1 deletion harnesses/token-quote-coverage/cmd/monitor/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ func main() {

fmt.Printf("Region: %s\n", cfg.MonitorRegion)
fmt.Printf("Mobula key set: %v\n", cfg.MobulaAPIKey != "")
fmt.Printf("OKX key set: %v\n", cfg.OKXDEXAPIKey != "")
fmt.Println("Metrics on :2112/metrics, logs on :2112/logs (LOGS_TOKEN gated)")
fmt.Println()

Expand Down
101 changes: 0 additions & 101 deletions harnesses/token-quote-coverage/cmd/monitor/okxdex.go

This file was deleted.

99 changes: 0 additions & 99 deletions harnesses/token-quote-coverage/cmd/monitor/oneinch.go

This file was deleted.

1 change: 0 additions & 1 deletion harnesses/token-quote-coverage/cmd/monitor/scheduler.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ func buildProviders(cfg *Config) []Provider {
NewJupiterProvider(),
NewMobulaProvider(cfg.MobulaAPIKey),
NewKyberSwapProvider(),
NewOKXDEXProvider(cfg.OKXDEXAPIKey),
NewOdosProvider(),
}
return providers
Expand Down
16 changes: 8 additions & 8 deletions src/lib/bench-template.ts
Original file line number Diff line number Diff line change
Expand Up @@ -111,19 +111,19 @@ export function renderTemplate(text: string, benchmark: Benchmark): string {
const k = keyword.toLowerCase();
if (k === "best_name") {
const lead = bestForChain(benchmark, chain);
return lead ? lead.name : whole;
return lead ? lead.name : "";
}
if (k === "best_p50") {
const lead = bestForChain(benchmark, chain);
return lead ? fmtUnit(lead.ms.p50, benchmark.unit) : whole;
return lead ? fmtUnit(lead.ms.p50, benchmark.unit) : "";
}
if (k === "worst_name") {
const trailer = worstForChain(benchmark, chain);
return trailer ? trailer.name : whole;
return trailer ? trailer.name : "";
}
if (k === "worst_p50") {
const trailer = worstForChain(benchmark, chain);
return trailer ? fmtUnit(trailer.ms.p50, benchmark.unit) : whole;
return trailer ? fmtUnit(trailer.ms.p50, benchmark.unit) : "";
}
return whole;
},
Expand All @@ -146,13 +146,13 @@ export function renderTemplate(text: string, benchmark: Benchmark): string {
return fmtUnit(raw, benchmark.unit);
}
case "best_name":
return best ? best.name : whole;
return best ? best.name : "";
case "best_p50":
return best ? fmtUnit(best.ms.p50, benchmark.unit) : whole;
return best ? fmtUnit(best.ms.p50, benchmark.unit) : "";
case "worst_name":
return worst ? worst.name : whole;
return worst ? worst.name : "";
case "worst_p50":
return worst ? fmtUnit(worst.ms.p50, benchmark.unit) : whole;
return worst ? fmtUnit(worst.ms.p50, benchmark.unit) : "";
case "count":
return String(live.length);
default:
Expand Down
Loading