Benchmark data and API
Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.
Published collection: dataset-c7c493902ee4473d65d1886e06f885af4adac2f4f5a4ea5fc70f2efa5f230c2f. Generated ; measurements through . Metadata refresh is separate from measurement time. This is a collection of independent runs, not a claim of one complete simultaneous field run.
API discovery · Latest JSON · Markdown · Compact CSV · OpenAPI · JSON schemas · Historical runs
Agent discovery (llms.txt) · Full agent guide · Version 1 measurement schema
Per-gateway exports
- agentgateway: Markdown · Compact CSV
- AISIX (api7): Markdown · Compact CSV
- APISIX: Markdown · Compact CSV
- Bifrost: Markdown · Compact CSV
- Busbar: Markdown · Compact CSV
- GoModel: Markdown · Compact CSV
- Helicone: Markdown · Compact CSV
- Higress: Markdown · Compact CSV
- Kong: Markdown · Compact CSV
- LiteLLM · Python: Markdown · Compact CSV
- LiteLLM · Rust: Markdown · Compact CSV
- One-API: Markdown · Compact CSV
- Plano: Markdown · Compact CSV
- Portkey: Markdown · Compact CSV
- TensorZero: Markdown · Compact CSV
Compact results and freshness
Start with the compact summary (one protocol pair, core metrics and provenance). Filter with gateway, ingress and egress. Full collections may exceed retrieval-tool limits; follow paginated measurement links for details. Each compact result and comparison measurement includes measured_at, published_at, generated_at, age, age_days and age_as_of. Age is measured at collection generation, not at request time. published_at is recorded for new releases when publishing begins; it remains null for local builds and older releases without that record.
current_metadata contains repository stars and their stars_as_of date. Legacy gateway.stars fields refer to that same metadata snapshot, not the benchmark date. Stars at measurement time were not recorded and are not inferred.
Context-efficient retrieval
Use /api/v1/measurements?compact=1&limit=10 to receive definitions once, keyed by definition_id. Compact summary uses the same dictionary pattern. Prefer each gateway’s Markdown and /gateways/data.csv?gateway=ID&compact=1 links; the CSV references the definition catalog. Full CSV and full JSON are bulk exports. The paginated run catalog supports exact gateway IDs and families such as family=busbar.
Retrieval
All resources are public and read-only. HTML contains measurements without JavaScript. Explicit Markdown and JSON links avoid content negotiation. Query endpoints return 400 for invalid filters and 404 for unknown entities.
GET /api/v1/summary?ingress=openai&egress=openai GET /api/v1/gateways GET /api/v1/gateways/busbar GET /api/v1/measurements?gateway=busbar&ingress=openai&egress=openai&limit=50 GET /api/v1/compare?a=busbar&b=bifrost&ingress=openai&egress=openai&metric=added_latency_p99_us
Measurements support gateway, ingress, egress, metric, limit (1–100), and offset (0 or greater). Responses are capped at 1 MiB of uncompressed UTF-8 JSON, so a page may contain fewer than limit rows. Follow the returned next URL until null; do not infer completion from row count. An individual oversized record returns 413 with narrowing guidance. Comparisons require two distinct published gateway IDs and a recorded metric. The default protocol pair is OpenAI → OpenAI and the default metric is added_latency_p99_us. Available gateway IDs and metrics are listed by the API.
Compare and cite using HTTP only
- Discover gateway IDs with the catalog and choose an explicit ingress/upstream pair. The API calls the upstream protocol
egress. - Retrieve OpenAI → Anthropic measurements; follow
nextuntil it is null. Unavailable values are not zero. - Request Busbar vs LiteLLM on that pair and inspect
comparability.status. Onlycomparablesupports a ratio;incomparableandinsufficient_evidenceinclude reasons. Matching protocol names alone is not enough. - Cite each record’s
provenance.run_url, testedgateway_build,measured_at, protocol pair and metric definition.provenance.raw_result_urlandraw_sha256identify the exact source bytes. Include the methodology and operator disclosure.
The default dashboard is not a like-for-like ranking. Its representative protocol cell can differ between products. The API returns per-cell evidence and conservative pairwise comparisons; it does not publish an unconditional fastest-gateway ranking. These are independent gateway runs against a local mock, not end-to-end model latency.
Interpretation
Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.
Every metric carries its original definition, state, units in its field name, and provenance when recorded. Missing evidence remains null. A measured zero is retained; a below-resolution reading has a null value and an explicit state. Throughput readings carry their tail-latency bound, operating concurrency, and lower-bound status. Compare conditions before ranking.
Immutable evidence
Run IDs include the SHA-256 of their exact raw JSON bytes. Raw files contain the complete samples; dashboard drawing series are thinned. Corrected raw content receives a different ID. The latest collection points to individual runs and retains their actual dates. A collection ID identifies its generated evidence representation; it is distinct from a per-gateway run ID.
Dates and licensing
published_at is the release timestamp recorded when publishing begins, not the original publication date of historical measurements; generated_at is collection build time; measured_at belongs to the instrument; stars_as_of belongs to metadata. An original first-publication timestamp is not inferred. Original measurements and dataset compilation are licensed under CC BY 4.0. Credit On the Bench — Busbar team, link the source and license, and indicate modifications. Dataset license and scope; software and third-party materials retain their own terms.