LiteLLM · Rust benchmark summary
Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.
Selected pair: OpenAI → OpenAI. Other protocol pairs are available through paginated measurements. This summary is not a universal product ranking.
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| OpenAI → OpenAI | Added latency p50 (µs) | 249 | Valid · n=22,021 gateway / 267,021 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 279 | Valid · n=22,021 gateway / 267,021 direct | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | — | Not measured | — |
- Build
- built from source; recorded binary
src/litellm-rust/target/release/litellm-gateway - Measured
- Published
- Age
- 3 days when this collection was generated
Immutable run and definitions · Raw evidence (large download)
Complete evidence: all protocol pairs, configuration and history (large page) · Paginated historical runs · Data & API guidance