LiteLLM · Rust benchmark summary

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Selected pair: OpenAI → OpenAI. Other protocol pairs are available through paginated measurements. This summary is not a universal product ranking.

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
OpenAI → OpenAIAdded latency p50 (µs)249Valid · n=22,021 gateway / 267,021 direct—
OpenAI → OpenAIAdded latency p99 (µs)279Valid · n=22,021 gateway / 267,021 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)—Not measured—
Build
built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Measured
Published
Age
3 days when this collection was generated

Immutable run and definitions · Raw evidence (large download)

Complete evidence: all protocol pairs, configuration and history (large page) · Paginated historical runs · Data & API guidance