LiteLLM · Rust vs Busbar
Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.
Requested cell: OpenAI → OpenAI. Metric: Added latency p99 (µs).
Comparability: Comparable. Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.
- Load-generator binary digests were not recorded; instrument equivalence is based on the identical clean harness source commit and mock digest.
- Measurements were made at different times on independently provisioned machines.
Ratio (Busbar ÷ LiteLLM · Rust): 0.319. No statistical significance is asserted.
Comparison JSON · All LiteLLM · Rust evidence · All Busbar evidence
LiteLLM · Rust
Build src/litellm-rust/target/release/litellm-gateway. Measured . Run evidence
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| OpenAI → OpenAI | Added latency p50 (µs) | 249 | Valid · n=22,021 gateway / 267,021 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 279 | Valid · n=22,021 gateway / 267,021 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 306 | Valid · n=22,021 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 27 | Valid · n=267,021 | — |
| OpenAI → OpenAI | CPU per request (µs) | 272.6 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 3,668.4 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.679 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 59,941 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 9,989 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 5 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | — | Not measured | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | — | Not measured | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | — | Not measured | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | — | Not measured | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 0 | Valid — measured failure: the gateway was offered stream load and sustained none of it | — |
| OpenAI → OpenAI | Streams sustained | 0 | Valid — measured failure: the gateway was offered stream load and sustained none of it | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 12.74 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 18.99 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 19 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 19.88 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 19.48 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | — | Not measured | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | -0.179 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 300 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | 0 | Valid | — |
| OpenAI → OpenAI | Idle memory shape | — | Not measured | — |
| OpenAI → OpenAI | Idle memory static | 1 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | 0 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | 8,404 | concurrency 4; n=50,435 | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | Unbounded |
Busbar
Build getbusbar/busbar:1.5.5. Measured . Run evidence
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| OpenAI → OpenAI | Added latency p50 (µs) | 78 | Valid · n=59,266 gateway / 266,330 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 89 | Valid · n=59,266 gateway / 266,330 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 117 | Valid · n=59,266 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=266,330 | — |
| OpenAI → OpenAI | CPU per request (µs) | 68.74 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 14,547.2 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.957 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 335,167 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 55,854 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 10 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | 118 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | 254 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 39,240.2 | Valid | — |
| OpenAI → OpenAI | Streams sustained | 1,452 | Valid | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 7.43 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 21.49 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 15.7 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 22.35 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 22.27 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | — | Not measured | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | 0.391 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 300 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | 1 | Valid | — |
| OpenAI → OpenAI | Idle memory shape | 0 | Valid | — |
| OpenAI → OpenAI | Idle memory static | 0 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | 0.004 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | 62,231 | concurrency 32; n=373,417 | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | 62,587 | concurrency 128; n=375,578 | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | 62,587 | concurrency 128; n=375,578 | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | 62,587 | concurrency 128; n=375,578 | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | 62,587 | concurrency 128; n=375,578 | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 62,587 | concurrency 128; n=375,578 | Unbounded |