# one-api vs litellm-rust

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency.  [Methodology and limitations](https://onthebench.ai/gateways/method) .

Requested cell: openai → openai. Metric: added_latency_p99_us.

Comparability: comparable. Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.

- Load-generator binary digests were not recorded; instrument equivalence is based on the identical clean harness source commit and mock digest.
- Measurements were made at different times on independently provisioned machines.

Ratio (B ÷ A): 0.0004731949478383132. No statistical significance is asserted.

[Comparison JSON](https://onthebench.ai/api/v1/compare?a=one-api&b=litellm-rust&ingress=openai&egress=openai&metric=added_latency_p99_us)  ·  [All One-API evidence](https://onthebench.ai/gateways/one-api)  ·  [All LiteLLM · Rust evidence](https://onthebench.ai/gateways/litellm-rust)

## one-api

Build justsong/one-api:v0.6.10@sha256:af6b361630bdf408b86fce7aa8fc131b37ea5177ae866d121ba7c002e773e477. Measured 2026-11-10T17:39:58Z.  [Run evidence](https://onthebench.ai/gateways/runs/run-one-api-c9427cc70e392f48beb8e68d908f8924a7330748ac291dab1cd902b3f14c542f)

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 34017 | valid; gateway_c1_samples=102; direct_c1_samples=263027 | — |
| openai → openai | added_latency_p99_us | 589609 | valid; gateway_c1_samples=102; direct_c1_samples=263027 | — |
| openai → openai | gateway_c1_p99_us | 589637 | valid; gateway_c1_samples=102 | — |
| openai → openai | direct_c1_p99_us | 28 | valid; direct_c1_samples=263027 | — |
| openai → openai | cpu_us_per_request | not_measured | not_measured | — |
| openai → openai | rps_per_cpu_second | not_measured | not_measured | — |
| openai → openai | cost_window_conc | not_measured | not_measured | — |
| openai → openai | cost_core_utilisation | not_measured | not_measured | — |
| openai → openai | cost_window_ok | not_measured | not_measured | — |
| openai → openai | cost_window_rps | not_measured | not_measured | — |
| openai → openai | cost_threads | not_measured | not_measured | — |
| openai → openai | cost_nonvol_ctxt_per_request | not_measured | not_measured | — |
| openai → openai | cost_majflt | not_measured | not_measured | — |
| openai → openai | added_ttft_p50_us | 728 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_ttft_p99_us | 770 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_gap_p50_us | 12 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | added_gap_p99_us | 27 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | streams_sustained_fps | 7497.836670339175 | valid | — |
| openai → openai | streams_sustained | 213 | valid | — |
| openai → openai | idle_rss_mib | 87.15625 | valid | — |
| openai → openai | steady_state_rss_mib | 141.876953125 | valid | — |
| openai → openai | recovered_rss_mib | 136.578125 | valid | — |
| openai → openai | peak_rss_mib | 142.4921875 | valid | — |
| openai → openai | peak_rss_hwm_mib | 142.47265625 | valid | — |
| openai → openai | time_to_plateau_s | 134 | valid | — |
| openai → openai | growth_rate_mib_per_min | 0.1159464817427444 | valid | — |
| openai → openai | load_s | 300 | valid | — |
| openai → openai | shape | not_measured | not_measured | — |
| openai → openai | idle_shape | not_measured | not_measured | — |
| openai → openai | idle_static | 1 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | -0.1148904573303369 | valid | — |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → openai | throughput_rps | 32 req/s | concurrency 1; samples 196 | 50 |
| openai → openai | throughput_rps | 32 req/s | concurrency 1; samples 196 | 100 |
| openai → openai | throughput_rps | 38 req/s | concurrency 16; samples 229 | unbounded |

## litellm-rust

Build src/litellm-rust/target/release/litellm-gateway. Measured 2026-11-10T17:39:51Z.  [Run evidence](https://onthebench.ai/gateways/runs/run-litellm-rust-69d8e6ed325740d9da7f0aaf31b815418c4801eb6549b2e2c68f768a92af3ad4)

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 249 | valid; gateway_c1_samples=22021; direct_c1_samples=267021 | — |
| openai → openai | added_latency_p99_us | 279 | valid; gateway_c1_samples=22021; direct_c1_samples=267021 | — |
| openai → openai | gateway_c1_p99_us | 306 | valid; gateway_c1_samples=22021 | — |
| openai → openai | direct_c1_p99_us | 27 | valid; direct_c1_samples=267021 | — |
| openai → openai | cpu_us_per_request | 272.6013913681787 | valid | — |
| openai → openai | rps_per_cpu_second | 3668.359853121175 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.6790579018170796 | valid | — |
| openai → openai | cost_window_ok | 59941 | valid | — |
| openai → openai | cost_window_rps | 9989 | valid | — |
| openai → openai | cost_threads | 5 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | not_measured | not_measured | — |
| openai → openai | added_ttft_p99_us | not_measured | not_measured | — |
| openai → openai | added_gap_p50_us | not_measured | not_measured | — |
| openai → openai | added_gap_p99_us | not_measured | not_measured | — |
| openai → openai | streams_sustained_fps | 0 (measured_failure) | valid | — |
| openai → openai | streams_sustained | 0 (measured_failure) | valid | — |
| openai → openai | idle_rss_mib | 12.73828125 | valid | — |
| openai → openai | steady_state_rss_mib | 18.9921875 | valid | — |
| openai → openai | recovered_rss_mib | 18.99609375 | valid | — |
| openai → openai | peak_rss_mib | 19.875 | valid | — |
| openai → openai | peak_rss_hwm_mib | 19.484375 | valid | — |
| openai → openai | time_to_plateau_s | not_measured | not_measured | — |
| openai → openai | growth_rate_mib_per_min | -0.1791634760562065 | valid | — |
| openai → openai | load_s | 300 | valid | — |
| openai → openai | shape | 0 | valid | — |
| openai → openai | idle_shape | not_measured | not_measured | — |
| openai → openai | idle_static | 1 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → openai | throughput_rps | 8404 req/s | concurrency 4; samples 50435 | 1 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | 5 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | 10 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | 50 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | 100 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | unbounded |
