# litellm-python vs busbar

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency.  [Methodology and limitations](https://onthebench.ai/gateways/method) .

Requested cell: openai → openai. Metric: added_latency_p99_us.

Comparability: comparable. Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.

- Load-generator binary digests were not recorded; instrument equivalence is based on the identical clean harness source commit and mock digest.
- Measurements were made at different times on independently provisioned machines.

Ratio (B ÷ A): 0.009346775887418609. No statistical significance is asserted.

[Comparison JSON](https://onthebench.ai/api/v1/compare?a=litellm-python&b=busbar&ingress=openai&egress=openai&metric=added_latency_p99_us)  ·  [All LiteLLM · Python evidence](https://onthebench.ai/gateways/litellm-python)  ·  [All Busbar evidence](https://onthebench.ai/gateways/busbar)

## litellm-python

Build ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5. Measured 2026-11-10T17:39:41Z.  [Run evidence](https://onthebench.ai/gateways/runs/run-litellm-python-10bb57507b26d310b59f12956034785b615cd5338cdaf5cee0e587c350c53acf)

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 8512 | valid; gateway_c1_samples=701; direct_c1_samples=267297 | — |
| openai → openai | added_latency_p99_us | 9522 | valid; gateway_c1_samples=701; direct_c1_samples=267297 | — |
| openai → openai | gateway_c1_p99_us | 9549 | valid; gateway_c1_samples=701 | — |
| openai → openai | direct_c1_p99_us | 27 | valid; direct_c1_samples=267297 | — |
| openai → openai | cpu_us_per_request | 8644.793152639088 | valid | — |
| openai → openai | rps_per_cpu_second | 115.67656765676568 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.24990558455671374 | valid | — |
| openai → openai | cost_window_ok | 701 | valid | — |
| openai → openai | cost_window_rps | 116 | valid | — |
| openai → openai | cost_threads | 32 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 0.028530670470756064 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | 11154 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_ttft_p99_us | 11974 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_gap_p50_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | added_gap_p99_us | 9 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | streams_sustained_fps | 1207.9948436569616 | valid | — |
| openai → openai | streams_sustained | 62 | valid | — |
| openai → openai | idle_rss_mib | 592.95703125 | valid | — |
| openai → openai | steady_state_rss_mib | 628.60546875 | valid | — |
| openai → openai | recovered_rss_mib | 628.60546875 | valid | — |
| openai → openai | peak_rss_mib | 628.60546875 | valid | — |
| openai → openai | peak_rss_hwm_mib | 628.60546875 | valid | — |
| openai → openai | time_to_plateau_s | 61 | valid | — |
| openai → openai | growth_rate_mib_per_min | 0 | valid | — |
| openai → openai | load_s | 302 | valid | — |
| openai → openai | shape | not_measured | not_measured | — |
| openai → openai | idle_shape | not_measured | not_measured | — |
| openai → openai | idle_static | 1 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → openai | throughput_rps | 117 req/s | concurrency 1; samples 708 | 10 |
| openai → openai | throughput_rps | 119 req/s | concurrency 2; samples 718 | 50 |
| openai → openai | throughput_rps | 119 req/s | concurrency 2; samples 718 | 100 |
| openai → openai | throughput_rps | 119 req/s | concurrency 2; samples 718 | unbounded |

## busbar

Build getbusbar/busbar:1.5.5@sha256:b4cfbc7431946204aef5b1f627616e33c654a4b1e12a13758b62a9e6f71501d3. Measured 2026-11-10T17:39:31Z.  [Run evidence](https://onthebench.ai/gateways/runs/run-busbar-16d2da7bc28375e55402b153f0f49b1550d5556f6ebb71eda60b8c3c47ad8d5f)

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 78 | valid; gateway_c1_samples=59266; direct_c1_samples=266330 | — |
| openai → openai | added_latency_p99_us | 89 | valid; gateway_c1_samples=59266; direct_c1_samples=266330 | — |
| openai → openai | gateway_c1_p99_us | 117 | valid; gateway_c1_samples=59266 | — |
| openai → openai | direct_c1_p99_us | 28 | valid; direct_c1_samples=266330 | — |
| openai → openai | cpu_us_per_request | 68.74185107722418 | valid | — |
| openai → openai | rps_per_cpu_second | 14547.178819444445 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.9565490360420432 | valid | — |
| openai → openai | cost_window_ok | 335167 | valid | — |
| openai → openai | cost_window_rps | 55854 | valid | — |
| openai → openai | cost_threads | 10 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | 118 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_ttft_p99_us | 254 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_gap_p50_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | added_gap_p99_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | streams_sustained_fps | 39240.21276535891 | valid | — |
| openai → openai | streams_sustained | 1452 | valid | — |
| openai → openai | idle_rss_mib | 7.42578125 | valid | — |
| openai → openai | steady_state_rss_mib | 21.4921875 | valid | — |
| openai → openai | recovered_rss_mib | 15.703125 | valid | — |
| openai → openai | peak_rss_mib | 22.3515625 | valid | — |
| openai → openai | peak_rss_hwm_mib | 22.26953125 | valid | — |
| openai → openai | time_to_plateau_s | not_measured | not_measured | — |
| openai → openai | growth_rate_mib_per_min | 0.39135543613149704 | valid | — |
| openai → openai | load_s | 300 | valid | — |
| openai → openai | shape | 1 | valid | — |
| openai → openai | idle_shape | 0 | valid | — |
| openai → openai | idle_static | 0 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | 0.004006197677784261 | valid | — |
| openai → openai | throughput_rps | 62231 req/s | concurrency 32; samples 373417 | 1 |
| openai → openai | throughput_rps | 62587 req/s | concurrency 128; samples 375578 | 5 |
| openai → openai | throughput_rps | 62587 req/s | concurrency 128; samples 375578 | 10 |
| openai → openai | throughput_rps | 62587 req/s | concurrency 128; samples 375578 | 50 |
| openai → openai | throughput_rps | 62587 req/s | concurrency 128; samples 375578 | 100 |
| openai → openai | throughput_rps | 62587 req/s | concurrency 128; samples 375578 | unbounded |
