Busbar vs LiteLLM · Python

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Requested cell: OpenAI → OpenAI. Metric: Added latency p99 (µs).

Comparability: Comparable. Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.

Ratio (LiteLLM · Python ÷ Busbar): 107. No statistical significance is asserted.

Comparison JSON · All Busbar evidence · All LiteLLM · Python evidence

Busbar

Build getbusbar/busbar:1.5.5. Measured . Run evidence

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
OpenAI → OpenAIAdded latency p50 (µs)78Valid · n=59,266 gateway / 266,330 direct—
OpenAI → OpenAIAdded latency p99 (µs)89Valid · n=59,266 gateway / 266,330 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)117Valid · n=59,266—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=266,330—
OpenAI → OpenAICPU per request (µs)68.74Valid—
OpenAI → OpenAIRequests per CPU-second14,547.2Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.957Valid—
OpenAI → OpenAICost window successful requests335,167Valid—
OpenAI → OpenAICost window throughput (req/s)55,854Valid—
OpenAI → OpenAICost window threads10Valid—
OpenAI → OpenAINon-voluntary context switches per request0Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)118Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)254Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)39,240.2Valid—
OpenAI → OpenAIStreams sustained1,452Valid—
OpenAI → OpenAIIdle RSS (MiB)7.43Valid—
OpenAI → OpenAISteady-state RSS (MiB)21.49Valid—
OpenAI → OpenAIRecovered RSS (MiB)15.7Valid—
OpenAI → OpenAIPeak RSS (MiB)22.35Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)22.27Valid—
OpenAI → OpenAITime to plateau (s)—Not measured—
OpenAI → OpenAIMemory growth (MiB/min)0.391Valid—
OpenAI → OpenAIMemory load duration (s)300Valid—
OpenAI → OpenAIMemory shape under load1Valid—
OpenAI → OpenAIIdle memory shape0Valid—
OpenAI → OpenAIIdle memory static0Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)0.004Valid—
OpenAI → OpenAIThroughput (req/s)62,231concurrency 32; n=373,4171 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,5785 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,57810 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,57850 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,578100 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,578Unbounded

LiteLLM · Python

Build ghcr.io/berriai/litellm:v1.108.3. Measured . Run evidence

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
OpenAI → OpenAIAdded latency p50 (µs)8,512Valid · n=701 gateway / 267,297 direct—
OpenAI → OpenAIAdded latency p99 (µs)9,522Valid · n=701 gateway / 267,297 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)9,549Valid · n=701—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=267,297—
OpenAI → OpenAICPU per request (µs)8,644.8Valid—
OpenAI → OpenAIRequests per CPU-second115.7Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.25Valid—
OpenAI → OpenAICost window successful requests701Valid—
OpenAI → OpenAICost window throughput (req/s)116Valid—
OpenAI → OpenAICost window threads32Valid—
OpenAI → OpenAINon-voluntary context switches per request0.029Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)11,154Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)11,974Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)9Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)1,208Valid—
OpenAI → OpenAIStreams sustained62Valid—
OpenAI → OpenAIIdle RSS (MiB)593Valid—
OpenAI → OpenAISteady-state RSS (MiB)628.6Valid—
OpenAI → OpenAIRecovered RSS (MiB)628.6Valid—
OpenAI → OpenAIPeak RSS (MiB)628.6Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)628.6Valid—
OpenAI → OpenAITime to plateau (s)61Valid—
OpenAI → OpenAIMemory growth (MiB/min)0Valid—
OpenAI → OpenAIMemory load duration (s)302Valid—
OpenAI → OpenAIMemory shape under load—Not measured—
OpenAI → OpenAIIdle memory shape—Not measured—
OpenAI → OpenAIIdle memory static1Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)0Valid—
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI → OpenAIThroughput (req/s)117concurrency 1; n=70810 ms
OpenAI → OpenAIThroughput (req/s)119concurrency 2; n=71850 ms
OpenAI → OpenAIThroughput (req/s)119concurrency 2; n=718100 ms
OpenAI → OpenAIThroughput (req/s)119concurrency 2; n=718Unbounded