# plano vs portkey

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency.  [Methodology and limitations](https://onthebench.ai/gateways/method) .

Requested cell: openai → openai. Metric: added_latency_p99_us.

Comparability: comparable. Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.

- Load-generator binary digests were not recorded; instrument equivalence is based on the identical clean harness source commit and mock digest.
- Measurements were made at different times on independently provisioned machines.

Ratio (B ÷ A): 0.018944874396745096. No statistical significance is asserted.

[Comparison JSON](https://onthebench.ai/api/v1/compare?a=plano&b=portkey&ingress=openai&egress=openai&metric=added_latency_p99_us)  ·  [All Plano evidence](https://onthebench.ai/gateways/plano)  ·  [All Portkey evidence](https://onthebench.ai/gateways/portkey)

## plano

Build katanemo/plano:0.4.39@sha256:35a84cb873ee5f16cded33eeda72736f90798962b7489d58650c0db7b546e9ac. Measured 2026-11-10T17:40:07Z.  [Run evidence](https://onthebench.ai/gateways/runs/run-plano-0a4ce97da36f11e7c407eef9280f69688b7ea1f007616709f0b914993cb8ba8e)

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 198988 | valid; gateway_c1_samples=30; direct_c1_samples=266566 | — |
| openai → openai | added_latency_p99_us | 205966 | valid; gateway_c1_samples=30; direct_c1_samples=266566 | — |
| openai → openai | gateway_c1_p99_us | 205993 | valid; gateway_c1_samples=30 | — |
| openai → openai | direct_c1_p99_us | 27 | valid; direct_c1_samples=266566 | — |
| openai → openai | cpu_us_per_request | 182230.76923076922 | valid | — |
| openai → openai | rps_per_cpu_second | 5.487547488391726 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.9645138811621244 | valid | — |
| openai → openai | cost_window_ok | 130 | valid | — |
| openai → openai | cost_window_rps | 21 | valid | — |
| openai → openai | cost_threads | 68 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 6.276923076923077 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | 158773 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_ttft_p99_us | 170494 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_gap_p50_us | 1 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | added_gap_p99_us | 38 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | streams_sustained_fps | 669.7194949393278 | valid | — |
| openai → openai | streams_sustained | 29 | valid | — |
| openai → openai | idle_rss_mib | 617.203125 | valid | — |
| openai → openai | steady_state_rss_mib | 971.84765625 | valid | — |
| openai → openai | recovered_rss_mib | 972.3203125 | valid | — |
| openai → openai | peak_rss_mib | 972.4453125 | valid | — |
| openai → openai | peak_rss_hwm_mib | 972.31640625 | valid | — |
| openai → openai | time_to_plateau_s | 61 | valid | — |
| openai → openai | growth_rate_mib_per_min | 0.12924042130805258 | valid | — |
| openai → openai | load_s | 305 | valid | — |
| openai → openai | shape | not_measured | not_measured | — |
| openai → openai | idle_shape | not_measured | not_measured | — |
| openai → openai | idle_static | 1 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | 0.052286481252150616 | valid | — |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound | 50 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound | 100 |
| openai → openai | throughput_rps | 19 req/s | concurrency 8; samples 118 | unbounded |

## portkey

Build portkeyai/gateway:1.15.2@sha256:9065e15701d5f90ae1728c57369f8cec137d8c7dffcd027a402a67e64f27b9d3. Measured 2026-11-10T17:39:55Z.  [Run evidence](https://onthebench.ai/gateways/runs/run-portkey-670dce412ba19689b30f84f64cb8b9c31fc07125970bd210617247c56c23bfcc)

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 1196 | valid; gateway_c1_samples=4304; direct_c1_samples=246357 | — |
| openai → openai | added_latency_p99_us | 3902 | valid; gateway_c1_samples=4304; direct_c1_samples=246357 | — |
| openai → openai | gateway_c1_p99_us | 3938 | valid; gateway_c1_samples=4304 | — |
| openai → openai | direct_c1_p99_us | 36 | valid; direct_c1_samples=246357 | — |
| openai → openai | cpu_us_per_request | 1528.3748925193465 | valid | — |
| openai → openai | rps_per_cpu_second | 654.2897327707454 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.2950181957762656 | valid | — |
| openai → openai | cost_window_ok | 4652 | valid | — |
| openai → openai | cost_window_rps | 775 | valid | — |
| openai → openai | cost_threads | 22 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 0.007308684436801376 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | 28114 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_ttft_p99_us | 29752 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_gap_p50_us | 0 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | added_gap_p99_us | 440 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | streams_sustained_fps | 4996.208831040471 | valid | — |
| openai → openai | streams_sustained | 1519 | valid | — |
| openai → openai | idle_rss_mib | 125.0703125 | valid | — |
| openai → openai | steady_state_rss_mib | 245.171875 | valid | — |
| openai → openai | recovered_rss_mib | 142.08203125 | valid | — |
| openai → openai | peak_rss_mib | 250.01953125 | valid | — |
| openai → openai | peak_rss_hwm_mib | 263.984375 | valid | — |
| openai → openai | time_to_plateau_s | not_measured | not_measured | — |
| openai → openai | growth_rate_mib_per_min | 0.20228106254272332 | valid | — |
| openai → openai | load_s | 301 | valid | — |
| openai → openai | shape | 0 | valid | — |
| openai → openai | idle_shape | -1 | valid | — |
| openai → openai | idle_static | 0 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | -13.958014762556811 | valid | — |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → openai | throughput_rps | 831 req/s | concurrency 1; samples 4992 | 5 |
| openai → openai | throughput_rps | 835 req/s | concurrency 2; samples 5012 | 10 |
| openai → openai | throughput_rps | 836 req/s | concurrency 8; samples 5018 | 50 |
| openai → openai | throughput_rps | 836 req/s | concurrency 8; samples 5018 | 100 |
| openai → openai | throughput_rps | 836 req/s | concurrency 8; samples 5018 | unbounded |
