# agentgateway vs tensorzero

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency.  [Methodology and limitations](https://onthebench.ai/gateways/method) .

Requested cell: openai → openai. Metric: added_latency_p99_us.

Comparability: comparable. Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.

- Load-generator binary digests were not recorded; instrument equivalence is based on the identical clean harness source commit and mock digest.
- Measurements were made at different times on independently provisioned machines.

Ratio (B ÷ A): 167.19521912350598. No statistical significance is asserted.

[Comparison JSON](https://onthebench.ai/api/v1/compare?a=agentgateway&b=tensorzero&ingress=openai&egress=openai&metric=added_latency_p99_us)  ·  [All agentgateway evidence](https://onthebench.ai/gateways/agentgateway)  ·  [All TensorZero evidence](https://onthebench.ai/gateways/tensorzero)

## agentgateway

Build ghcr.io/agentgateway/agentgateway:v1.7.0@sha256:4562330cd6e580bfac2de5b8905fee608dcf3fff6f1f4900744d27bdddb3b41b. Measured 2026-11-10T17:39:19Z.  [Run evidence](https://onthebench.ai/gateways/runs/run-agentgateway-fa351efdcee949e977dd1d4331440512b3e913a74b878f4bd1b19a7c7ca2900c)

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 216 | valid; gateway_c1_samples=24618; direct_c1_samples=233879 | — |
| openai → openai | added_latency_p99_us | 251 | valid; gateway_c1_samples=24618; direct_c1_samples=233879 | — |
| openai → openai | gateway_c1_p99_us | 282 | valid; gateway_c1_samples=24618 | — |
| openai → openai | direct_c1_p99_us | 31 | valid; direct_c1_samples=233879 | — |
| openai → openai | cpu_us_per_request | 188.57584939720198 | valid | — |
| openai → openai | rps_per_cpu_second | 5302.905982905983 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.9718871854587116 | valid | — |
| openai → openai | cost_window_ok | 124088 | valid | — |
| openai → openai | cost_window_rps | 20679 | valid | — |
| openai → openai | cost_threads | 13 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | 327 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_ttft_p99_us | 360 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_gap_p50_us | 10 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | added_gap_p99_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | streams_sustained_fps | 32614.06218906021 | valid | — |
| openai → openai | streams_sustained | 2048 | valid | — |
| openai → openai | idle_rss_mib | 34.00390625 | valid | — |
| openai → openai | steady_state_rss_mib | 53.4921875 | valid | — |
| openai → openai | recovered_rss_mib | 49.99609375 | valid | — |
| openai → openai | peak_rss_mib | 54.5234375 | valid | — |
| openai → openai | peak_rss_hwm_mib | 54.2265625 | valid | — |
| openai → openai | time_to_plateau_s | not_measured | not_measured | — |
| openai → openai | growth_rate_mib_per_min | -2.0749110660555545 | valid | — |
| openai → openai | load_s | 300 | valid | — |
| openai → openai | shape | -1 | valid | — |
| openai → openai | idle_shape | not_measured | not_measured | — |
| openai → openai | idle_static | 1 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | 0.03612607648971414 | valid | — |
| openai → openai | throughput_rps | 21156 req/s | concurrency 8; samples 126952 | 1 |
| openai → openai | throughput_rps | 22239 req/s | concurrency 64; samples 133447 | 5 |
| openai → openai | throughput_rps | 22380 req/s | concurrency 128; samples 134304 | 10 |
| openai → openai | throughput_rps | 22545 req/s | concurrency 256; samples 135283 | 50 |
| openai → openai | throughput_rps | 22545 req/s | concurrency 256; samples 135283 | 100 |
| openai → openai | throughput_rps | 22545 req/s | concurrency 256; samples 135283 | unbounded |

## tensorzero

Build tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89. Measured 2026-11-10T17:40:04Z.  [Run evidence](https://onthebench.ai/gateways/runs/run-tensorzero-e1970318de8934076256f5f37dbf5d7fb6019f1dd5bb339bd959ac71a0a1a8e1)

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 40976 | valid; gateway_c1_samples=150; direct_c1_samples=253662 | — |
| openai → openai | added_latency_p99_us | 41966 | valid; gateway_c1_samples=150; direct_c1_samples=253662 | — |
| openai → openai | gateway_c1_p99_us | 41996 | valid; gateway_c1_samples=150 | — |
| openai → openai | direct_c1_p99_us | 30 | valid; direct_c1_samples=253662 | — |
| openai → openai | cpu_us_per_request | 459.0984974958264 | valid | — |
| openai → openai | rps_per_cpu_second | 2178.181818181818 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.022719677197451632 | valid | — |
| openai → openai | cost_window_ok | 1198 | valid | — |
| openai → openai | cost_window_rps | 199 | valid | — |
| openai → openai | cost_threads | 6 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | 707 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_ttft_p99_us | 746 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_gap_p50_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | added_gap_p99_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | streams_sustained_fps | 8048.563390252819 | valid | — |
| openai → openai | streams_sustained | 1991 | valid | — |
| openai → openai | idle_rss_mib | 48.75390625 | valid | — |
| openai → openai | steady_state_rss_mib | 67.6171875 | valid | — |
| openai → openai | recovered_rss_mib | 67.6171875 | valid | — |
| openai → openai | peak_rss_mib | 67.6171875 | valid | — |
| openai → openai | peak_rss_hwm_mib | 67.6171875 | valid | — |
| openai → openai | time_to_plateau_s | 60 | valid | — |
| openai → openai | growth_rate_mib_per_min | 0 | valid | — |
| openai → openai | load_s | 302 | valid | — |
| openai → openai | shape | not_measured | not_measured | — |
| openai → openai | idle_shape | not_measured | not_measured | — |
| openai → openai | idle_static | 1 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | 0.024793797064739817 | valid | — |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → openai | throughput_rps | 6275 req/s | concurrency 256; samples 37658 | 50 |
| openai → openai | throughput_rps | 11879 req/s | concurrency 512; samples 71288 | 100 |
| openai → openai | throughput_rps | 12324 req/s | concurrency 1024; samples 73956 | unbounded |
