# TensorZero benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency.  [Methodology and limitations](https://onthebench.ai/gateways/method) .

Rust · recorded build: tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89. Measured: 2026-11-10T17:40:04Z.

[JSON](https://onthebench.ai/api/v1/gateways/tensorzero)  ·  [Markdown](https://onthebench.ai/gateways/tensorzero.md)  ·  [CSV](https://onthebench.ai/gateways/data.csv?gateway=tensorzero&compact=1)  ·  [Project source](https://github.com/tensorzero/tensorzero)  ·  [Interactive dashboard](https://onthebench.ai/gateways)

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

## Published measurements

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| bedrock → openai | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| gemini → openai | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| anthropic → openai | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| openai → openai | added_latency_p50_us | 40976 | valid; gateway_c1_samples=150; direct_c1_samples=253662 | — |
| openai → openai | added_latency_p99_us | 41966 | valid; gateway_c1_samples=150; direct_c1_samples=253662 | — |
| openai → openai | gateway_c1_p99_us | 41996 | valid; gateway_c1_samples=150 | — |
| openai → openai | direct_c1_p99_us | 30 | valid; direct_c1_samples=253662 | — |
| openai → openai | cpu_us_per_request | 459.0984974958264 | valid | — |
| openai → openai | rps_per_cpu_second | 2178.181818181818 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.022719677197451632 | valid | — |
| openai → openai | cost_window_ok | 1198 | valid | — |
| openai → openai | cost_window_rps | 199 | valid | — |
| openai → openai | cost_threads | 6 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | 707 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_ttft_p99_us | 746 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_gap_p50_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | added_gap_p99_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | streams_sustained_fps | 8048.563390252819 | valid | — |
| openai → openai | streams_sustained | 1991 | valid | — |
| openai → openai | idle_rss_mib | 48.75390625 | valid | — |
| openai → openai | steady_state_rss_mib | 67.6171875 | valid | — |
| openai → openai | recovered_rss_mib | 67.6171875 | valid | — |
| openai → openai | peak_rss_mib | 67.6171875 | valid | — |
| openai → openai | peak_rss_hwm_mib | 67.6171875 | valid | — |
| openai → openai | time_to_plateau_s | 60 | valid | — |
| openai → openai | growth_rate_mib_per_min | 0 | valid | — |
| openai → openai | load_s | 302 | valid | — |
| openai → openai | shape | not_measured | not_measured | — |
| openai → openai | idle_shape | not_measured | not_measured | — |
| openai → openai | idle_static | 1 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | 0.024793797064739817 | valid | — |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → openai | throughput_rps | 6275 req/s | concurrency 256; samples 37658 | 50 |
| openai → openai | throughput_rps | 11879 req/s | concurrency 512; samples 71288 | 100 |
| openai → openai | throughput_rps | 12324 req/s | concurrency 1024; samples 73956 | unbounded |
| openai-responses → openai | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| cohere → openai | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| openai-responses → gemini | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| bedrock → gemini | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| openai → gemini (upstream over HTTPS) | added_latency_p50_us | 40971 | valid; gateway_c1_samples=148; direct_c1_samples=210647 | — |
| openai → gemini (upstream over HTTPS) | added_latency_p99_us | 40996 | valid; gateway_c1_samples=148; direct_c1_samples=210647 | — |
| openai → gemini (upstream over HTTPS) | gateway_c1_p99_us | 41035 | valid; gateway_c1_samples=148 | — |
| openai → gemini (upstream over HTTPS) | direct_c1_p99_us | 39 | valid; direct_c1_samples=210647 | — |
| openai → gemini (upstream over HTTPS) | cpu_us_per_request | 578.1758957654723 | valid | — |
| openai → gemini (upstream over HTTPS) | rps_per_cpu_second | 1729.5774647887324 | valid | — |
| openai → gemini (upstream over HTTPS) | cost_window_conc | 8 | valid | — |
| openai → gemini (upstream over HTTPS) | cost_core_utilisation | 0.02929945152374234 | valid | — |
| openai → gemini (upstream over HTTPS) | cost_window_ok | 1228 | valid | — |
| openai → gemini (upstream over HTTPS) | cost_window_rps | 204 | valid | — |
| openai → gemini (upstream over HTTPS) | cost_threads | 6 | valid | — |
| openai → gemini (upstream over HTTPS) | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → gemini (upstream over HTTPS) | cost_majflt | 0 | valid | — |
| openai → gemini (upstream over HTTPS) | added_ttft_p50_us | untestable | untestable | — |
| openai → gemini (upstream over HTTPS) | added_ttft_p99_us | untestable | untestable | — |
| openai → gemini (upstream over HTTPS) | added_gap_p50_us | untestable | untestable | — |
| openai → gemini (upstream over HTTPS) | added_gap_p99_us | untestable | untestable | — |
| openai → gemini (upstream over HTTPS) | streams_sustained_fps | untestable | untestable | — |
| openai → gemini (upstream over HTTPS) | streams_sustained | untestable | untestable | — |
| openai → gemini (upstream over HTTPS) | idle_rss_mib | 50.7421875 | valid | — |
| openai → gemini (upstream over HTTPS) | steady_state_rss_mib | 67.95703125 | valid | — |
| openai → gemini (upstream over HTTPS) | recovered_rss_mib | 67.95703125 | valid | — |
| openai → gemini (upstream over HTTPS) | peak_rss_mib | 67.95703125 | valid | — |
| openai → gemini (upstream over HTTPS) | peak_rss_hwm_mib | 67.95703125 | valid | — |
| openai → gemini (upstream over HTTPS) | time_to_plateau_s | 66 | valid | — |
| openai → gemini (upstream over HTTPS) | growth_rate_mib_per_min | 0 | valid | — |
| openai → gemini (upstream over HTTPS) | load_s | 302 | valid | — |
| openai → gemini (upstream over HTTPS) | shape | not_measured | not_measured | — |
| openai → gemini (upstream over HTTPS) | idle_shape | not_measured | not_measured | — |
| openai → gemini (upstream over HTTPS) | idle_static | 1 | valid | — |
| openai → gemini (upstream over HTTPS) | idle_growth_rate_mib_per_min | 0.02471548973980937 | valid | — |
| openai → gemini (upstream over HTTPS) | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → gemini (upstream over HTTPS) | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → gemini (upstream over HTTPS) | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → gemini (upstream over HTTPS) | throughput_rps | 6222 req/s | concurrency 256; samples 37338 | 50 |
| openai → gemini (upstream over HTTPS) | throughput_rps | 13987 req/s | concurrency 2048; samples 83936 | 100 |
| openai → gemini (upstream over HTTPS) | throughput_rps | 13987 req/s | concurrency 2048; samples 83936 | unbounded |
| gemini → gemini | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| anthropic → gemini | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| cohere → gemini | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| gemini → bedrock | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| cohere → bedrock | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| anthropic → bedrock | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| bedrock → bedrock | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| openai-responses → bedrock | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| openai → bedrock | added_latency_p50_us | 40977 | valid; gateway_c1_samples=148; direct_c1_samples=261020 | — |
| openai → bedrock | added_latency_p99_us | 41012 | valid; gateway_c1_samples=148; direct_c1_samples=261020 | — |
| openai → bedrock | gateway_c1_p99_us | 41040 | valid; gateway_c1_samples=148 | — |
| openai → bedrock | direct_c1_p99_us | 28 | valid; direct_c1_samples=261020 | — |
| openai → bedrock | cpu_us_per_request | 468.6192468619247 | valid | — |
| openai → bedrock | rps_per_cpu_second | 2133.928571428571 | valid | — |
| openai → bedrock | cost_window_conc | 8 | valid | — |
| openai → bedrock | cost_core_utilisation | 0.02312914819538198 | valid | — |
| openai → bedrock | cost_window_ok | 1195 | valid | — |
| openai → bedrock | cost_window_rps | 199 | valid | — |
| openai → bedrock | cost_threads | 6 | valid | — |
| openai → bedrock | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → bedrock | cost_majflt | 0 | valid | — |
| openai → bedrock | added_ttft_p50_us | untestable | untestable | — |
| openai → bedrock | added_ttft_p99_us | untestable | untestable | — |
| openai → bedrock | added_gap_p50_us | untestable | untestable | — |
| openai → bedrock | added_gap_p99_us | untestable | untestable | — |
| openai → bedrock | streams_sustained_fps | untestable | untestable | — |
| openai → bedrock | streams_sustained | untestable | untestable | — |
| openai → bedrock | idle_rss_mib | 50.73828125 | valid | — |
| openai → bedrock | steady_state_rss_mib | 67.5546875 | valid | — |
| openai → bedrock | recovered_rss_mib | 67.5546875 | valid | — |
| openai → bedrock | peak_rss_mib | 67.5546875 | valid | — |
| openai → bedrock | peak_rss_hwm_mib | 67.5546875 | valid | — |
| openai → bedrock | time_to_plateau_s | 72 | valid | — |
| openai → bedrock | growth_rate_mib_per_min | 0 | valid | — |
| openai → bedrock | load_s | 302 | valid | — |
| openai → bedrock | shape | not_measured | not_measured | — |
| openai → bedrock | idle_shape | not_measured | not_measured | — |
| openai → bedrock | idle_static | 1 | valid | — |
| openai → bedrock | idle_growth_rate_mib_per_min | 0.02471213797252101 | valid | — |
| openai → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → bedrock | throughput_rps | 6243 req/s | concurrency 256; samples 37466 | 50 |
| openai → bedrock | throughput_rps | 11572 req/s | concurrency 512; samples 69447 | 100 |
| openai → bedrock | throughput_rps | 11572 req/s | concurrency 512; samples 69447 | unbounded |
| openai → openai-responses | added_latency_p50_us | 40976 | valid; gateway_c1_samples=148; direct_c1_samples=258742 | — |
| openai → openai-responses | added_latency_p99_us | 40988 | valid; gateway_c1_samples=148; direct_c1_samples=258742 | — |
| openai → openai-responses | gateway_c1_p99_us | 41017 | valid; gateway_c1_samples=148 | — |
| openai → openai-responses | direct_c1_p99_us | 29 | valid; direct_c1_samples=258742 | — |
| openai → openai-responses | cpu_us_per_request | 452.2613065326633 | valid | — |
| openai → openai-responses | rps_per_cpu_second | 2211.111111111111 | valid | — |
| openai → openai-responses | cost_window_conc | 8 | valid | — |
| openai → openai-responses | cost_core_utilisation | 0.022306389008679293 | valid | — |
| openai → openai-responses | cost_window_ok | 1194 | valid | — |
| openai → openai-responses | cost_window_rps | 198 | valid | — |
| openai → openai-responses | cost_threads | 6 | valid | — |
| openai → openai-responses | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → openai-responses | cost_majflt | 0 | valid | — |
| openai → openai-responses | added_ttft_p50_us | untestable | untestable | — |
| openai → openai-responses | added_ttft_p99_us | untestable | untestable | — |
| openai → openai-responses | added_gap_p50_us | untestable | untestable | — |
| openai → openai-responses | added_gap_p99_us | untestable | untestable | — |
| openai → openai-responses | streams_sustained_fps | untestable | untestable | — |
| openai → openai-responses | streams_sustained | untestable | untestable | — |
| openai → openai-responses | idle_rss_mib | 50.70703125 | valid | — |
| openai → openai-responses | steady_state_rss_mib | 67.7265625 | valid | — |
| openai → openai-responses | recovered_rss_mib | 67.7265625 | valid | — |
| openai → openai-responses | peak_rss_mib | 67.7265625 | valid | — |
| openai → openai-responses | peak_rss_hwm_mib | 67.7265625 | valid | — |
| openai → openai-responses | time_to_plateau_s | 78 | valid | — |
| openai → openai-responses | growth_rate_mib_per_min | 0 | valid | — |
| openai → openai-responses | load_s | 302 | valid | — |
| openai → openai-responses | shape | not_measured | not_measured | — |
| openai → openai-responses | idle_shape | not_measured | not_measured | — |
| openai → openai-responses | idle_static | 1 | valid | — |
| openai → openai-responses | idle_growth_rate_mib_per_min | 0.024782967493901836 | valid | — |
| openai → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → openai-responses | throughput_rps | 6238 req/s | concurrency 256; samples 37439 | 50 |
| openai → openai-responses | throughput_rps | 11643 req/s | concurrency 512; samples 69870 | 100 |
| openai → openai-responses | throughput_rps | 11643 req/s | concurrency 512; samples 69870 | unbounded |
| gemini → openai-responses | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| cohere → openai-responses | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| openai-responses → openai-responses | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| bedrock → openai-responses | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| anthropic → openai-responses | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| anthropic → anthropic | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| cohere → anthropic | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| openai-responses → anthropic | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| gemini → anthropic | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| bedrock → anthropic | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| openai → anthropic | added_latency_p50_us | 40977 | valid; gateway_c1_samples=151; direct_c1_samples=259563 | — |
| openai → anthropic | added_latency_p99_us | 41418 | valid; gateway_c1_samples=151; direct_c1_samples=259563 | — |
| openai → anthropic | gateway_c1_p99_us | 41446 | valid; gateway_c1_samples=151 | — |
| openai → anthropic | direct_c1_p99_us | 28 | valid; direct_c1_samples=259563 | — |
| openai → anthropic | cpu_us_per_request | 460.63651591289783 | valid | — |
| openai → anthropic | rps_per_cpu_second | 2170.9090909090905 | valid | — |
| openai → anthropic | cost_window_conc | 8 | valid | — |
| openai → anthropic | cost_core_utilisation | 0.02271457592885274 | valid | — |
| openai → anthropic | cost_window_ok | 1194 | valid | — |
| openai → anthropic | cost_window_rps | 198 | valid | — |
| openai → anthropic | cost_threads | 6 | valid | — |
| openai → anthropic | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → anthropic | cost_majflt | 0 | valid | — |
| openai → anthropic | added_ttft_p50_us | 780 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → anthropic | added_ttft_p99_us | 856 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → anthropic | added_gap_p50_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → anthropic | added_gap_p99_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → anthropic | streams_sustained_fps | 8248.116178163695 | valid | — |
| openai → anthropic | streams_sustained | 2008 | valid | — |
| openai → anthropic | idle_rss_mib | 50.7265625 | valid | — |
| openai → anthropic | steady_state_rss_mib | 67.6328125 | valid | — |
| openai → anthropic | recovered_rss_mib | 67.6328125 | valid | — |
| openai → anthropic | peak_rss_mib | 67.6328125 | valid | — |
| openai → anthropic | peak_rss_hwm_mib | 67.6328125 | valid | — |
| openai → anthropic | time_to_plateau_s | 90 | valid | — |
| openai → anthropic | growth_rate_mib_per_min | 0 | valid | — |
| openai → anthropic | load_s | 302 | valid | — |
| openai → anthropic | shape | not_measured | not_measured | — |
| openai → anthropic | idle_shape | not_measured | not_measured | — |
| openai → anthropic | idle_static | 1 | valid | — |
| openai → anthropic | idle_growth_rate_mib_per_min | 0.024722742409502847 | valid | — |
| openai → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → anthropic | throughput_rps | 6258 req/s | concurrency 256; samples 37557 | 50 |
| openai → anthropic | throughput_rps | 11681 req/s | concurrency 512; samples 70099 | 100 |
| openai → anthropic | throughput_rps | 12514 req/s | concurrency 1024; samples 75094 | unbounded |
| openai-responses → cohere | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| openai → cohere | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| anthropic → cohere | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| cohere → cohere | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| bedrock → cohere | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |
| gemini → cohere | — | — | not_declared: Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json | — |

## Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

| Ingress → upstream | Cell state | Probe HTTP status | Recorded reason | Recorded detail | Evidence scope | Response excerpt | Tested build |
| --- | --- | --- | --- | --- | --- | --- | --- |
| bedrock → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| gemini → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| anthropic → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai → openai | served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai-responses → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| cohere → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai-responses → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| bedrock → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai → gemini | served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| gemini → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| anthropic → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| cohere → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| gemini → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| cohere → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| anthropic → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| bedrock → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai-responses → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai → bedrock | served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai → openai-responses | served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| gemini → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| cohere → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai-responses → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| bedrock → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| anthropic → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| anthropic → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| cohere → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai-responses → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| gemini → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| bedrock → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai → anthropic | served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai-responses → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| openai → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| anthropic → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| cohere → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| bedrock → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |
| gemini → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89 |

**Harness commit**: `d240badd966933fa19eec0606b2d1f3945f18ad2`

**Mock SHA-256**: `99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8`

**Load generator SHA-256**: `not recorded`

**somaxconn**: 65535

**tcp_max_syn_backlog**: 65535

## Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

| Ingress → upstream | Concurrency | Lanes by HTTP status | Lanes with no response |
| --- | --- | --- | --- |
| openai → openai | 1 | {"200":1} | 0 |
| openai → openai | 2 | {"200":2} | 0 |
| openai → openai | 4 | {"200":4} | 0 |
| openai → openai | 8 | {"200":8} | 0 |
| openai → openai | 16 | {"200":16} | 0 |
| openai → openai | 32 | {"200":32} | 0 |
| openai → openai | 64 | {"200":64} | 0 |
| openai → openai | 128 | {"200":128} | 0 |
| openai → openai | 256 | {"200":256} | 0 |
| openai → openai | 512 | {"200":512} | 0 |
| openai → openai | 1024 | {"200":1024} | 0 |
| openai → openai | 2048 | {"200":2048} | 0 |
| openai → openai | 1536 | {"200":1536} | 0 |
| openai → openai | 1792 | {"200":1792} | 0 |
| openai → openai | 1920 | {"200":1920} | 0 |
| openai → openai | 1984 | {"200":1984} | 0 |
| openai → openai | 2016 | {"200":1921} | 95 |
| openai → openai | 2000 | {"200":2000} | 0 |
| openai → openai | 1992 | {"200":1992} | 0 |
| openai → openai | 1988 | {"200":1988} | 0 |
| openai → openai | 1990 | {"200":1990} | 0 |
| openai → openai | 1991 | {"200":1991} | 0 |
| openai → openai | 1991 | {"200":1991} | 0 |
| openai → openai | 1991 | {"200":1991} | 0 |
| openai → anthropic | 1 | {"200":1} | 0 |
| openai → anthropic | 2 | {"200":2} | 0 |
| openai → anthropic | 4 | {"200":4} | 0 |
| openai → anthropic | 8 | {"200":8} | 0 |
| openai → anthropic | 16 | {"200":16} | 0 |
| openai → anthropic | 32 | {"200":32} | 0 |
| openai → anthropic | 64 | {"200":64} | 0 |
| openai → anthropic | 128 | {"200":128} | 0 |
| openai → anthropic | 256 | {"200":256} | 0 |
| openai → anthropic | 512 | {"200":512} | 0 |
| openai → anthropic | 1024 | {"200":1024} | 0 |
| openai → anthropic | 2048 | {"200":2048} | 0 |
| openai → anthropic | 1536 | {"200":1536} | 0 |
| openai → anthropic | 1792 | {"200":1792} | 0 |
| openai → anthropic | 1920 | {"200":1920} | 0 |
| openai → anthropic | 1984 | {"200":1984} | 0 |
| openai → anthropic | 2016 | {"200":2016} | 0 |
| openai → anthropic | 2000 | {"200":2000} | 0 |
| openai → anthropic | 2008 | {"200":2008} | 0 |
| openai → anthropic | 2012 | {"200":2012} | 0 |
| openai → anthropic | 2010 | {"200":2010} | 0 |
| openai → anthropic | 2009 | {"200":2009} | 0 |
| openai → anthropic | 2008 | {"200":2008} | 0 |
| openai → anthropic | 2008 | {"200":2008} | 0 |

## Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

- Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json
- Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

## Evidence for the current measurements

- [run-tensorzero-e1970318de8934076256f5f37dbf5d7fb6019f1dd5bb339bd959ac71a0a1a8e1](https://onthebench.ai/gateways/runs/run-tensorzero-e1970318de8934076256f5f37dbf5d7fb6019f1dd5bb339bd959ac71a0a1a8e1)  ·  [complete raw record including RSS series](https://onthebench.ai/api/v1/runs/run-tensorzero-e1970318de8934076256f5f37dbf5d7fb6019f1dd5bb339bd959ac71a0a1a8e1/raw.json)

## Configuration recorded by the run

```
# ONE static config, four upstream models. The client selects the upstream per request with the
# OpenAI-compatible \`model\` field: tensorzero::model_name::mock-openai | mock-openai-responses |
# mock-anthropic | mock-gemini | mock-bedrock. Nothing here changes between egress columns.
[gateway.observability]
enabled = false

# openai -> POST {api_base}/chat/completions
[models.mock-openai]
routing = ["mock"]
[models.mock-openai.providers.mock]
type = "openai"
api_base = "http://127.0.0.1:8000/v1"
model_name = "gpt-4o-mini"
api_key_location = "none"

# openai + api_type="responses" -> POST {api_base}/responses
[models.mock-openai-responses]
routing = ["mock"]
[models.mock-openai-responses.providers.mock]
type = "openai"
api_type = "responses"
api_base = "http://127.0.0.1:8000/v1"
model_name = "gpt-4o-mini"
api_key_location = "none"

# anthropic -> POST {api_base}messages  (api_base is the API ROOT; the gateway appends \`messages\`).
# api_key_location is left at its default env::ANTHROPIC_API_KEY: the anthropic provider REJECTS
# "none" (providers/anthropic.rs:189-208 has no Credential::None arm), and credentials are validated
# at boot, so the dummy ANTHROPIC_API_KEY in gw_launch is what lets this model exist in the config.
[models.mock-anthropic]
routing = ["mock"]
[models.mock-anthropic.providers.mock]
type = "anthropic"
api_base = "http://127.0.0.1:8000/v1/"
model_name = "claude-3-5-sonnet-20241022"

# google_ai_studio_gemini -> POST https://generativelanguage.googleapis.com/v1beta/models/{model_name}:generateContent
# The provider has no api_base override, so it keeps its default host; engine v1.2's catch-all DNS
# resolves that host to the mock and the gateway trusts the run's CA. The key is the dummy
# GOOGLE_AI_STUDIO_API_KEY in env (the provider's default api_key_location).
[models.mock-gemini]
routing = ["mock"]
[models.mock-gemini.providers.mock]
type = "google_ai_studio_gemini"
model_name = "gemini-1.5-pro"

# aws_bedrock -> POST {endpoint_url}/model/{urlencoded model_id}/converse (SigV4-signed with the
# dummy AWS keys in gw_launch; the mock ignores the signature).
[models.mock-bedrock]
routing = ["mock"]
[models.mock-bedrock.providers.mock]
type = "aws_bedrock"
endpoint_url = "http://127.0.0.1:8000"
model_id = "anthropic.claude-3-sonnet-20240229-v1:0"
region = "us-east-1"

```

## Compare a shared protocol cell

- [TensorZero vs agentgateway](https://onthebench.ai/gateways/compare/tensorzero-vs-agentgateway)
- [TensorZero vs AISIX (api7)](https://onthebench.ai/gateways/compare/tensorzero-vs-aisix)
- [TensorZero vs APISIX](https://onthebench.ai/gateways/compare/tensorzero-vs-apisix)
- [TensorZero vs Bifrost](https://onthebench.ai/gateways/compare/tensorzero-vs-bifrost)
- [TensorZero vs Busbar](https://onthebench.ai/gateways/compare/tensorzero-vs-busbar)
- [TensorZero vs GoModel](https://onthebench.ai/gateways/compare/tensorzero-vs-gomodel)
- [TensorZero vs Helicone](https://onthebench.ai/gateways/compare/tensorzero-vs-helicone)
- [TensorZero vs Higress](https://onthebench.ai/gateways/compare/tensorzero-vs-higress)
- [TensorZero vs Kong](https://onthebench.ai/gateways/compare/tensorzero-vs-kong)
- [TensorZero vs LiteLLM · Python](https://onthebench.ai/gateways/compare/tensorzero-vs-litellm-python)
- [TensorZero vs LiteLLM · Rust](https://onthebench.ai/gateways/compare/tensorzero-vs-litellm-rust)
- [TensorZero vs One-API](https://onthebench.ai/gateways/compare/tensorzero-vs-one-api)
- [TensorZero vs Plano](https://onthebench.ai/gateways/compare/tensorzero-vs-plano)
- [TensorZero vs Portkey](https://onthebench.ai/gateways/compare/tensorzero-vs-portkey)

## Historical measurements

- [2026-11-10T17:40:04Z — tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89](https://onthebench.ai/gateways/runs/run-tensorzero-e1970318de8934076256f5f37dbf5d7fb6019f1dd5bb339bd959ac71a0a1a8e1)
- [2026-08-03T22:17:45Z — tensorzero/gateway:2026.6.0](https://onthebench.ai/gateways/runs/run-tensorzero-8d3c2b0262d67569e6d1c72c9670e49674df1989438d4705f67c8db9897abc67)
