TensorZero benchmark measurements
Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.
Rust · recorded build: tensorzero/gateway:2026.6.0. Measured: .
JSON · Markdown · CSV · Project source · Interactive dashboard
Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.
Published measurements
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| Bedrock → OpenAI | — | — | Not declared: recorded note 1 | — |
| Gemini → OpenAI | — | — | Not declared: recorded note 1 | — |
| Anthropic → OpenAI | — | — | Not declared: recorded note 1 | — |
| OpenAI → OpenAI | Added latency p50 (µs) | 40,976 | Valid · n=150 gateway / 253,662 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 41,966 | Valid · n=150 gateway / 253,662 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 41,996 | Valid · n=150 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 30 | Valid · n=253,662 | — |
| OpenAI → OpenAI | CPU per request (µs) | 459.1 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 2,178.2 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.023 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 1,198 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 199 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 6 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | 707 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | 746 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 8,048.6 | Valid | — |
| OpenAI → OpenAI | Streams sustained | 1,991 | Valid | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 48.75 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 67.62 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 67.62 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 67.62 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 67.62 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | 60 | Valid | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | 0 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 302 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | — | Not measured | — |
| OpenAI → OpenAI | Idle memory shape | — | Not measured | — |
| OpenAI → OpenAI | Idle memory static | 1 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | 0.025 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | 6,275 | concurrency 256; n=37,658 | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | 11,879 | concurrency 512; n=71,288 | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 12,324 | concurrency 1,024; n=73,956 | Unbounded |
| OpenAI Responses → OpenAI | — | — | Not declared: recorded note 1 | — |
| Cohere → OpenAI | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Gemini | — | — | Not declared: recorded note 1 | — |
| Bedrock → Gemini | — | — | Not declared: recorded note 1 | — |
| OpenAI → Gemini (upstream over HTTPS) | Added latency p50 (µs) | 40,971 | Valid · n=148 gateway / 210,647 direct | — |
| OpenAI → Gemini (upstream over HTTPS) | Added latency p99 (µs) | 40,996 | Valid · n=148 gateway / 210,647 direct | — |
| OpenAI → Gemini (upstream over HTTPS) | Gateway p99 at concurrency 1 (µs) | 41,035 | Valid · n=148 | — |
| OpenAI → Gemini (upstream over HTTPS) | Direct-to-mock p99 at concurrency 1 (µs) | 39 | Valid · n=210,647 | — |
| OpenAI → Gemini (upstream over HTTPS) | CPU per request (µs) | 578.2 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Requests per CPU-second | 1,729.6 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Cost window concurrency | 8 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Cost window core utilisation | 0.029 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Cost window successful requests | 1,228 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Cost window throughput (req/s) | 204 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Cost window threads | 6 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Cost window major page faults | 0 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI → Gemini (upstream over HTTPS) | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI → Gemini (upstream over HTTPS) | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI → Gemini (upstream over HTTPS) | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI → Gemini (upstream over HTTPS) | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI → Gemini (upstream over HTTPS) | Streams sustained | — | Untestable | — |
| OpenAI → Gemini (upstream over HTTPS) | Idle RSS (MiB) | 50.74 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Steady-state RSS (MiB) | 67.96 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Recovered RSS (MiB) | 67.96 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Peak RSS (MiB) | 67.96 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Peak RSS high-water mark (MiB) | 67.96 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Time to plateau (s) | 66 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Memory growth (MiB/min) | 0 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Memory load duration (s) | 302 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Memory shape under load | — | Not measured | — |
| OpenAI → Gemini (upstream over HTTPS) | Idle memory shape | — | Not measured | — |
| OpenAI → Gemini (upstream over HTTPS) | Idle memory static | 1 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Idle memory growth (MiB/min) | 0.025 | Valid | — |
| OpenAI → Gemini (upstream over HTTPS) | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI → Gemini (upstream over HTTPS) | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| OpenAI → Gemini (upstream over HTTPS) | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| OpenAI → Gemini (upstream over HTTPS) | Throughput (req/s) | 6,222 | concurrency 256; n=37,338 | 50 ms |
| OpenAI → Gemini (upstream over HTTPS) | Throughput (req/s) | 13,987 | concurrency 2,048; n=83,936 | 100 ms |
| OpenAI → Gemini (upstream over HTTPS) | Throughput (req/s) | 13,987 | concurrency 2,048; n=83,936 | Unbounded |
| Gemini → Gemini | — | — | Not declared: recorded note 1 | — |
| Anthropic → Gemini | — | — | Not declared: recorded note 1 | — |
| Cohere → Gemini | — | — | Not declared: recorded note 1 | — |
| Gemini → Bedrock | — | — | Not declared: recorded note 1 | — |
| Cohere → Bedrock | — | — | Not declared: recorded note 1 | — |
| Anthropic → Bedrock | — | — | Not declared: recorded note 1 | — |
| Bedrock → Bedrock | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Bedrock | — | — | Not declared: recorded note 1 | — |
| OpenAI → Bedrock | Added latency p50 (µs) | 40,977 | Valid · n=148 gateway / 261,020 direct | — |
| OpenAI → Bedrock | Added latency p99 (µs) | 41,012 | Valid · n=148 gateway / 261,020 direct | — |
| OpenAI → Bedrock | Gateway p99 at concurrency 1 (µs) | 41,040 | Valid · n=148 | — |
| OpenAI → Bedrock | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=261,020 | — |
| OpenAI → Bedrock | CPU per request (µs) | 468.6 | Valid | — |
| OpenAI → Bedrock | Requests per CPU-second | 2,133.9 | Valid | — |
| OpenAI → Bedrock | Cost window concurrency | 8 | Valid | — |
| OpenAI → Bedrock | Cost window core utilisation | 0.023 | Valid | — |
| OpenAI → Bedrock | Cost window successful requests | 1,195 | Valid | — |
| OpenAI → Bedrock | Cost window throughput (req/s) | 199 | Valid | — |
| OpenAI → Bedrock | Cost window threads | 6 | Valid | — |
| OpenAI → Bedrock | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → Bedrock | Cost window major page faults | 0 | Valid | — |
| OpenAI → Bedrock | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI → Bedrock | Streams sustained | — | Untestable | — |
| OpenAI → Bedrock | Idle RSS (MiB) | 50.74 | Valid | — |
| OpenAI → Bedrock | Steady-state RSS (MiB) | 67.55 | Valid | — |
| OpenAI → Bedrock | Recovered RSS (MiB) | 67.55 | Valid | — |
| OpenAI → Bedrock | Peak RSS (MiB) | 67.55 | Valid | — |
| OpenAI → Bedrock | Peak RSS high-water mark (MiB) | 67.55 | Valid | — |
| OpenAI → Bedrock | Time to plateau (s) | 72 | Valid | — |
| OpenAI → Bedrock | Memory growth (MiB/min) | 0 | Valid | — |
| OpenAI → Bedrock | Memory load duration (s) | 302 | Valid | — |
| OpenAI → Bedrock | Memory shape under load | — | Not measured | — |
| OpenAI → Bedrock | Idle memory shape | — | Not measured | — |
| OpenAI → Bedrock | Idle memory static | 1 | Valid | — |
| OpenAI → Bedrock | Idle memory growth (MiB/min) | 0.025 | Valid | — |
| OpenAI → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| OpenAI → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| OpenAI → Bedrock | Throughput (req/s) | 6,243 | concurrency 256; n=37,466 | 50 ms |
| OpenAI → Bedrock | Throughput (req/s) | 11,572 | concurrency 512; n=69,447 | 100 ms |
| OpenAI → Bedrock | Throughput (req/s) | 11,572 | concurrency 512; n=69,447 | Unbounded |
| OpenAI → OpenAI Responses | Added latency p50 (µs) | 40,976 | Valid · n=148 gateway / 258,742 direct | — |
| OpenAI → OpenAI Responses | Added latency p99 (µs) | 40,988 | Valid · n=148 gateway / 258,742 direct | — |
| OpenAI → OpenAI Responses | Gateway p99 at concurrency 1 (µs) | 41,017 | Valid · n=148 | — |
| OpenAI → OpenAI Responses | Direct-to-mock p99 at concurrency 1 (µs) | 29 | Valid · n=258,742 | — |
| OpenAI → OpenAI Responses | CPU per request (µs) | 452.3 | Valid | — |
| OpenAI → OpenAI Responses | Requests per CPU-second | 2,211.1 | Valid | — |
| OpenAI → OpenAI Responses | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI Responses | Cost window core utilisation | 0.022 | Valid | — |
| OpenAI → OpenAI Responses | Cost window successful requests | 1,194 | Valid | — |
| OpenAI → OpenAI Responses | Cost window throughput (req/s) | 198 | Valid | — |
| OpenAI → OpenAI Responses | Cost window threads | 6 | Valid | — |
| OpenAI → OpenAI Responses | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → OpenAI Responses | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI Responses | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI → OpenAI Responses | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI → OpenAI Responses | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI → OpenAI Responses | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI → OpenAI Responses | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI → OpenAI Responses | Streams sustained | — | Untestable | — |
| OpenAI → OpenAI Responses | Idle RSS (MiB) | 50.71 | Valid | — |
| OpenAI → OpenAI Responses | Steady-state RSS (MiB) | 67.73 | Valid | — |
| OpenAI → OpenAI Responses | Recovered RSS (MiB) | 67.73 | Valid | — |
| OpenAI → OpenAI Responses | Peak RSS (MiB) | 67.73 | Valid | — |
| OpenAI → OpenAI Responses | Peak RSS high-water mark (MiB) | 67.73 | Valid | — |
| OpenAI → OpenAI Responses | Time to plateau (s) | 78 | Valid | — |
| OpenAI → OpenAI Responses | Memory growth (MiB/min) | 0 | Valid | — |
| OpenAI → OpenAI Responses | Memory load duration (s) | 302 | Valid | — |
| OpenAI → OpenAI Responses | Memory shape under load | — | Not measured | — |
| OpenAI → OpenAI Responses | Idle memory shape | — | Not measured | — |
| OpenAI → OpenAI Responses | Idle memory static | 1 | Valid | — |
| OpenAI → OpenAI Responses | Idle memory growth (MiB/min) | 0.025 | Valid | — |
| OpenAI → OpenAI Responses | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI → OpenAI Responses | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| OpenAI → OpenAI Responses | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| OpenAI → OpenAI Responses | Throughput (req/s) | 6,238 | concurrency 256; n=37,439 | 50 ms |
| OpenAI → OpenAI Responses | Throughput (req/s) | 11,643 | concurrency 512; n=69,870 | 100 ms |
| OpenAI → OpenAI Responses | Throughput (req/s) | 11,643 | concurrency 512; n=69,870 | Unbounded |
| Gemini → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Cohere → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Bedrock → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Anthropic → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Anthropic → Anthropic | — | — | Not declared: recorded note 1 | — |
| Cohere → Anthropic | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Anthropic | — | — | Not declared: recorded note 1 | — |
| Gemini → Anthropic | — | — | Not declared: recorded note 1 | — |
| Bedrock → Anthropic | — | — | Not declared: recorded note 1 | — |
| OpenAI → Anthropic | Added latency p50 (µs) | 40,977 | Valid · n=151 gateway / 259,563 direct | — |
| OpenAI → Anthropic | Added latency p99 (µs) | 41,418 | Valid · n=151 gateway / 259,563 direct | — |
| OpenAI → Anthropic | Gateway p99 at concurrency 1 (µs) | 41,446 | Valid · n=151 | — |
| OpenAI → Anthropic | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=259,563 | — |
| OpenAI → Anthropic | CPU per request (µs) | 460.6 | Valid | — |
| OpenAI → Anthropic | Requests per CPU-second | 2,170.9 | Valid | — |
| OpenAI → Anthropic | Cost window concurrency | 8 | Valid | — |
| OpenAI → Anthropic | Cost window core utilisation | 0.023 | Valid | — |
| OpenAI → Anthropic | Cost window successful requests | 1,194 | Valid | — |
| OpenAI → Anthropic | Cost window throughput (req/s) | 198 | Valid | — |
| OpenAI → Anthropic | Cost window threads | 6 | Valid | — |
| OpenAI → Anthropic | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → Anthropic | Cost window major page faults | 0 | Valid | — |
| OpenAI → Anthropic | Added time to first event p50 (µs) | 780 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → Anthropic | Added time to first event p99 (µs) | 856 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → Anthropic | Added inter-frame gap p50 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → Anthropic | Added inter-frame gap p99 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → Anthropic | Streams sustained (frames/s) | 8,248.1 | Valid | — |
| OpenAI → Anthropic | Streams sustained | 2,008 | Valid | — |
| OpenAI → Anthropic | Idle RSS (MiB) | 50.73 | Valid | — |
| OpenAI → Anthropic | Steady-state RSS (MiB) | 67.63 | Valid | — |
| OpenAI → Anthropic | Recovered RSS (MiB) | 67.63 | Valid | — |
| OpenAI → Anthropic | Peak RSS (MiB) | 67.63 | Valid | — |
| OpenAI → Anthropic | Peak RSS high-water mark (MiB) | 67.63 | Valid | — |
| OpenAI → Anthropic | Time to plateau (s) | 90 | Valid | — |
| OpenAI → Anthropic | Memory growth (MiB/min) | 0 | Valid | — |
| OpenAI → Anthropic | Memory load duration (s) | 302 | Valid | — |
| OpenAI → Anthropic | Memory shape under load | — | Not measured | — |
| OpenAI → Anthropic | Idle memory shape | — | Not measured | — |
| OpenAI → Anthropic | Idle memory static | 1 | Valid | — |
| OpenAI → Anthropic | Idle memory growth (MiB/min) | 0.025 | Valid | — |
| OpenAI → Anthropic | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI → Anthropic | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| OpenAI → Anthropic | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| OpenAI → Anthropic | Throughput (req/s) | 6,258 | concurrency 256; n=37,557 | 50 ms |
| OpenAI → Anthropic | Throughput (req/s) | 11,681 | concurrency 512; n=70,099 | 100 ms |
| OpenAI → Anthropic | Throughput (req/s) | 12,514 | concurrency 1,024; n=75,094 | Unbounded |
| OpenAI Responses → Cohere | — | — | Not declared: recorded note 1 | — |
| OpenAI → Cohere | — | — | Not declared: recorded note 1 | — |
| Anthropic → Cohere | — | — | Not declared: recorded note 1 | — |
| Cohere → Cohere | — | — | Not declared: recorded note 1 | — |
| Bedrock → Cohere | — | — | Not declared: recorded note 1 | — |
| Gemini → Cohere | — | — | Not declared: recorded note 1 | — |
Probe evidence and instrument provenance
Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.
| Ingress → upstream | Cell state | Probe HTTP status | Recorded reason | Recorded detail | Evidence scope | Response excerpt | Tested build |
|---|---|---|---|---|---|---|---|
| Bedrock → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Gemini → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Anthropic → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| OpenAI → OpenAI | Served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0 |
| OpenAI Responses → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Cohere → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| OpenAI Responses → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Bedrock → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| OpenAI → Gemini | Served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0 |
| Gemini → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Anthropic → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Cohere → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Gemini → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Cohere → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Anthropic → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Bedrock → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| OpenAI Responses → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| OpenAI → Bedrock | Served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0 |
| OpenAI → OpenAI Responses | Served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0 |
| Gemini → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Cohere → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| OpenAI Responses → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Bedrock → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Anthropic → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Anthropic → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Cohere → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| OpenAI Responses → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Gemini → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Bedrock → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| OpenAI → Anthropic | Served | — | — | — | Recorded run evidence | — | tensorzero/gateway:2026.6.0 |
| OpenAI Responses → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| OpenAI → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Anthropic → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Cohere → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Bedrock → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
| Gemini → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | tensorzero/gateway:2026.6.0 |
- Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2- Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8- Load generator SHA-256
not recorded- net.core.somaxconn
- 65,535
- net.ipv4.tcp_max_syn_backlog
- 65,535
Stream sweep diagnostics
Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.
| Ingress → upstream | Concurrency | Lanes by HTTP status | Lanes with no response |
|---|---|---|---|
| OpenAI → OpenAI | 1 | HTTP 200: 1 | 0 |
| OpenAI → OpenAI | 2 | HTTP 200: 2 | 0 |
| OpenAI → OpenAI | 4 | HTTP 200: 4 | 0 |
| OpenAI → OpenAI | 8 | HTTP 200: 8 | 0 |
| OpenAI → OpenAI | 16 | HTTP 200: 16 | 0 |
| OpenAI → OpenAI | 32 | HTTP 200: 32 | 0 |
| OpenAI → OpenAI | 64 | HTTP 200: 64 | 0 |
| OpenAI → OpenAI | 128 | HTTP 200: 128 | 0 |
| OpenAI → OpenAI | 256 | HTTP 200: 256 | 0 |
| OpenAI → OpenAI | 512 | HTTP 200: 512 | 0 |
| OpenAI → OpenAI | 1,024 | HTTP 200: 1,024 | 0 |
| OpenAI → OpenAI | 2,048 | HTTP 200: 2,048 | 0 |
| OpenAI → OpenAI | 1,536 | HTTP 200: 1,536 | 0 |
| OpenAI → OpenAI | 1,792 | HTTP 200: 1,792 | 0 |
| OpenAI → OpenAI | 1,920 | HTTP 200: 1,920 | 0 |
| OpenAI → OpenAI | 1,984 | HTTP 200: 1,984 | 0 |
| OpenAI → OpenAI | 2,016 | HTTP 200: 1,921 | 95 |
| OpenAI → OpenAI | 2,000 | HTTP 200: 2,000 | 0 |
| OpenAI → OpenAI | 1,992 | HTTP 200: 1,992 | 0 |
| OpenAI → OpenAI | 1,988 | HTTP 200: 1,988 | 0 |
| OpenAI → OpenAI | 1,990 | HTTP 200: 1,990 | 0 |
| OpenAI → OpenAI | 1,991 | HTTP 200: 1,991 | 0 |
| OpenAI → OpenAI | 1,991 | HTTP 200: 1,991 | 0 |
| OpenAI → OpenAI | 1,991 | HTTP 200: 1,991 | 0 |
| OpenAI → Anthropic | 1 | HTTP 200: 1 | 0 |
| OpenAI → Anthropic | 2 | HTTP 200: 2 | 0 |
| OpenAI → Anthropic | 4 | HTTP 200: 4 | 0 |
| OpenAI → Anthropic | 8 | HTTP 200: 8 | 0 |
| OpenAI → Anthropic | 16 | HTTP 200: 16 | 0 |
| OpenAI → Anthropic | 32 | HTTP 200: 32 | 0 |
| OpenAI → Anthropic | 64 | HTTP 200: 64 | 0 |
| OpenAI → Anthropic | 128 | HTTP 200: 128 | 0 |
| OpenAI → Anthropic | 256 | HTTP 200: 256 | 0 |
| OpenAI → Anthropic | 512 | HTTP 200: 512 | 0 |
| OpenAI → Anthropic | 1,024 | HTTP 200: 1,024 | 0 |
| OpenAI → Anthropic | 2,048 | HTTP 200: 2,048 | 0 |
| OpenAI → Anthropic | 1,536 | HTTP 200: 1,536 | 0 |
| OpenAI → Anthropic | 1,792 | HTTP 200: 1,792 | 0 |
| OpenAI → Anthropic | 1,920 | HTTP 200: 1,920 | 0 |
| OpenAI → Anthropic | 1,984 | HTTP 200: 1,984 | 0 |
| OpenAI → Anthropic | 2,016 | HTTP 200: 2,016 | 0 |
| OpenAI → Anthropic | 2,000 | HTTP 200: 2,000 | 0 |
| OpenAI → Anthropic | 2,008 | HTTP 200: 2,008 | 0 |
| OpenAI → Anthropic | 2,012 | HTTP 200: 2,012 | 0 |
| OpenAI → Anthropic | 2,010 | HTTP 200: 2,010 | 0 |
| OpenAI → Anthropic | 2,009 | HTTP 200: 2,009 | 0 |
| OpenAI → Anthropic | 2,008 | HTTP 200: 2,008 | 0 |
| OpenAI → Anthropic | 2,008 | HTTP 200: 2,008 | 0 |
Recorded notes
Text recorded with the cells above. Each note is shown once; rows refer to it by number.
- Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json
- Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.
Evidence for the current measurements
- run-tensorzero-e1970318de8934076256f5f37dbf5d7fb6019f1dd5bb339bd959ac71a0a1a8e1 · complete raw record including RSS series
Configuration recorded by the run
# ONE static config, four upstream models. The client selects the upstream per request with the
# OpenAI-compatible \`model\` field: tensorzero::model_name::mock-openai | mock-openai-responses |
# mock-anthropic | mock-gemini | mock-bedrock. Nothing here changes between egress columns.
[gateway.observability]
enabled = false
# openai -> POST {api_base}/chat/completions
[models.mock-openai]
routing = ["mock"]
[models.mock-openai.providers.mock]
type = "openai"
api_base = "http://127.0.0.1:8000/v1"
model_name = "gpt-4o-mini"
api_key_location = "none"
# openai + api_type="responses" -> POST {api_base}/responses
[models.mock-openai-responses]
routing = ["mock"]
[models.mock-openai-responses.providers.mock]
type = "openai"
api_type = "responses"
api_base = "http://127.0.0.1:8000/v1"
model_name = "gpt-4o-mini"
api_key_location = "none"
# anthropic -> POST {api_base}messages (api_base is the API ROOT; the gateway appends \`messages\`).
# api_key_location is left at its default env::ANTHROPIC_API_KEY: the anthropic provider REJECTS
# "none" (providers/anthropic.rs:189-208 has no Credential::None arm), and credentials are validated
# at boot, so the dummy ANTHROPIC_API_KEY in gw_launch is what lets this model exist in the config.
[models.mock-anthropic]
routing = ["mock"]
[models.mock-anthropic.providers.mock]
type = "anthropic"
api_base = "http://127.0.0.1:8000/v1/"
model_name = "claude-3-5-sonnet-20241022"
# google_ai_studio_gemini -> POST https://generativelanguage.googleapis.com/v1beta/models/{model_name}:generateContent
# The provider has no api_base override, so it keeps its default host; engine v1.2's catch-all DNS
# resolves that host to the mock and the gateway trusts the run's CA. The key is the dummy
# GOOGLE_AI_STUDIO_API_KEY in env (the provider's default api_key_location).
[models.mock-gemini]
routing = ["mock"]
[models.mock-gemini.providers.mock]
type = "google_ai_studio_gemini"
model_name = "gemini-1.5-pro"
# aws_bedrock -> POST {endpoint_url}/model/{urlencoded model_id}/converse (SigV4-signed with the
# dummy AWS keys in gw_launch; the mock ignores the signature).
[models.mock-bedrock]
routing = ["mock"]
[models.mock-bedrock.providers.mock]
type = "aws_bedrock"
endpoint_url = "http://127.0.0.1:8000"
model_id = "anthropic.claude-3-sonnet-20240229-v1:0"
region = "us-east-1"
Compare a shared protocol cell
- TensorZero vs agentgateway
- TensorZero vs AISIX (api7)
- TensorZero vs APISIX
- TensorZero vs Bifrost
- TensorZero vs Busbar
- TensorZero vs GoModel
- TensorZero vs Helicone
- TensorZero vs Higress
- TensorZero vs Kong
- TensorZero vs LiteLLM · Python
- TensorZero vs LiteLLM · Rust
- TensorZero vs One-API
- TensorZero vs Plano
- TensorZero vs Portkey