TensorZero benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Rust · recorded build: tensorzero/gateway:2026.6.0. Measured: .

JSON · Markdown · CSV · Project source · Interactive dashboard

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

Published measurements

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
Bedrock → OpenAI——Not declared: recorded note 1—
Gemini → OpenAI——Not declared: recorded note 1—
Anthropic → OpenAI——Not declared: recorded note 1—
OpenAI → OpenAIAdded latency p50 (µs)40,976Valid · n=150 gateway / 253,662 direct—
OpenAI → OpenAIAdded latency p99 (µs)41,966Valid · n=150 gateway / 253,662 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)41,996Valid · n=150—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)30Valid · n=253,662—
OpenAI → OpenAICPU per request (µs)459.1Valid—
OpenAI → OpenAIRequests per CPU-second2,178.2Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.023Valid—
OpenAI → OpenAICost window successful requests1,198Valid—
OpenAI → OpenAICost window throughput (req/s)199Valid—
OpenAI → OpenAICost window threads6Valid—
OpenAI → OpenAINon-voluntary context switches per request0Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)707Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)746Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)8,048.6Valid—
OpenAI → OpenAIStreams sustained1,991Valid—
OpenAI → OpenAIIdle RSS (MiB)48.75Valid—
OpenAI → OpenAISteady-state RSS (MiB)67.62Valid—
OpenAI → OpenAIRecovered RSS (MiB)67.62Valid—
OpenAI → OpenAIPeak RSS (MiB)67.62Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)67.62Valid—
OpenAI → OpenAITime to plateau (s)60Valid—
OpenAI → OpenAIMemory growth (MiB/min)0Valid—
OpenAI → OpenAIMemory load duration (s)302Valid—
OpenAI → OpenAIMemory shape under load—Not measured—
OpenAI → OpenAIIdle memory shape—Not measured—
OpenAI → OpenAIIdle memory static1Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)0.025Valid—
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI → OpenAIThroughput (req/s)6,275concurrency 256; n=37,65850 ms
OpenAI → OpenAIThroughput (req/s)11,879concurrency 512; n=71,288100 ms
OpenAI → OpenAIThroughput (req/s)12,324concurrency 1,024; n=73,956Unbounded
OpenAI Responses → OpenAI——Not declared: recorded note 1—
Cohere → OpenAI——Not declared: recorded note 1—
OpenAI Responses → Gemini——Not declared: recorded note 1—
Bedrock → Gemini——Not declared: recorded note 1—
OpenAI → Gemini (upstream over HTTPS)Added latency p50 (µs)40,971Valid · n=148 gateway / 210,647 direct—
OpenAI → Gemini (upstream over HTTPS)Added latency p99 (µs)40,996Valid · n=148 gateway / 210,647 direct—
OpenAI → Gemini (upstream over HTTPS)Gateway p99 at concurrency 1 (µs)41,035Valid · n=148—
OpenAI → Gemini (upstream over HTTPS)Direct-to-mock p99 at concurrency 1 (µs)39Valid · n=210,647—
OpenAI → Gemini (upstream over HTTPS)CPU per request (µs)578.2Valid—
OpenAI → Gemini (upstream over HTTPS)Requests per CPU-second1,729.6Valid—
OpenAI → Gemini (upstream over HTTPS)Cost window concurrency8Valid—
OpenAI → Gemini (upstream over HTTPS)Cost window core utilisation0.029Valid—
OpenAI → Gemini (upstream over HTTPS)Cost window successful requests1,228Valid—
OpenAI → Gemini (upstream over HTTPS)Cost window throughput (req/s)204Valid—
OpenAI → Gemini (upstream over HTTPS)Cost window threads6Valid—
OpenAI → Gemini (upstream over HTTPS)Non-voluntary context switches per request0Valid—
OpenAI → Gemini (upstream over HTTPS)Cost window major page faults0Valid—
OpenAI → Gemini (upstream over HTTPS)Added time to first event p50 (µs)—Untestable—
OpenAI → Gemini (upstream over HTTPS)Added time to first event p99 (µs)—Untestable—
OpenAI → Gemini (upstream over HTTPS)Added inter-frame gap p50 (µs)—Untestable—
OpenAI → Gemini (upstream over HTTPS)Added inter-frame gap p99 (µs)—Untestable—
OpenAI → Gemini (upstream over HTTPS)Streams sustained (frames/s)—Untestable—
OpenAI → Gemini (upstream over HTTPS)Streams sustained—Untestable—
OpenAI → Gemini (upstream over HTTPS)Idle RSS (MiB)50.74Valid—
OpenAI → Gemini (upstream over HTTPS)Steady-state RSS (MiB)67.96Valid—
OpenAI → Gemini (upstream over HTTPS)Recovered RSS (MiB)67.96Valid—
OpenAI → Gemini (upstream over HTTPS)Peak RSS (MiB)67.96Valid—
OpenAI → Gemini (upstream over HTTPS)Peak RSS high-water mark (MiB)67.96Valid—
OpenAI → Gemini (upstream over HTTPS)Time to plateau (s)66Valid—
OpenAI → Gemini (upstream over HTTPS)Memory growth (MiB/min)0Valid—
OpenAI → Gemini (upstream over HTTPS)Memory load duration (s)302Valid—
OpenAI → Gemini (upstream over HTTPS)Memory shape under load—Not measured—
OpenAI → Gemini (upstream over HTTPS)Idle memory shape—Not measured—
OpenAI → Gemini (upstream over HTTPS)Idle memory static1Valid—
OpenAI → Gemini (upstream over HTTPS)Idle memory growth (MiB/min)0.025Valid—
OpenAI → Gemini (upstream over HTTPS)Throughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → Gemini (upstream over HTTPS)Throughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI → Gemini (upstream over HTTPS)Throughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI → Gemini (upstream over HTTPS)Throughput (req/s)6,222concurrency 256; n=37,33850 ms
OpenAI → Gemini (upstream over HTTPS)Throughput (req/s)13,987concurrency 2,048; n=83,936100 ms
OpenAI → Gemini (upstream over HTTPS)Throughput (req/s)13,987concurrency 2,048; n=83,936Unbounded
Gemini → Gemini——Not declared: recorded note 1—
Anthropic → Gemini——Not declared: recorded note 1—
Cohere → Gemini——Not declared: recorded note 1—
Gemini → Bedrock——Not declared: recorded note 1—
Cohere → Bedrock——Not declared: recorded note 1—
Anthropic → Bedrock——Not declared: recorded note 1—
Bedrock → Bedrock——Not declared: recorded note 1—
OpenAI Responses → Bedrock——Not declared: recorded note 1—
OpenAI → BedrockAdded latency p50 (µs)40,977Valid · n=148 gateway / 261,020 direct—
OpenAI → BedrockAdded latency p99 (µs)41,012Valid · n=148 gateway / 261,020 direct—
OpenAI → BedrockGateway p99 at concurrency 1 (µs)41,040Valid · n=148—
OpenAI → BedrockDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=261,020—
OpenAI → BedrockCPU per request (µs)468.6Valid—
OpenAI → BedrockRequests per CPU-second2,133.9Valid—
OpenAI → BedrockCost window concurrency8Valid—
OpenAI → BedrockCost window core utilisation0.023Valid—
OpenAI → BedrockCost window successful requests1,195Valid—
OpenAI → BedrockCost window throughput (req/s)199Valid—
OpenAI → BedrockCost window threads6Valid—
OpenAI → BedrockNon-voluntary context switches per request0Valid—
OpenAI → BedrockCost window major page faults0Valid—
OpenAI → BedrockAdded time to first event p50 (µs)—Untestable—
OpenAI → BedrockAdded time to first event p99 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → BedrockStreams sustained (frames/s)—Untestable—
OpenAI → BedrockStreams sustained—Untestable—
OpenAI → BedrockIdle RSS (MiB)50.74Valid—
OpenAI → BedrockSteady-state RSS (MiB)67.55Valid—
OpenAI → BedrockRecovered RSS (MiB)67.55Valid—
OpenAI → BedrockPeak RSS (MiB)67.55Valid—
OpenAI → BedrockPeak RSS high-water mark (MiB)67.55Valid—
OpenAI → BedrockTime to plateau (s)72Valid—
OpenAI → BedrockMemory growth (MiB/min)0Valid—
OpenAI → BedrockMemory load duration (s)302Valid—
OpenAI → BedrockMemory shape under load—Not measured—
OpenAI → BedrockIdle memory shape—Not measured—
OpenAI → BedrockIdle memory static1Valid—
OpenAI → BedrockIdle memory growth (MiB/min)0.025Valid—
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI → BedrockThroughput (req/s)6,243concurrency 256; n=37,46650 ms
OpenAI → BedrockThroughput (req/s)11,572concurrency 512; n=69,447100 ms
OpenAI → BedrockThroughput (req/s)11,572concurrency 512; n=69,447Unbounded
OpenAI → OpenAI ResponsesAdded latency p50 (µs)40,976Valid · n=148 gateway / 258,742 direct—
OpenAI → OpenAI ResponsesAdded latency p99 (µs)40,988Valid · n=148 gateway / 258,742 direct—
OpenAI → OpenAI ResponsesGateway p99 at concurrency 1 (µs)41,017Valid · n=148—
OpenAI → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)29Valid · n=258,742—
OpenAI → OpenAI ResponsesCPU per request (µs)452.3Valid—
OpenAI → OpenAI ResponsesRequests per CPU-second2,211.1Valid—
OpenAI → OpenAI ResponsesCost window concurrency8Valid—
OpenAI → OpenAI ResponsesCost window core utilisation0.022Valid—
OpenAI → OpenAI ResponsesCost window successful requests1,194Valid—
OpenAI → OpenAI ResponsesCost window throughput (req/s)198Valid—
OpenAI → OpenAI ResponsesCost window threads6Valid—
OpenAI → OpenAI ResponsesNon-voluntary context switches per request0Valid—
OpenAI → OpenAI ResponsesCost window major page faults0Valid—
OpenAI → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
OpenAI → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
OpenAI → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
OpenAI → OpenAI ResponsesStreams sustained—Untestable—
OpenAI → OpenAI ResponsesIdle RSS (MiB)50.71Valid—
OpenAI → OpenAI ResponsesSteady-state RSS (MiB)67.73Valid—
OpenAI → OpenAI ResponsesRecovered RSS (MiB)67.73Valid—
OpenAI → OpenAI ResponsesPeak RSS (MiB)67.73Valid—
OpenAI → OpenAI ResponsesPeak RSS high-water mark (MiB)67.73Valid—
OpenAI → OpenAI ResponsesTime to plateau (s)78Valid—
OpenAI → OpenAI ResponsesMemory growth (MiB/min)0Valid—
OpenAI → OpenAI ResponsesMemory load duration (s)302Valid—
OpenAI → OpenAI ResponsesMemory shape under load—Not measured—
OpenAI → OpenAI ResponsesIdle memory shape—Not measured—
OpenAI → OpenAI ResponsesIdle memory static1Valid—
OpenAI → OpenAI ResponsesIdle memory growth (MiB/min)0.025Valid—
OpenAI → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI → OpenAI ResponsesThroughput (req/s)6,238concurrency 256; n=37,43950 ms
OpenAI → OpenAI ResponsesThroughput (req/s)11,643concurrency 512; n=69,870100 ms
OpenAI → OpenAI ResponsesThroughput (req/s)11,643concurrency 512; n=69,870Unbounded
Gemini → OpenAI Responses——Not declared: recorded note 1—
Cohere → OpenAI Responses——Not declared: recorded note 1—
OpenAI Responses → OpenAI Responses——Not declared: recorded note 1—
Bedrock → OpenAI Responses——Not declared: recorded note 1—
Anthropic → OpenAI Responses——Not declared: recorded note 1—
Anthropic → Anthropic——Not declared: recorded note 1—
Cohere → Anthropic——Not declared: recorded note 1—
OpenAI Responses → Anthropic——Not declared: recorded note 1—
Gemini → Anthropic——Not declared: recorded note 1—
Bedrock → Anthropic——Not declared: recorded note 1—
OpenAI → AnthropicAdded latency p50 (µs)40,977Valid · n=151 gateway / 259,563 direct—
OpenAI → AnthropicAdded latency p99 (µs)41,418Valid · n=151 gateway / 259,563 direct—
OpenAI → AnthropicGateway p99 at concurrency 1 (µs)41,446Valid · n=151—
OpenAI → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=259,563—
OpenAI → AnthropicCPU per request (µs)460.6Valid—
OpenAI → AnthropicRequests per CPU-second2,170.9Valid—
OpenAI → AnthropicCost window concurrency8Valid—
OpenAI → AnthropicCost window core utilisation0.023Valid—
OpenAI → AnthropicCost window successful requests1,194Valid—
OpenAI → AnthropicCost window throughput (req/s)198Valid—
OpenAI → AnthropicCost window threads6Valid—
OpenAI → AnthropicNon-voluntary context switches per request0Valid—
OpenAI → AnthropicCost window major page faults0Valid—
OpenAI → AnthropicAdded time to first event p50 (µs)780Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded time to first event p99 (µs)856Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → AnthropicAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → AnthropicStreams sustained (frames/s)8,248.1Valid—
OpenAI → AnthropicStreams sustained2,008Valid—
OpenAI → AnthropicIdle RSS (MiB)50.73Valid—
OpenAI → AnthropicSteady-state RSS (MiB)67.63Valid—
OpenAI → AnthropicRecovered RSS (MiB)67.63Valid—
OpenAI → AnthropicPeak RSS (MiB)67.63Valid—
OpenAI → AnthropicPeak RSS high-water mark (MiB)67.63Valid—
OpenAI → AnthropicTime to plateau (s)90Valid—
OpenAI → AnthropicMemory growth (MiB/min)0Valid—
OpenAI → AnthropicMemory load duration (s)302Valid—
OpenAI → AnthropicMemory shape under load—Not measured—
OpenAI → AnthropicIdle memory shape—Not measured—
OpenAI → AnthropicIdle memory static1Valid—
OpenAI → AnthropicIdle memory growth (MiB/min)0.025Valid—
OpenAI → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI → AnthropicThroughput (req/s)6,258concurrency 256; n=37,55750 ms
OpenAI → AnthropicThroughput (req/s)11,681concurrency 512; n=70,099100 ms
OpenAI → AnthropicThroughput (req/s)12,514concurrency 1,024; n=75,094Unbounded
OpenAI Responses → Cohere——Not declared: recorded note 1—
OpenAI → Cohere——Not declared: recorded note 1—
Anthropic → Cohere——Not declared: recorded note 1—
Cohere → Cohere——Not declared: recorded note 1—
Bedrock → Cohere——Not declared: recorded note 1—
Gemini → Cohere——Not declared: recorded note 1—

Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

Ingress → upstreamCell stateProbe HTTP statusRecorded reasonRecorded detailEvidence scopeResponse excerptTested build
Bedrock → OpenAINot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Gemini → OpenAINot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Anthropic → OpenAINot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
OpenAI → OpenAIServed———Recorded run evidence—tensorzero/gateway:2026.6.0
OpenAI Responses → OpenAINot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Cohere → OpenAINot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
OpenAI Responses → GeminiNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Bedrock → GeminiNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
OpenAI → GeminiServed———Recorded run evidence—tensorzero/gateway:2026.6.0
Gemini → GeminiNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Anthropic → GeminiNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Cohere → GeminiNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Gemini → BedrockNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Cohere → BedrockNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Anthropic → BedrockNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Bedrock → BedrockNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
OpenAI Responses → BedrockNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
OpenAI → BedrockServed———Recorded run evidence—tensorzero/gateway:2026.6.0
OpenAI → OpenAI ResponsesServed———Recorded run evidence—tensorzero/gateway:2026.6.0
Gemini → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Cohere → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
OpenAI Responses → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Bedrock → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Anthropic → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Anthropic → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Cohere → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
OpenAI Responses → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Gemini → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Bedrock → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
OpenAI → AnthropicServed———Recorded run evidence—tensorzero/gateway:2026.6.0
OpenAI Responses → CohereNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
OpenAI → CohereNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Anthropic → CohereNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Cohere → CohereNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Bedrock → CohereNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Gemini → CohereNot declared—recorded note 1recorded note 1recorded note 2—tensorzero/gateway:2026.6.0
Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2
Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8
Load generator SHA-256
not recorded
net.core.somaxconn
65,535
net.ipv4.tcp_max_syn_backlog
65,535

Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

Ingress → upstreamConcurrencyLanes by HTTP statusLanes with no response
OpenAI → OpenAI1HTTP 200: 10
OpenAI → OpenAI2HTTP 200: 20
OpenAI → OpenAI4HTTP 200: 40
OpenAI → OpenAI8HTTP 200: 80
OpenAI → OpenAI16HTTP 200: 160
OpenAI → OpenAI32HTTP 200: 320
OpenAI → OpenAI64HTTP 200: 640
OpenAI → OpenAI128HTTP 200: 1280
OpenAI → OpenAI256HTTP 200: 2560
OpenAI → OpenAI512HTTP 200: 5120
OpenAI → OpenAI1,024HTTP 200: 1,0240
OpenAI → OpenAI2,048HTTP 200: 2,0480
OpenAI → OpenAI1,536HTTP 200: 1,5360
OpenAI → OpenAI1,792HTTP 200: 1,7920
OpenAI → OpenAI1,920HTTP 200: 1,9200
OpenAI → OpenAI1,984HTTP 200: 1,9840
OpenAI → OpenAI2,016HTTP 200: 1,92195
OpenAI → OpenAI2,000HTTP 200: 2,0000
OpenAI → OpenAI1,992HTTP 200: 1,9920
OpenAI → OpenAI1,988HTTP 200: 1,9880
OpenAI → OpenAI1,990HTTP 200: 1,9900
OpenAI → OpenAI1,991HTTP 200: 1,9910
OpenAI → OpenAI1,991HTTP 200: 1,9910
OpenAI → OpenAI1,991HTTP 200: 1,9910
OpenAI → Anthropic1HTTP 200: 10
OpenAI → Anthropic2HTTP 200: 20
OpenAI → Anthropic4HTTP 200: 40
OpenAI → Anthropic8HTTP 200: 80
OpenAI → Anthropic16HTTP 200: 160
OpenAI → Anthropic32HTTP 200: 320
OpenAI → Anthropic64HTTP 200: 640
OpenAI → Anthropic128HTTP 200: 1280
OpenAI → Anthropic256HTTP 200: 2560
OpenAI → Anthropic512HTTP 200: 5120
OpenAI → Anthropic1,024HTTP 200: 1,0240
OpenAI → Anthropic2,048HTTP 200: 2,0480
OpenAI → Anthropic1,536HTTP 200: 1,5360
OpenAI → Anthropic1,792HTTP 200: 1,7920
OpenAI → Anthropic1,920HTTP 200: 1,9200
OpenAI → Anthropic1,984HTTP 200: 1,9840
OpenAI → Anthropic2,016HTTP 200: 2,0160
OpenAI → Anthropic2,000HTTP 200: 2,0000
OpenAI → Anthropic2,008HTTP 200: 2,0080
OpenAI → Anthropic2,012HTTP 200: 2,0120
OpenAI → Anthropic2,010HTTP 200: 2,0100
OpenAI → Anthropic2,009HTTP 200: 2,0090
OpenAI → Anthropic2,008HTTP 200: 2,0080
OpenAI → Anthropic2,008HTTP 200: 2,0080

Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

  1. Pinned 2026.6.0 source 62eb8f63e8ec62018d70420dbf1a8c5d1c026315: crates/tensorzero-core/src/endpoints/openai_compatible/mod.rs exposes Chat Completions ingress; native TensorZero inference is outside this six-wire matrix. Four configured upstream adapters passed response-content and exact upstream-recorder checks: OpenAI Chat Completions, OpenAI Responses, Anthropic and Bedrock Converse. Gemini (google_ai_studio_gemini) hardcodes generativelanguage.googleapis.com over HTTPS; the engine v1.2 catch-all DNS resolves it to the mock, so that cell is probed and carries upstream_transport "tls". Evidence: results/diagnostics/refresh-configuration/tensorzero-protocol-preflight.json
  2. Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

Evidence for the current measurements

Configuration recorded by the run

# ONE static config, four upstream models. The client selects the upstream per request with the
# OpenAI-compatible \`model\` field: tensorzero::model_name::mock-openai | mock-openai-responses |
# mock-anthropic | mock-gemini | mock-bedrock. Nothing here changes between egress columns.
[gateway.observability]
enabled = false

# openai -> POST {api_base}/chat/completions
[models.mock-openai]
routing = ["mock"]
[models.mock-openai.providers.mock]
type = "openai"
api_base = "http://127.0.0.1:8000/v1"
model_name = "gpt-4o-mini"
api_key_location = "none"

# openai + api_type="responses" -> POST {api_base}/responses
[models.mock-openai-responses]
routing = ["mock"]
[models.mock-openai-responses.providers.mock]
type = "openai"
api_type = "responses"
api_base = "http://127.0.0.1:8000/v1"
model_name = "gpt-4o-mini"
api_key_location = "none"

# anthropic -> POST {api_base}messages  (api_base is the API ROOT; the gateway appends \`messages\`).
# api_key_location is left at its default env::ANTHROPIC_API_KEY: the anthropic provider REJECTS
# "none" (providers/anthropic.rs:189-208 has no Credential::None arm), and credentials are validated
# at boot, so the dummy ANTHROPIC_API_KEY in gw_launch is what lets this model exist in the config.
[models.mock-anthropic]
routing = ["mock"]
[models.mock-anthropic.providers.mock]
type = "anthropic"
api_base = "http://127.0.0.1:8000/v1/"
model_name = "claude-3-5-sonnet-20241022"

# google_ai_studio_gemini -> POST https://generativelanguage.googleapis.com/v1beta/models/{model_name}:generateContent
# The provider has no api_base override, so it keeps its default host; engine v1.2's catch-all DNS
# resolves that host to the mock and the gateway trusts the run's CA. The key is the dummy
# GOOGLE_AI_STUDIO_API_KEY in env (the provider's default api_key_location).
[models.mock-gemini]
routing = ["mock"]
[models.mock-gemini.providers.mock]
type = "google_ai_studio_gemini"
model_name = "gemini-1.5-pro"

# aws_bedrock -> POST {endpoint_url}/model/{urlencoded model_id}/converse (SigV4-signed with the
# dummy AWS keys in gw_launch; the mock ignores the signature).
[models.mock-bedrock]
routing = ["mock"]
[models.mock-bedrock.providers.mock]
type = "aws_bedrock"
endpoint_url = "http://127.0.0.1:8000"
model_id = "anthropic.claude-3-sonnet-20240229-v1:0"
region = "us-east-1"

Compare a shared protocol cell

Historical measurements