Busbar benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Rust · recorded build: getbusbar/busbar:1.5.5. Measured: .

JSON · Markdown · CSV · Project source · Interactive dashboard

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

Published measurements

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
Cohere → AnthropicAdded latency p50 (µs)99Valid · n=49,012 gateway / 264,547 direct—
Cohere → AnthropicAdded latency p99 (µs)111Valid · n=49,012 gateway / 264,547 direct—
Cohere → AnthropicGateway p99 at concurrency 1 (µs)139Valid · n=49,012—
Cohere → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=264,547—
Cohere → AnthropicCPU per request (µs)88.93Valid—
Cohere → AnthropicRequests per CPU-second11,245.2Valid—
Cohere → AnthropicCost window concurrency8Valid—
Cohere → AnthropicCost window core utilisation0.967Valid—
Cohere → AnthropicCost window successful requests261,900Valid—
Cohere → AnthropicCost window throughput (req/s)43,646Valid—
Cohere → AnthropicCost window threads10Valid—
Cohere → AnthropicNon-voluntary context switches per request0Valid—
Cohere → AnthropicCost window major page faults0Valid—
Cohere → AnthropicAdded time to first event p50 (µs)—Untestable—
Cohere → AnthropicAdded time to first event p99 (µs)—Untestable—
Cohere → AnthropicAdded inter-frame gap p50 (µs)—Untestable—
Cohere → AnthropicAdded inter-frame gap p99 (µs)—Untestable—
Cohere → AnthropicStreams sustained (frames/s)—Untestable—
Cohere → AnthropicStreams sustained—Untestable—
Cohere → AnthropicIdle RSS (MiB)7.43Valid—
Cohere → AnthropicSteady-state RSS (MiB)24.29Valid—
Cohere → AnthropicRecovered RSS (MiB)16.4Valid—
Cohere → AnthropicPeak RSS (MiB)24.69Valid—
Cohere → AnthropicPeak RSS high-water mark (MiB)24.31Valid—
Cohere → AnthropicTime to plateau (s)—Not measured—
Cohere → AnthropicMemory growth (MiB/min)0.523Valid—
Cohere → AnthropicMemory load duration (s)300Valid—
Cohere → AnthropicMemory shape under load1Valid—
Cohere → AnthropicIdle memory shape0Valid—
Cohere → AnthropicIdle memory static0Valid—
Cohere → AnthropicIdle memory growth (MiB/min)0.004Valid—
Cohere → AnthropicThroughput (req/s)46,645concurrency 16; n=279,9031 ms
Cohere → AnthropicThroughput (req/s)47,855concurrency 128; n=287,1745 ms
Cohere → AnthropicThroughput (req/s)47,855concurrency 128; n=287,17410 ms
Cohere → AnthropicThroughput (req/s)47,855concurrency 128; n=287,17450 ms
Cohere → AnthropicThroughput (req/s)47,855concurrency 128; n=287,174100 ms
Cohere → AnthropicThroughput (req/s)47,855concurrency 128; n=287,174Unbounded
OpenAI → AnthropicAdded latency p50 (µs)101Valid · n=47,933 gateway / 264,624 direct—
OpenAI → AnthropicAdded latency p99 (µs)115Valid · n=47,933 gateway / 264,624 direct—
OpenAI → AnthropicGateway p99 at concurrency 1 (µs)143Valid · n=47,933—
OpenAI → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=264,624—
OpenAI → AnthropicCPU per request (µs)89.26Valid—
OpenAI → AnthropicRequests per CPU-second11,203.8Valid—
OpenAI → AnthropicCost window concurrency8Valid—
OpenAI → AnthropicCost window core utilisation0.963Valid—
OpenAI → AnthropicCost window successful requests259,816Valid—
OpenAI → AnthropicCost window throughput (req/s)43,297Valid—
OpenAI → AnthropicCost window threads10Valid—
OpenAI → AnthropicNon-voluntary context switches per request0Valid—
OpenAI → AnthropicCost window major page faults0Valid—
OpenAI → AnthropicAdded time to first event p50 (µs)154Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded time to first event p99 (µs)248Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded inter-frame gap p50 (µs)1Valid · n=63 gateway / 63 direct—
OpenAI → AnthropicAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → AnthropicStreams sustained (frames/s)41,099.6Valid—
OpenAI → AnthropicStreams sustained1,536Valid—
OpenAI → AnthropicIdle RSS (MiB)7.43Valid—
OpenAI → AnthropicSteady-state RSS (MiB)20.11Valid—
OpenAI → AnthropicRecovered RSS (MiB)15.37Valid—
OpenAI → AnthropicPeak RSS (MiB)20.68Valid—
OpenAI → AnthropicPeak RSS high-water mark (MiB)20.47Valid—
OpenAI → AnthropicTime to plateau (s)—Not measured—
OpenAI → AnthropicMemory growth (MiB/min)-0.144Valid—
OpenAI → AnthropicMemory load duration (s)300Valid—
OpenAI → AnthropicMemory shape under load0Valid—
OpenAI → AnthropicIdle memory shape0Valid—
OpenAI → AnthropicIdle memory static0Valid—
OpenAI → AnthropicIdle memory growth (MiB/min)0.004Valid—
OpenAI → AnthropicThroughput (req/s)46,006concurrency 16; n=276,0611 ms
OpenAI → AnthropicThroughput (req/s)47,478concurrency 128; n=284,9115 ms
OpenAI → AnthropicThroughput (req/s)47,478concurrency 128; n=284,91110 ms
OpenAI → AnthropicThroughput (req/s)47,478concurrency 128; n=284,91150 ms
OpenAI → AnthropicThroughput (req/s)47,478concurrency 128; n=284,911100 ms
OpenAI → AnthropicThroughput (req/s)47,478concurrency 128; n=284,911Unbounded
OpenAI Responses → AnthropicAdded latency p50 (µs)150Valid · n=34,460 gateway / 264,688 direct—
OpenAI Responses → AnthropicAdded latency p99 (µs)166Valid · n=34,460 gateway / 264,688 direct—
OpenAI Responses → AnthropicGateway p99 at concurrency 1 (µs)193Valid · n=34,460—
OpenAI Responses → AnthropicDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=264,688—
OpenAI Responses → AnthropicCPU per request (µs)136.4Valid—
OpenAI Responses → AnthropicRequests per CPU-second7,329.7Valid—
OpenAI Responses → AnthropicCost window concurrency8Valid—
OpenAI Responses → AnthropicCost window core utilisation0.98Valid—
OpenAI Responses → AnthropicCost window successful requests173,055Valid—
OpenAI Responses → AnthropicCost window throughput (req/s)28,839Valid—
OpenAI Responses → AnthropicCost window threads10Valid—
OpenAI Responses → AnthropicNon-voluntary context switches per request0Valid—
OpenAI Responses → AnthropicCost window major page faults0Valid—
OpenAI Responses → AnthropicAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → AnthropicAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → AnthropicAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → AnthropicAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → AnthropicStreams sustained (frames/s)—Untestable—
OpenAI Responses → AnthropicStreams sustained—Untestable—
OpenAI Responses → AnthropicIdle RSS (MiB)7.45Valid—
OpenAI Responses → AnthropicSteady-state RSS (MiB)20.87Valid—
OpenAI Responses → AnthropicRecovered RSS (MiB)15.52Valid—
OpenAI Responses → AnthropicPeak RSS (MiB)21.23Valid—
OpenAI Responses → AnthropicPeak RSS high-water mark (MiB)20.79Valid—
OpenAI Responses → AnthropicTime to plateau (s)—Not measured—
OpenAI Responses → AnthropicMemory growth (MiB/min)-0.131Valid—
OpenAI Responses → AnthropicMemory load duration (s)300Valid—
OpenAI Responses → AnthropicMemory shape under load0Valid—
OpenAI Responses → AnthropicIdle memory shape0Valid—
OpenAI Responses → AnthropicIdle memory static0Valid—
OpenAI Responses → AnthropicIdle memory growth (MiB/min)0.004Valid—
OpenAI Responses → AnthropicThroughput (req/s)29,647concurrency 16; n=177,9011 ms
OpenAI Responses → AnthropicThroughput (req/s)29,773concurrency 32; n=178,6575 ms
OpenAI Responses → AnthropicThroughput (req/s)29,773concurrency 32; n=178,65710 ms
OpenAI Responses → AnthropicThroughput (req/s)29,773concurrency 32; n=178,65750 ms
OpenAI Responses → AnthropicThroughput (req/s)29,773concurrency 32; n=178,657100 ms
OpenAI Responses → AnthropicThroughput (req/s)29,773concurrency 32; n=178,657Unbounded
Anthropic → AnthropicAdded latency p50 (µs)85Valid · n=55,235 gateway / 261,103 direct—
Anthropic → AnthropicAdded latency p99 (µs)96Valid · n=55,235 gateway / 261,103 direct—
Anthropic → AnthropicGateway p99 at concurrency 1 (µs)124Valid · n=55,235—
Anthropic → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=261,103—
Anthropic → AnthropicCPU per request (µs)74.12Valid—
Anthropic → AnthropicRequests per CPU-second13,491.6Valid—
Anthropic → AnthropicCost window concurrency8Valid—
Anthropic → AnthropicCost window core utilisation0.957Valid—
Anthropic → AnthropicCost window successful requests310,846Valid—
Anthropic → AnthropicCost window throughput (req/s)51,801Valid—
Anthropic → AnthropicCost window threads10Valid—
Anthropic → AnthropicNon-voluntary context switches per request0Valid—
Anthropic → AnthropicCost window major page faults0Valid—
Anthropic → AnthropicAdded time to first event p50 (µs)125Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded time to first event p99 (µs)229Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded inter-frame gap p50 (µs)2Valid · n=63 gateway / 63 direct—
Anthropic → AnthropicAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → AnthropicStreams sustained (frames/s)43,414.8Valid—
Anthropic → AnthropicStreams sustained1,698Valid—
Anthropic → AnthropicIdle RSS (MiB)7.44Valid—
Anthropic → AnthropicSteady-state RSS (MiB)22.21Valid—
Anthropic → AnthropicRecovered RSS (MiB)16.39Valid—
Anthropic → AnthropicPeak RSS (MiB)23.27Valid—
Anthropic → AnthropicPeak RSS high-water mark (MiB)22.98Valid—
Anthropic → AnthropicTime to plateau (s)—Not measured—
Anthropic → AnthropicMemory growth (MiB/min)0.584Valid—
Anthropic → AnthropicMemory load duration (s)300Valid—
Anthropic → AnthropicMemory shape under load1Valid—
Anthropic → AnthropicIdle memory shape0Valid—
Anthropic → AnthropicIdle memory static0Valid—
Anthropic → AnthropicIdle memory growth (MiB/min)0.004Valid—
Anthropic → AnthropicThroughput (req/s)57,786concurrency 32; n=346,7491 ms
Anthropic → AnthropicThroughput (req/s)58,879concurrency 128; n=353,3235 ms
Anthropic → AnthropicThroughput (req/s)58,879concurrency 128; n=353,32310 ms
Anthropic → AnthropicThroughput (req/s)58,879concurrency 128; n=353,32350 ms
Anthropic → AnthropicThroughput (req/s)58,879concurrency 128; n=353,323100 ms
Anthropic → AnthropicThroughput (req/s)58,879concurrency 128; n=353,323Unbounded
Gemini → AnthropicAdded latency p50 (µs)105Valid · n=46,775 gateway / 263,231 direct—
Gemini → AnthropicAdded latency p99 (µs)119Valid · n=46,775 gateway / 263,231 direct—
Gemini → AnthropicGateway p99 at concurrency 1 (µs)146Valid · n=46,775—
Gemini → AnthropicDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=263,231—
Gemini → AnthropicCPU per request (µs)93.11Valid—
Gemini → AnthropicRequests per CPU-second10,739.9Valid—
Gemini → AnthropicCost window concurrency8Valid—
Gemini → AnthropicCost window core utilisation0.967Valid—
Gemini → AnthropicCost window successful requests250,132Valid—
Gemini → AnthropicCost window throughput (req/s)41,684Valid—
Gemini → AnthropicCost window threads10Valid—
Gemini → AnthropicNon-voluntary context switches per request0Valid—
Gemini → AnthropicCost window major page faults0Valid—
Gemini → AnthropicAdded time to first event p50 (µs)—Untestable—
Gemini → AnthropicAdded time to first event p99 (µs)—Untestable—
Gemini → AnthropicAdded inter-frame gap p50 (µs)—Untestable—
Gemini → AnthropicAdded inter-frame gap p99 (µs)—Untestable—
Gemini → AnthropicStreams sustained (frames/s)—Untestable—
Gemini → AnthropicStreams sustained—Untestable—
Gemini → AnthropicIdle RSS (MiB)7.43Valid—
Gemini → AnthropicSteady-state RSS (MiB)22.05Valid—
Gemini → AnthropicRecovered RSS (MiB)16.32Valid—
Gemini → AnthropicPeak RSS (MiB)22.44Valid—
Gemini → AnthropicPeak RSS high-water mark (MiB)22.28Valid—
Gemini → AnthropicTime to plateau (s)—Not measured—
Gemini → AnthropicMemory growth (MiB/min)0.159Valid—
Gemini → AnthropicMemory load duration (s)300Valid—
Gemini → AnthropicMemory shape under load0Valid—
Gemini → AnthropicIdle memory shape0Valid—
Gemini → AnthropicIdle memory static0Valid—
Gemini → AnthropicIdle memory growth (MiB/min)0.004Valid—
Gemini → AnthropicThroughput (req/s)44,397concurrency 16; n=266,4171 ms
Gemini → AnthropicThroughput (req/s)45,198concurrency 32; n=271,2125 ms
Gemini → AnthropicThroughput (req/s)45,198concurrency 32; n=271,21210 ms
Gemini → AnthropicThroughput (req/s)45,198concurrency 32; n=271,21250 ms
Gemini → AnthropicThroughput (req/s)45,198concurrency 32; n=271,212100 ms
Gemini → AnthropicThroughput (req/s)45,198concurrency 32; n=271,212Unbounded
Bedrock → AnthropicAdded latency p50 (µs)106Valid · n=46,122 gateway / 266,395 direct—
Bedrock → AnthropicAdded latency p99 (µs)120Valid · n=46,122 gateway / 266,395 direct—
Bedrock → AnthropicGateway p99 at concurrency 1 (µs)147Valid · n=46,122—
Bedrock → AnthropicDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=266,395—
Bedrock → AnthropicCPU per request (µs)93.72Valid—
Bedrock → AnthropicRequests per CPU-second10,670.4Valid—
Bedrock → AnthropicCost window concurrency8Valid—
Bedrock → AnthropicCost window core utilisation0.965Valid—
Bedrock → AnthropicCost window successful requests248,087Valid—
Bedrock → AnthropicCost window throughput (req/s)41,343Valid—
Bedrock → AnthropicCost window threads10Valid—
Bedrock → AnthropicNon-voluntary context switches per request0Valid—
Bedrock → AnthropicCost window major page faults0Valid—
Bedrock → AnthropicAdded time to first event p50 (µs)—Untestable—
Bedrock → AnthropicAdded time to first event p99 (µs)—Untestable—
Bedrock → AnthropicAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → AnthropicAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → AnthropicStreams sustained (frames/s)—Untestable—
Bedrock → AnthropicStreams sustained—Untestable—
Bedrock → AnthropicIdle RSS (MiB)7.44Valid—
Bedrock → AnthropicSteady-state RSS (MiB)26.27Valid—
Bedrock → AnthropicRecovered RSS (MiB)16.05Valid—
Bedrock → AnthropicPeak RSS (MiB)26.76Valid—
Bedrock → AnthropicPeak RSS high-water mark (MiB)26.53Valid—
Bedrock → AnthropicTime to plateau (s)—Not measured—
Bedrock → AnthropicMemory growth (MiB/min)0.442Valid—
Bedrock → AnthropicMemory load duration (s)300Valid—
Bedrock → AnthropicMemory shape under load0Valid—
Bedrock → AnthropicIdle memory shape0Valid—
Bedrock → AnthropicIdle memory static0Valid—
Bedrock → AnthropicIdle memory growth (MiB/min)0.004Valid—
Bedrock → AnthropicThroughput (req/s)43,964concurrency 16; n=263,8111 ms
Bedrock → AnthropicThroughput (req/s)45,086concurrency 64; n=270,5455 ms
Bedrock → AnthropicThroughput (req/s)45,086concurrency 64; n=270,54510 ms
Bedrock → AnthropicThroughput (req/s)45,086concurrency 64; n=270,54550 ms
Bedrock → AnthropicThroughput (req/s)45,086concurrency 64; n=270,545100 ms
Bedrock → AnthropicThroughput (req/s)45,086concurrency 64; n=270,545Unbounded
OpenAI Responses → BedrockAdded latency p50 (µs)171Valid · n=30,814 gateway / 264,867 direct—
OpenAI Responses → BedrockAdded latency p99 (µs)192Valid · n=30,814 gateway / 264,867 direct—
OpenAI Responses → BedrockGateway p99 at concurrency 1 (µs)219Valid · n=30,814—
OpenAI Responses → BedrockDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=264,867—
OpenAI Responses → BedrockCPU per request (µs)157.1Valid—
OpenAI Responses → BedrockRequests per CPU-second6,363.6Valid—
OpenAI Responses → BedrockCost window concurrency8Valid—
OpenAI Responses → BedrockCost window core utilisation0.979Valid—
OpenAI Responses → BedrockCost window successful requests149,989Valid—
OpenAI Responses → BedrockCost window throughput (req/s)24,995Valid—
OpenAI Responses → BedrockCost window threads10Valid—
OpenAI Responses → BedrockNon-voluntary context switches per request0Valid—
OpenAI Responses → BedrockCost window major page faults0Valid—
OpenAI Responses → BedrockAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → BedrockAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → BedrockAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → BedrockAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → BedrockStreams sustained (frames/s)—Untestable—
OpenAI Responses → BedrockStreams sustained—Untestable—
OpenAI Responses → BedrockIdle RSS (MiB)7.44Valid—
OpenAI Responses → BedrockSteady-state RSS (MiB)23.22Valid—
OpenAI Responses → BedrockRecovered RSS (MiB)14.27Valid—
OpenAI Responses → BedrockPeak RSS (MiB)23.45Valid—
OpenAI Responses → BedrockPeak RSS high-water mark (MiB)23.23Valid—
OpenAI Responses → BedrockTime to plateau (s)300Valid—
OpenAI Responses → BedrockMemory growth (MiB/min)0.222Valid—
OpenAI Responses → BedrockMemory load duration (s)300Valid—
OpenAI Responses → BedrockMemory shape under load—Not measured—
OpenAI Responses → BedrockIdle memory shape0Valid—
OpenAI Responses → BedrockIdle memory static0Valid—
OpenAI Responses → BedrockIdle memory growth (MiB/min)0.004Valid—
OpenAI Responses → BedrockThroughput (req/s)25,676concurrency 16; n=154,0741 ms
OpenAI Responses → BedrockThroughput (req/s)25,676concurrency 16; n=154,0745 ms
OpenAI Responses → BedrockThroughput (req/s)25,676concurrency 16; n=154,07410 ms
OpenAI Responses → BedrockThroughput (req/s)25,676concurrency 16; n=154,07450 ms
OpenAI Responses → BedrockThroughput (req/s)25,676concurrency 16; n=154,074100 ms
OpenAI Responses → BedrockThroughput (req/s)25,676concurrency 16; n=154,074Unbounded
Gemini → BedrockAdded latency p50 (µs)129Valid · n=39,387 gateway / 270,577 direct—
Gemini → BedrockAdded latency p99 (µs)148Valid · n=39,387 gateway / 270,577 direct—
Gemini → BedrockGateway p99 at concurrency 1 (µs)175Valid · n=39,387—
Gemini → BedrockDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=270,577—
Gemini → BedrockCPU per request (µs)115.7Valid—
Gemini → BedrockRequests per CPU-second8,644.8Valid—
Gemini → BedrockCost window concurrency8Valid—
Gemini → BedrockCost window core utilisation0.969Valid—
Gemini → BedrockCost window successful requests201,769Valid—
Gemini → BedrockCost window throughput (req/s)33,624Valid—
Gemini → BedrockCost window threads10Valid—
Gemini → BedrockNon-voluntary context switches per request0Valid—
Gemini → BedrockCost window major page faults0Valid—
Gemini → BedrockAdded time to first event p50 (µs)—Untestable—
Gemini → BedrockAdded time to first event p99 (µs)—Untestable—
Gemini → BedrockAdded inter-frame gap p50 (µs)—Untestable—
Gemini → BedrockAdded inter-frame gap p99 (µs)—Untestable—
Gemini → BedrockStreams sustained (frames/s)—Untestable—
Gemini → BedrockStreams sustained—Untestable—
Gemini → BedrockIdle RSS (MiB)7.45Valid—
Gemini → BedrockSteady-state RSS (MiB)21.81Valid—
Gemini → BedrockRecovered RSS (MiB)15.04Valid—
Gemini → BedrockPeak RSS (MiB)22.15Valid—
Gemini → BedrockPeak RSS high-water mark (MiB)21.81Valid—
Gemini → BedrockTime to plateau (s)—Not measured—
Gemini → BedrockMemory growth (MiB/min)0.346Valid—
Gemini → BedrockMemory load duration (s)300Valid—
Gemini → BedrockMemory shape under load0Valid—
Gemini → BedrockIdle memory shape0Valid—
Gemini → BedrockIdle memory static0Valid—
Gemini → BedrockIdle memory growth (MiB/min)0.004Valid—
Gemini → BedrockThroughput (req/s)34,935concurrency 16; n=209,6411 ms
Gemini → BedrockThroughput (req/s)35,197concurrency 64; n=211,1905 ms
Gemini → BedrockThroughput (req/s)35,197concurrency 64; n=211,19010 ms
Gemini → BedrockThroughput (req/s)35,197concurrency 64; n=211,19050 ms
Gemini → BedrockThroughput (req/s)35,197concurrency 64; n=211,190100 ms
Gemini → BedrockThroughput (req/s)35,197concurrency 64; n=211,190Unbounded
Cohere → BedrockAdded latency p50 (µs)120Valid · n=41,840 gateway / 268,504 direct—
Cohere → BedrockAdded latency p99 (µs)136Valid · n=41,840 gateway / 268,504 direct—
Cohere → BedrockGateway p99 at concurrency 1 (µs)163Valid · n=41,840—
Cohere → BedrockDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=268,504—
Cohere → BedrockCPU per request (µs)109.9Valid—
Cohere → BedrockRequests per CPU-second9,099.7Valid—
Cohere → BedrockCost window concurrency8Valid—
Cohere → BedrockCost window core utilisation0.968Valid—
Cohere → BedrockCost window successful requests212,113Valid—
Cohere → BedrockCost window throughput (req/s)35,348Valid—
Cohere → BedrockCost window threads10Valid—
Cohere → BedrockNon-voluntary context switches per request0Valid—
Cohere → BedrockCost window major page faults0Valid—
Cohere → BedrockAdded time to first event p50 (µs)—Untestable—
Cohere → BedrockAdded time to first event p99 (µs)—Untestable—
Cohere → BedrockAdded inter-frame gap p50 (µs)—Untestable—
Cohere → BedrockAdded inter-frame gap p99 (µs)—Untestable—
Cohere → BedrockStreams sustained (frames/s)—Untestable—
Cohere → BedrockStreams sustained—Untestable—
Cohere → BedrockIdle RSS (MiB)7.42Valid—
Cohere → BedrockSteady-state RSS (MiB)20.3Valid—
Cohere → BedrockRecovered RSS (MiB)15.18Valid—
Cohere → BedrockPeak RSS (MiB)20.46Valid—
Cohere → BedrockPeak RSS high-water mark (MiB)20.18Valid—
Cohere → BedrockTime to plateau (s)—Not measured—
Cohere → BedrockMemory growth (MiB/min)0.344Valid—
Cohere → BedrockMemory load duration (s)300Valid—
Cohere → BedrockMemory shape under load0Valid—
Cohere → BedrockIdle memory shape0Valid—
Cohere → BedrockIdle memory static0Valid—
Cohere → BedrockIdle memory growth (MiB/min)0.004Valid—
Cohere → BedrockThroughput (req/s)36,699concurrency 16; n=220,2171 ms
Cohere → BedrockThroughput (req/s)36,731concurrency 32; n=220,4095 ms
Cohere → BedrockThroughput (req/s)36,731concurrency 32; n=220,40910 ms
Cohere → BedrockThroughput (req/s)36,731concurrency 32; n=220,40950 ms
Cohere → BedrockThroughput (req/s)36,731concurrency 32; n=220,409100 ms
Cohere → BedrockThroughput (req/s)36,731concurrency 32; n=220,409Unbounded
Bedrock → BedrockAdded latency p50 (µs)110Valid · n=44,888 gateway / 266,888 direct—
Bedrock → BedrockAdded latency p99 (µs)127Valid · n=44,888 gateway / 266,888 direct—
Bedrock → BedrockGateway p99 at concurrency 1 (µs)154Valid · n=44,888—
Bedrock → BedrockDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=266,888—
Bedrock → BedrockCPU per request (µs)101.1Valid—
Bedrock → BedrockRequests per CPU-second9,893.4Valid—
Bedrock → BedrockCost window concurrency8Valid—
Bedrock → BedrockCost window core utilisation0.966Valid—
Bedrock → BedrockCost window successful requests230,120Valid—
Bedrock → BedrockCost window throughput (req/s)38,348Valid—
Bedrock → BedrockCost window threads10Valid—
Bedrock → BedrockNon-voluntary context switches per request0Valid—
Bedrock → BedrockCost window major page faults0Valid—
Bedrock → BedrockAdded time to first event p50 (µs)—Untestable—
Bedrock → BedrockAdded time to first event p99 (µs)—Untestable—
Bedrock → BedrockAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → BedrockAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → BedrockStreams sustained (frames/s)—Untestable—
Bedrock → BedrockStreams sustained—Untestable—
Bedrock → BedrockIdle RSS (MiB)7.42Valid—
Bedrock → BedrockSteady-state RSS (MiB)20.46Valid—
Bedrock → BedrockRecovered RSS (MiB)15.18Valid—
Bedrock → BedrockPeak RSS (MiB)20.75Valid—
Bedrock → BedrockPeak RSS high-water mark (MiB)20.65Valid—
Bedrock → BedrockTime to plateau (s)—Not measured—
Bedrock → BedrockMemory growth (MiB/min)-0.062Valid—
Bedrock → BedrockMemory load duration (s)300Valid—
Bedrock → BedrockMemory shape under load0Valid—
Bedrock → BedrockIdle memory shape0Valid—
Bedrock → BedrockIdle memory static0Valid—
Bedrock → BedrockIdle memory growth (MiB/min)0.004Valid—
Bedrock → BedrockThroughput (req/s)39,956concurrency 16; n=239,7621 ms
Bedrock → BedrockThroughput (req/s)40,107concurrency 32; n=240,6665 ms
Bedrock → BedrockThroughput (req/s)40,107concurrency 32; n=240,66610 ms
Bedrock → BedrockThroughput (req/s)40,107concurrency 32; n=240,66650 ms
Bedrock → BedrockThroughput (req/s)40,107concurrency 32; n=240,666100 ms
Bedrock → BedrockThroughput (req/s)40,107concurrency 32; n=240,666Unbounded
OpenAI → BedrockAdded latency p50 (µs)125Valid · n=40,371 gateway / 265,424 direct—
OpenAI → BedrockAdded latency p99 (µs)142Valid · n=40,371 gateway / 265,424 direct—
OpenAI → BedrockGateway p99 at concurrency 1 (µs)170Valid · n=40,371—
OpenAI → BedrockDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=265,424—
OpenAI → BedrockCPU per request (µs)111.4Valid—
OpenAI → BedrockRequests per CPU-second8,978.8Valid—
OpenAI → BedrockCost window concurrency8Valid—
OpenAI → BedrockCost window core utilisation0.971Valid—
OpenAI → BedrockCost window successful requests209,835Valid—
OpenAI → BedrockCost window throughput (req/s)34,969Valid—
OpenAI → BedrockCost window threads10Valid—
OpenAI → BedrockNon-voluntary context switches per request0Valid—
OpenAI → BedrockCost window major page faults0Valid—
OpenAI → BedrockAdded time to first event p50 (µs)—Untestable—
OpenAI → BedrockAdded time to first event p99 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → BedrockStreams sustained (frames/s)—Untestable—
OpenAI → BedrockStreams sustained—Untestable—
OpenAI → BedrockIdle RSS (MiB)7.43Valid—
OpenAI → BedrockSteady-state RSS (MiB)19.93Valid—
OpenAI → BedrockRecovered RSS (MiB)15.18Valid—
OpenAI → BedrockPeak RSS (MiB)20.37Valid—
OpenAI → BedrockPeak RSS high-water mark (MiB)19.97Valid—
OpenAI → BedrockTime to plateau (s)—Not measured—
OpenAI → BedrockMemory growth (MiB/min)-0.073Valid—
OpenAI → BedrockMemory load duration (s)300Valid—
OpenAI → BedrockMemory shape under load0Valid—
OpenAI → BedrockIdle memory shape0Valid—
OpenAI → BedrockIdle memory static0Valid—
OpenAI → BedrockIdle memory growth (MiB/min)0.004Valid—
OpenAI → BedrockThroughput (req/s)36,468concurrency 16; n=218,8341 ms
OpenAI → BedrockThroughput (req/s)36,523concurrency 32; n=219,1575 ms
OpenAI → BedrockThroughput (req/s)36,523concurrency 32; n=219,15710 ms
OpenAI → BedrockThroughput (req/s)36,523concurrency 32; n=219,15750 ms
OpenAI → BedrockThroughput (req/s)36,523concurrency 32; n=219,157100 ms
OpenAI → BedrockThroughput (req/s)36,523concurrency 32; n=219,157Unbounded
Anthropic → BedrockAdded latency p50 (µs)123Valid · n=40,751 gateway / 263,427 direct—
Anthropic → BedrockAdded latency p99 (µs)140Valid · n=40,751 gateway / 263,427 direct—
Anthropic → BedrockGateway p99 at concurrency 1 (µs)168Valid · n=40,751—
Anthropic → BedrockDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=263,427—
Anthropic → BedrockCPU per request (µs)111.5Valid—
Anthropic → BedrockRequests per CPU-second8,967.3Valid—
Anthropic → BedrockCost window concurrency8Valid—
Anthropic → BedrockCost window core utilisation0.969Valid—
Anthropic → BedrockCost window successful requests209,296Valid—
Anthropic → BedrockCost window throughput (req/s)34,878Valid—
Anthropic → BedrockCost window threads10Valid—
Anthropic → BedrockNon-voluntary context switches per request0Valid—
Anthropic → BedrockCost window major page faults0Valid—
Anthropic → BedrockAdded time to first event p50 (µs)—Untestable—
Anthropic → BedrockAdded time to first event p99 (µs)—Untestable—
Anthropic → BedrockAdded inter-frame gap p50 (µs)—Untestable—
Anthropic → BedrockAdded inter-frame gap p99 (µs)—Untestable—
Anthropic → BedrockStreams sustained (frames/s)—Untestable—
Anthropic → BedrockStreams sustained—Untestable—
Anthropic → BedrockIdle RSS (MiB)7.42Valid—
Anthropic → BedrockSteady-state RSS (MiB)20.27Valid—
Anthropic → BedrockRecovered RSS (MiB)15.97Valid—
Anthropic → BedrockPeak RSS (MiB)21.07Valid—
Anthropic → BedrockPeak RSS high-water mark (MiB)21.01Valid—
Anthropic → BedrockTime to plateau (s)—Not measured—
Anthropic → BedrockMemory growth (MiB/min)0.559Valid—
Anthropic → BedrockMemory load duration (s)300Valid—
Anthropic → BedrockMemory shape under load1Valid—
Anthropic → BedrockIdle memory shape0Valid—
Anthropic → BedrockIdle memory static0Valid—
Anthropic → BedrockIdle memory growth (MiB/min)0.004Valid—
Anthropic → BedrockThroughput (req/s)36,181concurrency 16; n=217,1031 ms
Anthropic → BedrockThroughput (req/s)36,453concurrency 64; n=218,7315 ms
Anthropic → BedrockThroughput (req/s)36,453concurrency 64; n=218,73110 ms
Anthropic → BedrockThroughput (req/s)36,453concurrency 64; n=218,73150 ms
Anthropic → BedrockThroughput (req/s)36,453concurrency 64; n=218,731100 ms
Anthropic → BedrockThroughput (req/s)36,453concurrency 64; n=218,731Unbounded
OpenAI → OpenAIAdded latency p50 (µs)78Valid · n=59,266 gateway / 266,330 direct—
OpenAI → OpenAIAdded latency p99 (µs)89Valid · n=59,266 gateway / 266,330 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)117Valid · n=59,266—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=266,330—
OpenAI → OpenAICPU per request (µs)68.74Valid—
OpenAI → OpenAIRequests per CPU-second14,547.2Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.957Valid—
OpenAI → OpenAICost window successful requests335,167Valid—
OpenAI → OpenAICost window throughput (req/s)55,854Valid—
OpenAI → OpenAICost window threads10Valid—
OpenAI → OpenAINon-voluntary context switches per request0Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)118Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)254Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)39,240.2Valid—
OpenAI → OpenAIStreams sustained1,452Valid—
OpenAI → OpenAIIdle RSS (MiB)7.43Valid—
OpenAI → OpenAISteady-state RSS (MiB)21.49Valid—
OpenAI → OpenAIRecovered RSS (MiB)15.7Valid—
OpenAI → OpenAIPeak RSS (MiB)22.35Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)22.27Valid—
OpenAI → OpenAITime to plateau (s)—Not measured—
OpenAI → OpenAIMemory growth (MiB/min)0.391Valid—
OpenAI → OpenAIMemory load duration (s)300Valid—
OpenAI → OpenAIMemory shape under load1Valid—
OpenAI → OpenAIIdle memory shape0Valid—
OpenAI → OpenAIIdle memory static0Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)0.004Valid—
OpenAI → OpenAIThroughput (req/s)62,231concurrency 32; n=373,4171 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,5785 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,57810 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,57850 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,578100 ms
OpenAI → OpenAIThroughput (req/s)62,587concurrency 128; n=375,578Unbounded
Cohere → OpenAIAdded latency p50 (µs)98Valid · n=49,364 gateway / 265,547 direct—
Cohere → OpenAIAdded latency p99 (µs)111Valid · n=49,364 gateway / 265,547 direct—
Cohere → OpenAIGateway p99 at concurrency 1 (µs)138Valid · n=49,364—
Cohere → OpenAIDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=265,547—
Cohere → OpenAICPU per request (µs)86.9Valid—
Cohere → OpenAIRequests per CPU-second11,507.9Valid—
Cohere → OpenAICost window concurrency8Valid—
Cohere → OpenAICost window core utilisation0.963Valid—
Cohere → OpenAICost window successful requests266,754Valid—
Cohere → OpenAICost window throughput (req/s)44,454Valid—
Cohere → OpenAICost window threads10Valid—
Cohere → OpenAINon-voluntary context switches per request0Valid—
Cohere → OpenAICost window major page faults0Valid—
Cohere → OpenAIAdded time to first event p50 (µs)—Untestable—
Cohere → OpenAIAdded time to first event p99 (µs)—Untestable—
Cohere → OpenAIAdded inter-frame gap p50 (µs)—Untestable—
Cohere → OpenAIAdded inter-frame gap p99 (µs)—Untestable—
Cohere → OpenAIStreams sustained (frames/s)—Untestable—
Cohere → OpenAIStreams sustained—Untestable—
Cohere → OpenAIIdle RSS (MiB)7.43Valid—
Cohere → OpenAISteady-state RSS (MiB)20.19Valid—
Cohere → OpenAIRecovered RSS (MiB)15.3Valid—
Cohere → OpenAIPeak RSS (MiB)20.41Valid—
Cohere → OpenAIPeak RSS high-water mark (MiB)20.15Valid—
Cohere → OpenAITime to plateau (s)276Valid—
Cohere → OpenAIMemory growth (MiB/min)-0.038Valid—
Cohere → OpenAIMemory load duration (s)300Valid—
Cohere → OpenAIMemory shape under load—Not measured—
Cohere → OpenAIIdle memory shape0Valid—
Cohere → OpenAIIdle memory static0Valid—
Cohere → OpenAIIdle memory growth (MiB/min)0.004Valid—
Cohere → OpenAIThroughput (req/s)47,293concurrency 16; n=283,7881 ms
Cohere → OpenAIThroughput (req/s)48,545concurrency 128; n=291,3075 ms
Cohere → OpenAIThroughput (req/s)48,545concurrency 128; n=291,30710 ms
Cohere → OpenAIThroughput (req/s)48,545concurrency 128; n=291,30750 ms
Cohere → OpenAIThroughput (req/s)48,545concurrency 128; n=291,307100 ms
Cohere → OpenAIThroughput (req/s)48,545concurrency 128; n=291,307Unbounded
Anthropic → OpenAIAdded latency p50 (µs)100Valid · n=48,328 gateway / 264,163 direct—
Anthropic → OpenAIAdded latency p99 (µs)113Valid · n=48,328 gateway / 264,163 direct—
Anthropic → OpenAIGateway p99 at concurrency 1 (µs)141Valid · n=48,328—
Anthropic → OpenAIDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=264,163—
Anthropic → OpenAICPU per request (µs)88.98Valid—
Anthropic → OpenAIRequests per CPU-second11,237.9Valid—
Anthropic → OpenAICost window concurrency8Valid—
Anthropic → OpenAICost window core utilisation0.963Valid—
Anthropic → OpenAICost window successful requests260,495Valid—
Anthropic → OpenAICost window throughput (req/s)43,410Valid—
Anthropic → OpenAICost window threads10Valid—
Anthropic → OpenAINon-voluntary context switches per request0Valid—
Anthropic → OpenAICost window major page faults0Valid—
Anthropic → OpenAIAdded time to first event p50 (µs)118Valid · n=100 gateway / 100 direct—
Anthropic → OpenAIAdded time to first event p99 (µs)213Valid · n=100 gateway / 100 direct—
Anthropic → OpenAIAdded inter-frame gap p50 (µs)0Valid · n=63 gateway / 63 direct—
Anthropic → OpenAIAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → OpenAIStreams sustained (frames/s)44,699.9Valid—
Anthropic → OpenAIStreams sustained1,610Valid—
Anthropic → OpenAIIdle RSS (MiB)7.45Valid—
Anthropic → OpenAISteady-state RSS (MiB)23.48Valid—
Anthropic → OpenAIRecovered RSS (MiB)16.34Valid—
Anthropic → OpenAIPeak RSS (MiB)23.84Valid—
Anthropic → OpenAIPeak RSS high-water mark (MiB)23.72Valid—
Anthropic → OpenAITime to plateau (s)—Not measured—
Anthropic → OpenAIMemory growth (MiB/min)0.3Valid—
Anthropic → OpenAIMemory load duration (s)300Valid—
Anthropic → OpenAIMemory shape under load0Valid—
Anthropic → OpenAIIdle memory shape0Valid—
Anthropic → OpenAIIdle memory static0Valid—
Anthropic → OpenAIIdle memory growth (MiB/min)0.004Valid—
Anthropic → OpenAIThroughput (req/s)46,675concurrency 16; n=280,0761 ms
Anthropic → OpenAIThroughput (req/s)47,965concurrency 64; n=287,7985 ms
Anthropic → OpenAIThroughput (req/s)47,965concurrency 64; n=287,79810 ms
Anthropic → OpenAIThroughput (req/s)47,965concurrency 64; n=287,79850 ms
Anthropic → OpenAIThroughput (req/s)47,965concurrency 64; n=287,798100 ms
Anthropic → OpenAIThroughput (req/s)47,965concurrency 64; n=287,798Unbounded
Gemini → OpenAIAdded latency p50 (µs)104Valid · n=46,858 gateway / 265,285 direct—
Gemini → OpenAIAdded latency p99 (µs)118Valid · n=46,858 gateway / 265,285 direct—
Gemini → OpenAIGateway p99 at concurrency 1 (µs)145Valid · n=46,858—
Gemini → OpenAIDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=265,285—
Gemini → OpenAICPU per request (µs)92.72Valid—
Gemini → OpenAIRequests per CPU-second10,784.7Valid—
Gemini → OpenAICost window concurrency8Valid—
Gemini → OpenAICost window core utilisation0.961Valid—
Gemini → OpenAICost window successful requests249,665Valid—
Gemini → OpenAICost window throughput (req/s)41,606Valid—
Gemini → OpenAICost window threads10Valid—
Gemini → OpenAINon-voluntary context switches per request0Valid—
Gemini → OpenAICost window major page faults0Valid—
Gemini → OpenAIAdded time to first event p50 (µs)—Untestable—
Gemini → OpenAIAdded time to first event p99 (µs)—Untestable—
Gemini → OpenAIAdded inter-frame gap p50 (µs)—Untestable—
Gemini → OpenAIAdded inter-frame gap p99 (µs)—Untestable—
Gemini → OpenAIStreams sustained (frames/s)—Untestable—
Gemini → OpenAIStreams sustained—Untestable—
Gemini → OpenAIIdle RSS (MiB)7.43Valid—
Gemini → OpenAISteady-state RSS (MiB)21.84Valid—
Gemini → OpenAIRecovered RSS (MiB)16.45Valid—
Gemini → OpenAIPeak RSS (MiB)22.51Valid—
Gemini → OpenAIPeak RSS high-water mark (MiB)22.48Valid—
Gemini → OpenAITime to plateau (s)—Not measured—
Gemini → OpenAIMemory growth (MiB/min)0.572Valid—
Gemini → OpenAIMemory load duration (s)300Valid—
Gemini → OpenAIMemory shape under load1Valid—
Gemini → OpenAIIdle memory shape0Valid—
Gemini → OpenAIIdle memory static0Valid—
Gemini → OpenAIIdle memory growth (MiB/min)0.004Valid—
Gemini → OpenAIThroughput (req/s)45,104concurrency 16; n=270,6581 ms
Gemini → OpenAIThroughput (req/s)46,080concurrency 32; n=276,5045 ms
Gemini → OpenAIThroughput (req/s)46,080concurrency 32; n=276,50410 ms
Gemini → OpenAIThroughput (req/s)46,080concurrency 32; n=276,50450 ms
Gemini → OpenAIThroughput (req/s)46,080concurrency 32; n=276,504100 ms
Gemini → OpenAIThroughput (req/s)46,080concurrency 32; n=276,504Unbounded
OpenAI Responses → OpenAIAdded latency p50 (µs)144Valid · n=35,866 gateway / 266,530 direct—
OpenAI Responses → OpenAIAdded latency p99 (µs)159Valid · n=35,866 gateway / 266,530 direct—
OpenAI Responses → OpenAIGateway p99 at concurrency 1 (µs)186Valid · n=35,866—
OpenAI Responses → OpenAIDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=266,530—
OpenAI Responses → OpenAICPU per request (µs)134.5Valid—
OpenAI Responses → OpenAIRequests per CPU-second7,437Valid—
OpenAI Responses → OpenAICost window concurrency8Valid—
OpenAI Responses → OpenAICost window core utilisation0.981Valid—
OpenAI Responses → OpenAICost window successful requests175,663Valid—
OpenAI Responses → OpenAICost window throughput (req/s)29,273Valid—
OpenAI Responses → OpenAICost window threads10Valid—
OpenAI Responses → OpenAINon-voluntary context switches per request0Valid—
OpenAI Responses → OpenAICost window major page faults0Valid—
OpenAI Responses → OpenAIAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → OpenAIAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → OpenAIAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → OpenAIAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → OpenAIStreams sustained (frames/s)—Untestable—
OpenAI Responses → OpenAIStreams sustained—Untestable—
OpenAI Responses → OpenAIIdle RSS (MiB)7.43Valid—
OpenAI Responses → OpenAISteady-state RSS (MiB)21.13Valid—
OpenAI Responses → OpenAIRecovered RSS (MiB)15.38Valid—
OpenAI Responses → OpenAIPeak RSS (MiB)21.75Valid—
OpenAI Responses → OpenAIPeak RSS high-water mark (MiB)21.45Valid—
OpenAI Responses → OpenAITime to plateau (s)—Not measured—
OpenAI Responses → OpenAIMemory growth (MiB/min)-0.37Valid—
OpenAI Responses → OpenAIMemory load duration (s)300Valid—
OpenAI Responses → OpenAIMemory shape under load0Valid—
OpenAI Responses → OpenAIIdle memory shape0Valid—
OpenAI Responses → OpenAIIdle memory static0Valid—
OpenAI Responses → OpenAIIdle memory growth (MiB/min)0.004Valid—
OpenAI Responses → OpenAIThroughput (req/s)30,322concurrency 16; n=181,9501 ms
OpenAI Responses → OpenAIThroughput (req/s)30,374concurrency 32; n=182,2565 ms
OpenAI Responses → OpenAIThroughput (req/s)30,374concurrency 32; n=182,25610 ms
OpenAI Responses → OpenAIThroughput (req/s)30,374concurrency 32; n=182,25650 ms
OpenAI Responses → OpenAIThroughput (req/s)30,374concurrency 32; n=182,256100 ms
OpenAI Responses → OpenAIThroughput (req/s)30,374concurrency 32; n=182,256Unbounded
Bedrock → OpenAIAdded latency p50 (µs)103Valid · n=47,501 gateway / 263,962 direct—
Bedrock → OpenAIAdded latency p99 (µs)115Valid · n=47,501 gateway / 263,962 direct—
Bedrock → OpenAIGateway p99 at concurrency 1 (µs)143Valid · n=47,501—
Bedrock → OpenAIDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=263,962—
Bedrock → OpenAICPU per request (µs)92.12Valid—
Bedrock → OpenAIRequests per CPU-second10,855.2Valid—
Bedrock → OpenAICost window concurrency8Valid—
Bedrock → OpenAICost window core utilisation0.961Valid—
Bedrock → OpenAICost window successful requests251,298Valid—
Bedrock → OpenAICost window throughput (req/s)41,879Valid—
Bedrock → OpenAICost window threads10Valid—
Bedrock → OpenAINon-voluntary context switches per request0Valid—
Bedrock → OpenAICost window major page faults0Valid—
Bedrock → OpenAIAdded time to first event p50 (µs)—Untestable—
Bedrock → OpenAIAdded time to first event p99 (µs)—Untestable—
Bedrock → OpenAIAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → OpenAIAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → OpenAIStreams sustained (frames/s)—Untestable—
Bedrock → OpenAIStreams sustained—Untestable—
Bedrock → OpenAIIdle RSS (MiB)7.42Valid—
Bedrock → OpenAISteady-state RSS (MiB)20.69Valid—
Bedrock → OpenAIRecovered RSS (MiB)15.17Valid—
Bedrock → OpenAIPeak RSS (MiB)21.05Valid—
Bedrock → OpenAIPeak RSS high-water mark (MiB)20.8Valid—
Bedrock → OpenAITime to plateau (s)192Valid—
Bedrock → OpenAIMemory growth (MiB/min)-0.05Valid—
Bedrock → OpenAIMemory load duration (s)300Valid—
Bedrock → OpenAIMemory shape under load—Not measured—
Bedrock → OpenAIIdle memory shape0Valid—
Bedrock → OpenAIIdle memory static0Valid—
Bedrock → OpenAIIdle memory growth (MiB/min)0.004Valid—
Bedrock → OpenAIThroughput (req/s)45,153concurrency 16; n=270,9451 ms
Bedrock → OpenAIThroughput (req/s)46,330concurrency 64; n=277,9935 ms
Bedrock → OpenAIThroughput (req/s)46,330concurrency 64; n=277,99310 ms
Bedrock → OpenAIThroughput (req/s)46,330concurrency 64; n=277,99350 ms
Bedrock → OpenAIThroughput (req/s)46,330concurrency 64; n=277,993100 ms
Bedrock → OpenAIThroughput (req/s)46,330concurrency 64; n=277,993Unbounded
Anthropic → CohereAdded latency p50 (µs)99Valid · n=48,830 gateway / 265,034 direct—
Anthropic → CohereAdded latency p99 (µs)112Valid · n=48,830 gateway / 265,034 direct—
Anthropic → CohereGateway p99 at concurrency 1 (µs)139Valid · n=48,830—
Anthropic → CohereDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=265,034—
Anthropic → CohereCPU per request (µs)87.39Valid—
Anthropic → CohereRequests per CPU-second11,443Valid—
Anthropic → CohereCost window concurrency8Valid—
Anthropic → CohereCost window core utilisation0.96Valid—
Anthropic → CohereCost window successful requests264,677Valid—
Anthropic → CohereCost window throughput (req/s)44,107Valid—
Anthropic → CohereCost window threads10Valid—
Anthropic → CohereNon-voluntary context switches per request0Valid—
Anthropic → CohereCost window major page faults0Valid—
Anthropic → CohereAdded time to first event p50 (µs)—Untestable—
Anthropic → CohereAdded time to first event p99 (µs)—Untestable—
Anthropic → CohereAdded inter-frame gap p50 (µs)—Untestable—
Anthropic → CohereAdded inter-frame gap p99 (µs)—Untestable—
Anthropic → CohereStreams sustained (frames/s)—Untestable—
Anthropic → CohereStreams sustained—Untestable—
Anthropic → CohereIdle RSS (MiB)7.41Valid—
Anthropic → CohereSteady-state RSS (MiB)23.14Valid—
Anthropic → CohereRecovered RSS (MiB)15.3Valid—
Anthropic → CoherePeak RSS (MiB)24.07Valid—
Anthropic → CoherePeak RSS high-water mark (MiB)23.85Valid—
Anthropic → CohereTime to plateau (s)—Not measured—
Anthropic → CohereMemory growth (MiB/min)0.453Valid—
Anthropic → CohereMemory load duration (s)300Valid—
Anthropic → CohereMemory shape under load0Valid—
Anthropic → CohereIdle memory shape0Valid—
Anthropic → CohereIdle memory static0Valid—
Anthropic → CohereIdle memory growth (MiB/min)0.004Valid—
Anthropic → CohereThroughput (req/s)47,488concurrency 16; n=284,9571 ms
Anthropic → CohereThroughput (req/s)48,799concurrency 64; n=292,8045 ms
Anthropic → CohereThroughput (req/s)48,799concurrency 64; n=292,80410 ms
Anthropic → CohereThroughput (req/s)48,799concurrency 64; n=292,80450 ms
Anthropic → CohereThroughput (req/s)48,799concurrency 64; n=292,804100 ms
Anthropic → CohereThroughput (req/s)48,799concurrency 64; n=292,804Unbounded
OpenAI → CohereAdded latency p50 (µs)96Valid · n=49,997 gateway / 264,312 direct—
OpenAI → CohereAdded latency p99 (µs)109Valid · n=49,997 gateway / 264,312 direct—
OpenAI → CohereGateway p99 at concurrency 1 (µs)136Valid · n=49,997—
OpenAI → CohereDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=264,312—
OpenAI → CohereCPU per request (µs)87.44Valid—
OpenAI → CohereRequests per CPU-second11,436.7Valid—
OpenAI → CohereCost window concurrency8Valid—
OpenAI → CohereCost window core utilisation0.963Valid—
OpenAI → CohereCost window successful requests265,216Valid—
OpenAI → CohereCost window throughput (req/s)44,197Valid—
OpenAI → CohereCost window threads10Valid—
OpenAI → CohereNon-voluntary context switches per request0Valid—
OpenAI → CohereCost window major page faults0Valid—
OpenAI → CohereAdded time to first event p50 (µs)—Untestable—
OpenAI → CohereAdded time to first event p99 (µs)—Untestable—
OpenAI → CohereAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → CohereAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → CohereStreams sustained (frames/s)—Untestable—
OpenAI → CohereStreams sustained—Untestable—
OpenAI → CohereIdle RSS (MiB)7.44Valid—
OpenAI → CohereSteady-state RSS (MiB)21.41Valid—
OpenAI → CohereRecovered RSS (MiB)16.35Valid—
OpenAI → CoherePeak RSS (MiB)21.8Valid—
OpenAI → CoherePeak RSS high-water mark (MiB)21.8Valid—
OpenAI → CohereTime to plateau (s)—Not measured—
OpenAI → CohereMemory growth (MiB/min)-0.051Valid—
OpenAI → CohereMemory load duration (s)300Valid—
OpenAI → CohereMemory shape under load0Valid—
OpenAI → CohereIdle memory shape0Valid—
OpenAI → CohereIdle memory static0Valid—
OpenAI → CohereIdle memory growth (MiB/min)0.004Valid—
OpenAI → CohereThroughput (req/s)47,491concurrency 16; n=284,9801 ms
OpenAI → CohereThroughput (req/s)49,018concurrency 128; n=294,1455 ms
OpenAI → CohereThroughput (req/s)49,018concurrency 128; n=294,14510 ms
OpenAI → CohereThroughput (req/s)49,018concurrency 128; n=294,14550 ms
OpenAI → CohereThroughput (req/s)49,018concurrency 128; n=294,145100 ms
OpenAI → CohereThroughput (req/s)49,018concurrency 128; n=294,145Unbounded
OpenAI Responses → CohereAdded latency p50 (µs)142Valid · n=36,123 gateway / 267,803 direct—
OpenAI Responses → CohereAdded latency p99 (µs)157Valid · n=36,123 gateway / 267,803 direct—
OpenAI Responses → CohereGateway p99 at concurrency 1 (µs)184Valid · n=36,123—
OpenAI Responses → CohereDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=267,803—
OpenAI Responses → CohereCPU per request (µs)133.2Valid—
OpenAI Responses → CohereRequests per CPU-second7,506.3Valid—
OpenAI Responses → CohereCost window concurrency8Valid—
OpenAI Responses → CohereCost window core utilisation0.98Valid—
OpenAI Responses → CohereCost window successful requests177,224Valid—
OpenAI Responses → CohereCost window throughput (req/s)29,533Valid—
OpenAI Responses → CohereCost window threads10Valid—
OpenAI Responses → CohereNon-voluntary context switches per request0Valid—
OpenAI Responses → CohereCost window major page faults0Valid—
OpenAI Responses → CohereAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → CohereAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → CohereAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → CohereAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → CohereStreams sustained (frames/s)—Untestable—
OpenAI Responses → CohereStreams sustained—Untestable—
OpenAI Responses → CohereIdle RSS (MiB)7.42Valid—
OpenAI Responses → CohereSteady-state RSS (MiB)23.78Valid—
OpenAI Responses → CohereRecovered RSS (MiB)15.77Valid—
OpenAI Responses → CoherePeak RSS (MiB)24.53Valid—
OpenAI Responses → CoherePeak RSS high-water mark (MiB)24Valid—
OpenAI Responses → CohereTime to plateau (s)—Not measured—
OpenAI Responses → CohereMemory growth (MiB/min)0.67Valid—
OpenAI Responses → CohereMemory load duration (s)300Valid—
OpenAI Responses → CohereMemory shape under load1Valid—
OpenAI Responses → CohereIdle memory shape0Valid—
OpenAI Responses → CohereIdle memory static0Valid—
OpenAI Responses → CohereIdle memory growth (MiB/min)0.004Valid—
OpenAI Responses → CohereThroughput (req/s)30,560concurrency 16; n=183,3781 ms
OpenAI Responses → CohereThroughput (req/s)30,675concurrency 32; n=184,0745 ms
OpenAI Responses → CohereThroughput (req/s)30,675concurrency 32; n=184,07410 ms
OpenAI Responses → CohereThroughput (req/s)30,675concurrency 32; n=184,07450 ms
OpenAI Responses → CohereThroughput (req/s)30,675concurrency 32; n=184,074100 ms
OpenAI Responses → CohereThroughput (req/s)30,675concurrency 32; n=184,074Unbounded
Gemini → CohereAdded latency p50 (µs)99Valid · n=48,806 gateway / 265,602 direct—
Gemini → CohereAdded latency p99 (µs)112Valid · n=48,806 gateway / 265,602 direct—
Gemini → CohereGateway p99 at concurrency 1 (µs)139Valid · n=48,806—
Gemini → CohereDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=265,602—
Gemini → CohereCPU per request (µs)91.58Valid—
Gemini → CohereRequests per CPU-second10,919.4Valid—
Gemini → CohereCost window concurrency8Valid—
Gemini → CohereCost window core utilisation0.961Valid—
Gemini → CohereCost window successful requests252,784Valid—
Gemini → CohereCost window throughput (req/s)42,125Valid—
Gemini → CohereCost window threads10Valid—
Gemini → CohereNon-voluntary context switches per request0Valid—
Gemini → CohereCost window major page faults0Valid—
Gemini → CohereAdded time to first event p50 (µs)—Untestable—
Gemini → CohereAdded time to first event p99 (µs)—Untestable—
Gemini → CohereAdded inter-frame gap p50 (µs)—Untestable—
Gemini → CohereAdded inter-frame gap p99 (µs)—Untestable—
Gemini → CohereStreams sustained (frames/s)—Untestable—
Gemini → CohereStreams sustained—Untestable—
Gemini → CohereIdle RSS (MiB)7.45Valid—
Gemini → CohereSteady-state RSS (MiB)20.93Valid—
Gemini → CohereRecovered RSS (MiB)15.79Valid—
Gemini → CoherePeak RSS (MiB)21.64Valid—
Gemini → CoherePeak RSS high-water mark (MiB)21.36Valid—
Gemini → CohereTime to plateau (s)—Not measured—
Gemini → CohereMemory growth (MiB/min)0.269Valid—
Gemini → CohereMemory load duration (s)300Valid—
Gemini → CohereMemory shape under load0Valid—
Gemini → CohereIdle memory shape0Valid—
Gemini → CohereIdle memory static0Valid—
Gemini → CohereIdle memory growth (MiB/min)0.004Valid—
Gemini → CohereThroughput (req/s)45,456concurrency 16; n=272,7671 ms
Gemini → CohereThroughput (req/s)46,559concurrency 128; n=279,3995 ms
Gemini → CohereThroughput (req/s)46,559concurrency 128; n=279,39910 ms
Gemini → CohereThroughput (req/s)46,559concurrency 128; n=279,39950 ms
Gemini → CohereThroughput (req/s)46,559concurrency 128; n=279,399100 ms
Gemini → CohereThroughput (req/s)46,559concurrency 128; n=279,399Unbounded
Cohere → CohereAdded latency p50 (µs)78Valid · n=59,104 gateway / 267,546 direct—
Cohere → CohereAdded latency p99 (µs)89Valid · n=59,104 gateway / 267,546 direct—
Cohere → CohereGateway p99 at concurrency 1 (µs)116Valid · n=59,104—
Cohere → CohereDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=267,546—
Cohere → CohereCPU per request (µs)69.34Valid—
Cohere → CohereRequests per CPU-second14,422.1Valid—
Cohere → CohereCost window concurrency8Valid—
Cohere → CohereCost window core utilisation0.957Valid—
Cohere → CohereCost window successful requests332,285Valid—
Cohere → CohereCost window throughput (req/s)55,375Valid—
Cohere → CohereCost window threads10Valid—
Cohere → CohereNon-voluntary context switches per request0Valid—
Cohere → CohereCost window major page faults0Valid—
Cohere → CohereAdded time to first event p50 (µs)—Untestable—
Cohere → CohereAdded time to first event p99 (µs)—Untestable—
Cohere → CohereAdded inter-frame gap p50 (µs)—Untestable—
Cohere → CohereAdded inter-frame gap p99 (µs)—Untestable—
Cohere → CohereStreams sustained (frames/s)—Untestable—
Cohere → CohereStreams sustained—Untestable—
Cohere → CohereIdle RSS (MiB)7.44Valid—
Cohere → CohereSteady-state RSS (MiB)21.1Valid—
Cohere → CohereRecovered RSS (MiB)15.61Valid—
Cohere → CoherePeak RSS (MiB)21.71Valid—
Cohere → CoherePeak RSS high-water mark (MiB)21.43Valid—
Cohere → CohereTime to plateau (s)—Not measured—
Cohere → CohereMemory growth (MiB/min)-0.038Valid—
Cohere → CohereMemory load duration (s)300Valid—
Cohere → CohereMemory shape under load0Valid—
Cohere → CohereIdle memory shape0Valid—
Cohere → CohereIdle memory static0Valid—
Cohere → CohereIdle memory growth (MiB/min)0.004Valid—
Cohere → CohereThroughput (req/s)61,466concurrency 32; n=368,8091 ms
Cohere → CohereThroughput (req/s)62,630concurrency 64; n=375,8435 ms
Cohere → CohereThroughput (req/s)62,630concurrency 64; n=375,84310 ms
Cohere → CohereThroughput (req/s)62,630concurrency 64; n=375,84350 ms
Cohere → CohereThroughput (req/s)62,630concurrency 64; n=375,843100 ms
Cohere → CohereThroughput (req/s)62,630concurrency 64; n=375,843Unbounded
Bedrock → CohereAdded latency p50 (µs)103Valid · n=47,542 gateway / 266,764 direct—
Bedrock → CohereAdded latency p99 (µs)115Valid · n=47,542 gateway / 266,764 direct—
Bedrock → CohereGateway p99 at concurrency 1 (µs)142Valid · n=47,542—
Bedrock → CohereDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=266,764—
Bedrock → CohereCPU per request (µs)91.37Valid—
Bedrock → CohereRequests per CPU-second10,944.3Valid—
Bedrock → CohereCost window concurrency8Valid—
Bedrock → CohereCost window core utilisation0.96Valid—
Bedrock → CohereCost window successful requests252,922Valid—
Bedrock → CohereCost window throughput (req/s)42,148Valid—
Bedrock → CohereCost window threads10Valid—
Bedrock → CohereNon-voluntary context switches per request0Valid—
Bedrock → CohereCost window major page faults0Valid—
Bedrock → CohereAdded time to first event p50 (µs)—Untestable—
Bedrock → CohereAdded time to first event p99 (µs)—Untestable—
Bedrock → CohereAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → CohereAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → CohereStreams sustained (frames/s)—Untestable—
Bedrock → CohereStreams sustained—Untestable—
Bedrock → CohereIdle RSS (MiB)7.43Valid—
Bedrock → CohereSteady-state RSS (MiB)22.61Valid—
Bedrock → CohereRecovered RSS (MiB)15.53Valid—
Bedrock → CoherePeak RSS (MiB)23.04Valid—
Bedrock → CoherePeak RSS high-water mark (MiB)22.74Valid—
Bedrock → CohereTime to plateau (s)—Not measured—
Bedrock → CohereMemory growth (MiB/min)0.056Valid—
Bedrock → CohereMemory load duration (s)300Valid—
Bedrock → CohereMemory shape under load0Valid—
Bedrock → CohereIdle memory shape0Valid—
Bedrock → CohereIdle memory static0Valid—
Bedrock → CohereIdle memory growth (MiB/min)0.004Valid—
Bedrock → CohereThroughput (req/s)45,878concurrency 16; n=275,2991 ms
Bedrock → CohereThroughput (req/s)46,932concurrency 64; n=281,6015 ms
Bedrock → CohereThroughput (req/s)46,932concurrency 64; n=281,60110 ms
Bedrock → CohereThroughput (req/s)46,932concurrency 64; n=281,60150 ms
Bedrock → CohereThroughput (req/s)46,932concurrency 64; n=281,601100 ms
Bedrock → CohereThroughput (req/s)46,932concurrency 64; n=281,601Unbounded
Gemini → OpenAI ResponsesAdded latency p50 (µs)108Valid · n=45,574 gateway / 262,793 direct—
Gemini → OpenAI ResponsesAdded latency p99 (µs)121Valid · n=45,574 gateway / 262,793 direct—
Gemini → OpenAI ResponsesGateway p99 at concurrency 1 (µs)149Valid · n=45,574—
Gemini → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=262,793—
Gemini → OpenAI ResponsesCPU per request (µs)96.57Valid—
Gemini → OpenAI ResponsesRequests per CPU-second10,355.5Valid—
Gemini → OpenAI ResponsesCost window concurrency8Valid—
Gemini → OpenAI ResponsesCost window core utilisation0.963Valid—
Gemini → OpenAI ResponsesCost window successful requests240,144Valid—
Gemini → OpenAI ResponsesCost window throughput (req/s)40,019Valid—
Gemini → OpenAI ResponsesCost window threads10Valid—
Gemini → OpenAI ResponsesNon-voluntary context switches per request0Valid—
Gemini → OpenAI ResponsesCost window major page faults0Valid—
Gemini → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
Gemini → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
Gemini → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
Gemini → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
Gemini → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
Gemini → OpenAI ResponsesStreams sustained—Untestable—
Gemini → OpenAI ResponsesIdle RSS (MiB)7.43Valid—
Gemini → OpenAI ResponsesSteady-state RSS (MiB)22.65Valid—
Gemini → OpenAI ResponsesRecovered RSS (MiB)15.47Valid—
Gemini → OpenAI ResponsesPeak RSS (MiB)23.53Valid—
Gemini → OpenAI ResponsesPeak RSS high-water mark (MiB)23.53Valid—
Gemini → OpenAI ResponsesTime to plateau (s)—Not measured—
Gemini → OpenAI ResponsesMemory growth (MiB/min)1.35Valid—
Gemini → OpenAI ResponsesMemory load duration (s)300Valid—
Gemini → OpenAI ResponsesMemory shape under load1Valid—
Gemini → OpenAI ResponsesIdle memory shape0Valid—
Gemini → OpenAI ResponsesIdle memory static0Valid—
Gemini → OpenAI ResponsesIdle memory growth (MiB/min)0.004Valid—
Gemini → OpenAI ResponsesThroughput (req/s)42,669concurrency 16; n=256,0461 ms
Gemini → OpenAI ResponsesThroughput (req/s)43,426concurrency 64; n=260,5775 ms
Gemini → OpenAI ResponsesThroughput (req/s)43,426concurrency 64; n=260,57710 ms
Gemini → OpenAI ResponsesThroughput (req/s)43,426concurrency 64; n=260,57750 ms
Gemini → OpenAI ResponsesThroughput (req/s)43,426concurrency 64; n=260,577100 ms
Gemini → OpenAI ResponsesThroughput (req/s)43,426concurrency 64; n=260,577Unbounded
Anthropic → OpenAI ResponsesAdded latency p50 (µs)105Valid · n=46,499 gateway / 259,447 direct—
Anthropic → OpenAI ResponsesAdded latency p99 (µs)118Valid · n=46,499 gateway / 259,447 direct—
Anthropic → OpenAI ResponsesGateway p99 at concurrency 1 (µs)146Valid · n=46,499—
Anthropic → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=259,447—
Anthropic → OpenAI ResponsesCPU per request (µs)93.29Valid—
Anthropic → OpenAI ResponsesRequests per CPU-second10,719.4Valid—
Anthropic → OpenAI ResponsesCost window concurrency8Valid—
Anthropic → OpenAI ResponsesCost window core utilisation0.961Valid—
Anthropic → OpenAI ResponsesCost window successful requests248,048Valid—
Anthropic → OpenAI ResponsesCost window throughput (req/s)41,335Valid—
Anthropic → OpenAI ResponsesCost window threads10Valid—
Anthropic → OpenAI ResponsesNon-voluntary context switches per request0Valid—
Anthropic → OpenAI ResponsesCost window major page faults0Valid—
Anthropic → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
Anthropic → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
Anthropic → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
Anthropic → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
Anthropic → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
Anthropic → OpenAI ResponsesStreams sustained—Untestable—
Anthropic → OpenAI ResponsesIdle RSS (MiB)7.42Valid—
Anthropic → OpenAI ResponsesSteady-state RSS (MiB)21.11Valid—
Anthropic → OpenAI ResponsesRecovered RSS (MiB)15.63Valid—
Anthropic → OpenAI ResponsesPeak RSS (MiB)21.59Valid—
Anthropic → OpenAI ResponsesPeak RSS high-water mark (MiB)21.47Valid—
Anthropic → OpenAI ResponsesTime to plateau (s)156Valid—
Anthropic → OpenAI ResponsesMemory growth (MiB/min)-0.108Valid—
Anthropic → OpenAI ResponsesMemory load duration (s)300Valid—
Anthropic → OpenAI ResponsesMemory shape under load—Not measured—
Anthropic → OpenAI ResponsesIdle memory shape0Valid—
Anthropic → OpenAI ResponsesIdle memory static0Valid—
Anthropic → OpenAI ResponsesIdle memory growth (MiB/min)0.004Valid—
Anthropic → OpenAI ResponsesThroughput (req/s)44,406concurrency 16; n=266,4671 ms
Anthropic → OpenAI ResponsesThroughput (req/s)45,442concurrency 64; n=272,6575 ms
Anthropic → OpenAI ResponsesThroughput (req/s)45,442concurrency 64; n=272,65710 ms
Anthropic → OpenAI ResponsesThroughput (req/s)45,442concurrency 64; n=272,65750 ms
Anthropic → OpenAI ResponsesThroughput (req/s)45,442concurrency 64; n=272,657100 ms
Anthropic → OpenAI ResponsesThroughput (req/s)45,442concurrency 64; n=272,657Unbounded
OpenAI → OpenAI ResponsesAdded latency p50 (µs)104Valid · n=46,929 gateway / 263,227 direct—
OpenAI → OpenAI ResponsesAdded latency p99 (µs)117Valid · n=46,929 gateway / 263,227 direct—
OpenAI → OpenAI ResponsesGateway p99 at concurrency 1 (µs)145Valid · n=46,929—
OpenAI → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=263,227—
OpenAI → OpenAI ResponsesCPU per request (µs)92.93Valid—
OpenAI → OpenAI ResponsesRequests per CPU-second10,760.5Valid—
OpenAI → OpenAI ResponsesCost window concurrency8Valid—
OpenAI → OpenAI ResponsesCost window core utilisation0.961Valid—
OpenAI → OpenAI ResponsesCost window successful requests248,997Valid—
OpenAI → OpenAI ResponsesCost window throughput (req/s)41,494Valid—
OpenAI → OpenAI ResponsesCost window threads10Valid—
OpenAI → OpenAI ResponsesNon-voluntary context switches per request0Valid—
OpenAI → OpenAI ResponsesCost window major page faults0Valid—
OpenAI → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
OpenAI → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
OpenAI → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
OpenAI → OpenAI ResponsesStreams sustained—Untestable—
OpenAI → OpenAI ResponsesIdle RSS (MiB)7.41Valid—
OpenAI → OpenAI ResponsesSteady-state RSS (MiB)21.27Valid—
OpenAI → OpenAI ResponsesRecovered RSS (MiB)15.77Valid—
OpenAI → OpenAI ResponsesPeak RSS (MiB)21.65Valid—
OpenAI → OpenAI ResponsesPeak RSS high-water mark (MiB)21.43Valid—
OpenAI → OpenAI ResponsesTime to plateau (s)246Valid—
OpenAI → OpenAI ResponsesMemory growth (MiB/min)0.069Valid—
OpenAI → OpenAI ResponsesMemory load duration (s)300Valid—
OpenAI → OpenAI ResponsesMemory shape under load—Not measured—
OpenAI → OpenAI ResponsesIdle memory shape—Not measured—
OpenAI → OpenAI ResponsesIdle memory static1Valid—
OpenAI → OpenAI ResponsesIdle memory growth (MiB/min)0.004Valid—
OpenAI → OpenAI ResponsesThroughput (req/s)44,763concurrency 16; n=268,6031 ms
OpenAI → OpenAI ResponsesThroughput (req/s)45,770concurrency 64; n=274,6295 ms
OpenAI → OpenAI ResponsesThroughput (req/s)45,770concurrency 64; n=274,62910 ms
OpenAI → OpenAI ResponsesThroughput (req/s)45,770concurrency 64; n=274,62950 ms
OpenAI → OpenAI ResponsesThroughput (req/s)45,770concurrency 64; n=274,629100 ms
OpenAI → OpenAI ResponsesThroughput (req/s)45,770concurrency 64; n=274,629Unbounded
Cohere → OpenAI ResponsesAdded latency p50 (µs)102Valid · n=47,752 gateway / 268,640 direct—
Cohere → OpenAI ResponsesAdded latency p99 (µs)116Valid · n=47,752 gateway / 268,640 direct—
Cohere → OpenAI ResponsesGateway p99 at concurrency 1 (µs)143Valid · n=47,752—
Cohere → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=268,640—
Cohere → OpenAI ResponsesCPU per request (µs)91.48Valid—
Cohere → OpenAI ResponsesRequests per CPU-second10,931.6Valid—
Cohere → OpenAI ResponsesCost window concurrency8Valid—
Cohere → OpenAI ResponsesCost window core utilisation0.96Valid—
Cohere → OpenAI ResponsesCost window successful requests252,738Valid—
Cohere → OpenAI ResponsesCost window throughput (req/s)42,119Valid—
Cohere → OpenAI ResponsesCost window threads10Valid—
Cohere → OpenAI ResponsesNon-voluntary context switches per request0Valid—
Cohere → OpenAI ResponsesCost window major page faults0Valid—
Cohere → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
Cohere → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
Cohere → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
Cohere → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
Cohere → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
Cohere → OpenAI ResponsesStreams sustained—Untestable—
Cohere → OpenAI ResponsesIdle RSS (MiB)7.44Valid—
Cohere → OpenAI ResponsesSteady-state RSS (MiB)21.58Valid—
Cohere → OpenAI ResponsesRecovered RSS (MiB)16.15Valid—
Cohere → OpenAI ResponsesPeak RSS (MiB)22.04Valid—
Cohere → OpenAI ResponsesPeak RSS high-water mark (MiB)21.62Valid—
Cohere → OpenAI ResponsesTime to plateau (s)—Not measured—
Cohere → OpenAI ResponsesMemory growth (MiB/min)-0.474Valid—
Cohere → OpenAI ResponsesMemory load duration (s)300Valid—
Cohere → OpenAI ResponsesMemory shape under load-1Valid—
Cohere → OpenAI ResponsesIdle memory shape0Valid—
Cohere → OpenAI ResponsesIdle memory static0Valid—
Cohere → OpenAI ResponsesIdle memory growth (MiB/min)0.004Valid—
Cohere → OpenAI ResponsesThroughput (req/s)45,427concurrency 16; n=272,5921 ms
Cohere → OpenAI ResponsesThroughput (req/s)46,419concurrency 64; n=278,5285 ms
Cohere → OpenAI ResponsesThroughput (req/s)46,419concurrency 64; n=278,52810 ms
Cohere → OpenAI ResponsesThroughput (req/s)46,419concurrency 64; n=278,52850 ms
Cohere → OpenAI ResponsesThroughput (req/s)46,419concurrency 64; n=278,528100 ms
Cohere → OpenAI ResponsesThroughput (req/s)46,419concurrency 64; n=278,528Unbounded
OpenAI Responses → OpenAI ResponsesAdded latency p50 (µs)79Valid · n=58,401 gateway / 264,340 direct—
OpenAI Responses → OpenAI ResponsesAdded latency p99 (µs)90Valid · n=58,401 gateway / 264,340 direct—
OpenAI Responses → OpenAI ResponsesGateway p99 at concurrency 1 (µs)118Valid · n=58,401—
OpenAI Responses → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=264,340—
OpenAI Responses → OpenAI ResponsesCPU per request (µs)70.64Valid—
OpenAI Responses → OpenAI ResponsesRequests per CPU-second14,157Valid—
OpenAI Responses → OpenAI ResponsesCost window concurrency8Valid—
OpenAI Responses → OpenAI ResponsesCost window core utilisation0.956Valid—
OpenAI Responses → OpenAI ResponsesCost window successful requests326,035Valid—
OpenAI Responses → OpenAI ResponsesCost window throughput (req/s)54,332Valid—
OpenAI Responses → OpenAI ResponsesCost window threads10Valid—
OpenAI Responses → OpenAI ResponsesNon-voluntary context switches per request0Valid—
OpenAI Responses → OpenAI ResponsesCost window major page faults0Valid—
OpenAI Responses → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained—Untestable—
OpenAI Responses → OpenAI ResponsesIdle RSS (MiB)7.43Valid—
OpenAI Responses → OpenAI ResponsesSteady-state RSS (MiB)20.77Valid—
OpenAI Responses → OpenAI ResponsesRecovered RSS (MiB)15.61Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS (MiB)21.49Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS high-water mark (MiB)21.43Valid—
OpenAI Responses → OpenAI ResponsesTime to plateau (s)—Not measured—
OpenAI Responses → OpenAI ResponsesMemory growth (MiB/min)0.682Valid—
OpenAI Responses → OpenAI ResponsesMemory load duration (s)300Valid—
OpenAI Responses → OpenAI ResponsesMemory shape under load1Valid—
OpenAI Responses → OpenAI ResponsesIdle memory shape0Valid—
OpenAI Responses → OpenAI ResponsesIdle memory static0Valid—
OpenAI Responses → OpenAI ResponsesIdle memory growth (MiB/min)0.004Valid—
OpenAI Responses → OpenAI ResponsesThroughput (req/s)59,818concurrency 32; n=358,9441 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)60,832concurrency 128; n=365,0445 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)60,832concurrency 128; n=365,04410 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)60,832concurrency 128; n=365,04450 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)60,832concurrency 128; n=365,044100 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)60,832concurrency 128; n=365,044Unbounded
Bedrock → OpenAI ResponsesAdded latency p50 (µs)110Valid · n=44,818 gateway / 263,285 direct—
Bedrock → OpenAI ResponsesAdded latency p99 (µs)123Valid · n=44,818 gateway / 263,285 direct—
Bedrock → OpenAI ResponsesGateway p99 at concurrency 1 (µs)151Valid · n=44,818—
Bedrock → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=263,285—
Bedrock → OpenAI ResponsesCPU per request (µs)97.04Valid—
Bedrock → OpenAI ResponsesRequests per CPU-second10,305.5Valid—
Bedrock → OpenAI ResponsesCost window concurrency8Valid—
Bedrock → OpenAI ResponsesCost window core utilisation0.96Valid—
Bedrock → OpenAI ResponsesCost window successful requests238,263Valid—
Bedrock → OpenAI ResponsesCost window throughput (req/s)39,706Valid—
Bedrock → OpenAI ResponsesCost window threads10Valid—
Bedrock → OpenAI ResponsesNon-voluntary context switches per request0Valid—
Bedrock → OpenAI ResponsesCost window major page faults0Valid—
Bedrock → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
Bedrock → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
Bedrock → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
Bedrock → OpenAI ResponsesStreams sustained—Untestable—
Bedrock → OpenAI ResponsesIdle RSS (MiB)7.43Valid—
Bedrock → OpenAI ResponsesSteady-state RSS (MiB)22.62Valid—
Bedrock → OpenAI ResponsesRecovered RSS (MiB)15.99Valid—
Bedrock → OpenAI ResponsesPeak RSS (MiB)23.49Valid—
Bedrock → OpenAI ResponsesPeak RSS high-water mark (MiB)23.33Valid—
Bedrock → OpenAI ResponsesTime to plateau (s)—Not measured—
Bedrock → OpenAI ResponsesMemory growth (MiB/min)0.59Valid—
Bedrock → OpenAI ResponsesMemory load duration (s)300Valid—
Bedrock → OpenAI ResponsesMemory shape under load1Valid—
Bedrock → OpenAI ResponsesIdle memory shape0Valid—
Bedrock → OpenAI ResponsesIdle memory static0Valid—
Bedrock → OpenAI ResponsesIdle memory growth (MiB/min)0.004Valid—
Bedrock → OpenAI ResponsesThroughput (req/s)42,923concurrency 16; n=257,5691 ms
Bedrock → OpenAI ResponsesThroughput (req/s)43,797concurrency 64; n=262,7965 ms
Bedrock → OpenAI ResponsesThroughput (req/s)43,797concurrency 64; n=262,79610 ms
Bedrock → OpenAI ResponsesThroughput (req/s)43,797concurrency 64; n=262,79650 ms
Bedrock → OpenAI ResponsesThroughput (req/s)43,797concurrency 64; n=262,796100 ms
Bedrock → OpenAI ResponsesThroughput (req/s)43,797concurrency 64; n=262,796Unbounded
Anthropic → GeminiAdded latency p50 (µs)101Valid · n=48,133 gateway / 261,686 direct—
Anthropic → GeminiAdded latency p99 (µs)113Valid · n=48,133 gateway / 261,686 direct—
Anthropic → GeminiGateway p99 at concurrency 1 (µs)141Valid · n=48,133—
Anthropic → GeminiDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=261,686—
Anthropic → GeminiCPU per request (µs)90.32Valid—
Anthropic → GeminiRequests per CPU-second11,071.3Valid—
Anthropic → GeminiCost window concurrency8Valid—
Anthropic → GeminiCost window core utilisation0.964Valid—
Anthropic → GeminiCost window successful requests256,964Valid—
Anthropic → GeminiCost window throughput (req/s)42,822Valid—
Anthropic → GeminiCost window threads10Valid—
Anthropic → GeminiNon-voluntary context switches per request0Valid—
Anthropic → GeminiCost window major page faults0Valid—
Anthropic → GeminiAdded time to first event p50 (µs)—Untestable—
Anthropic → GeminiAdded time to first event p99 (µs)—Untestable—
Anthropic → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Anthropic → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Anthropic → GeminiStreams sustained (frames/s)—Untestable—
Anthropic → GeminiStreams sustained—Untestable—
Anthropic → GeminiIdle RSS (MiB)7.44Valid—
Anthropic → GeminiSteady-state RSS (MiB)20.82Valid—
Anthropic → GeminiRecovered RSS (MiB)15.73Valid—
Anthropic → GeminiPeak RSS (MiB)21.05Valid—
Anthropic → GeminiPeak RSS high-water mark (MiB)20.84Valid—
Anthropic → GeminiTime to plateau (s)—Not measured—
Anthropic → GeminiMemory growth (MiB/min)0.347Valid—
Anthropic → GeminiMemory load duration (s)300Valid—
Anthropic → GeminiMemory shape under load0Valid—
Anthropic → GeminiIdle memory shape0Valid—
Anthropic → GeminiIdle memory static0Valid—
Anthropic → GeminiIdle memory growth (MiB/min)0.004Valid—
Anthropic → GeminiThroughput (req/s)45,587concurrency 16; n=273,5481 ms
Anthropic → GeminiThroughput (req/s)46,654concurrency 64; n=279,9445 ms
Anthropic → GeminiThroughput (req/s)46,654concurrency 64; n=279,94410 ms
Anthropic → GeminiThroughput (req/s)46,654concurrency 64; n=279,94450 ms
Anthropic → GeminiThroughput (req/s)46,654concurrency 64; n=279,944100 ms
Anthropic → GeminiThroughput (req/s)46,654concurrency 64; n=279,944Unbounded
Bedrock → GeminiAdded latency p50 (µs)104Valid · n=47,051 gateway / 267,236 direct—
Bedrock → GeminiAdded latency p99 (µs)117Valid · n=47,051 gateway / 267,236 direct—
Bedrock → GeminiGateway p99 at concurrency 1 (µs)144Valid · n=47,051—
Bedrock → GeminiDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=267,236—
Bedrock → GeminiCPU per request (µs)93.43Valid—
Bedrock → GeminiRequests per CPU-second10,702.9Valid—
Bedrock → GeminiCost window concurrency8Valid—
Bedrock → GeminiCost window core utilisation0.96Valid—
Bedrock → GeminiCost window successful requests247,451Valid—
Bedrock → GeminiCost window throughput (req/s)41,236Valid—
Bedrock → GeminiCost window threads10Valid—
Bedrock → GeminiNon-voluntary context switches per request0Valid—
Bedrock → GeminiCost window major page faults0Valid—
Bedrock → GeminiAdded time to first event p50 (µs)—Untestable—
Bedrock → GeminiAdded time to first event p99 (µs)—Untestable—
Bedrock → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → GeminiStreams sustained (frames/s)—Untestable—
Bedrock → GeminiStreams sustained—Untestable—
Bedrock → GeminiIdle RSS (MiB)7.42Valid—
Bedrock → GeminiSteady-state RSS (MiB)21.51Valid—
Bedrock → GeminiRecovered RSS (MiB)15.64Valid—
Bedrock → GeminiPeak RSS (MiB)22.76Valid—
Bedrock → GeminiPeak RSS high-water mark (MiB)22.25Valid—
Bedrock → GeminiTime to plateau (s)—Not measured—
Bedrock → GeminiMemory growth (MiB/min)0.651Valid—
Bedrock → GeminiMemory load duration (s)300Valid—
Bedrock → GeminiMemory shape under load1Valid—
Bedrock → GeminiIdle memory shape0Valid—
Bedrock → GeminiIdle memory static0Valid—
Bedrock → GeminiIdle memory growth (MiB/min)0.004Valid—
Bedrock → GeminiThroughput (req/s)44,045concurrency 16; n=264,3011 ms
Bedrock → GeminiThroughput (req/s)45,269concurrency 64; n=271,6225 ms
Bedrock → GeminiThroughput (req/s)45,269concurrency 64; n=271,62210 ms
Bedrock → GeminiThroughput (req/s)45,269concurrency 64; n=271,62250 ms
Bedrock → GeminiThroughput (req/s)45,269concurrency 64; n=271,622100 ms
Bedrock → GeminiThroughput (req/s)45,269concurrency 64; n=271,622Unbounded
Gemini → GeminiAdded latency p50 (µs)87Valid · n=54,333 gateway / 264,586 direct—
Gemini → GeminiAdded latency p99 (µs)97Valid · n=54,333 gateway / 264,586 direct—
Gemini → GeminiGateway p99 at concurrency 1 (µs)125Valid · n=54,333—
Gemini → GeminiDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=264,586—
Gemini → GeminiCPU per request (µs)77.73Valid—
Gemini → GeminiRequests per CPU-second12,864.3Valid—
Gemini → GeminiCost window concurrency8Valid—
Gemini → GeminiCost window core utilisation0.962Valid—
Gemini → GeminiCost window successful requests298,065Valid—
Gemini → GeminiCost window throughput (req/s)49,672Valid—
Gemini → GeminiCost window threads10Valid—
Gemini → GeminiNon-voluntary context switches per request0Valid—
Gemini → GeminiCost window major page faults0Valid—
Gemini → GeminiAdded time to first event p50 (µs)—Untestable—
Gemini → GeminiAdded time to first event p99 (µs)—Untestable—
Gemini → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Gemini → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Gemini → GeminiStreams sustained (frames/s)—Untestable—
Gemini → GeminiStreams sustained—Untestable—
Gemini → GeminiIdle RSS (MiB)7.42Valid—
Gemini → GeminiSteady-state RSS (MiB)21.42Valid—
Gemini → GeminiRecovered RSS (MiB)15.91Valid—
Gemini → GeminiPeak RSS (MiB)21.93Valid—
Gemini → GeminiPeak RSS high-water mark (MiB)21.64Valid—
Gemini → GeminiTime to plateau (s)—Not measured—
Gemini → GeminiMemory growth (MiB/min)0.002Valid—
Gemini → GeminiMemory load duration (s)300Valid—
Gemini → GeminiMemory shape under load0Valid—
Gemini → GeminiIdle memory shape0Valid—
Gemini → GeminiIdle memory static0Valid—
Gemini → GeminiIdle memory growth (MiB/min)0.004Valid—
Gemini → GeminiThroughput (req/s)54,786concurrency 32; n=328,7451 ms
Gemini → GeminiThroughput (req/s)55,374concurrency 64; n=332,2695 ms
Gemini → GeminiThroughput (req/s)55,374concurrency 64; n=332,26910 ms
Gemini → GeminiThroughput (req/s)55,374concurrency 64; n=332,26950 ms
Gemini → GeminiThroughput (req/s)55,374concurrency 64; n=332,269100 ms
Gemini → GeminiThroughput (req/s)55,374concurrency 64; n=332,269Unbounded
OpenAI → GeminiAdded latency p50 (µs)99Valid · n=48,998 gateway / 264,583 direct—
OpenAI → GeminiAdded latency p99 (µs)111Valid · n=48,998 gateway / 264,583 direct—
OpenAI → GeminiGateway p99 at concurrency 1 (µs)138Valid · n=48,998—
OpenAI → GeminiDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=264,583—
OpenAI → GeminiCPU per request (µs)89.82Valid—
OpenAI → GeminiRequests per CPU-second11,133.8Valid—
OpenAI → GeminiCost window concurrency8Valid—
OpenAI → GeminiCost window core utilisation0.963Valid—
OpenAI → GeminiCost window successful requests258,193Valid—
OpenAI → GeminiCost window throughput (req/s)43,027Valid—
OpenAI → GeminiCost window threads10Valid—
OpenAI → GeminiNon-voluntary context switches per request0Valid—
OpenAI → GeminiCost window major page faults0Valid—
OpenAI → GeminiAdded time to first event p50 (µs)—Untestable—
OpenAI → GeminiAdded time to first event p99 (µs)—Untestable—
OpenAI → GeminiAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → GeminiAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → GeminiStreams sustained (frames/s)—Untestable—
OpenAI → GeminiStreams sustained—Untestable—
OpenAI → GeminiIdle RSS (MiB)7.42Valid—
OpenAI → GeminiSteady-state RSS (MiB)22.4Valid—
OpenAI → GeminiRecovered RSS (MiB)14.95Valid—
OpenAI → GeminiPeak RSS (MiB)22.73Valid—
OpenAI → GeminiPeak RSS high-water mark (MiB)22.73Valid—
OpenAI → GeminiTime to plateau (s)—Not measured—
OpenAI → GeminiMemory growth (MiB/min)0.078Valid—
OpenAI → GeminiMemory load duration (s)300Valid—
OpenAI → GeminiMemory shape under load0Valid—
OpenAI → GeminiIdle memory shape0Valid—
OpenAI → GeminiIdle memory static0Valid—
OpenAI → GeminiIdle memory growth (MiB/min)0.004Valid—
OpenAI → GeminiThroughput (req/s)46,221concurrency 16; n=277,3571 ms
OpenAI → GeminiThroughput (req/s)47,238concurrency 64; n=283,4425 ms
OpenAI → GeminiThroughput (req/s)47,238concurrency 64; n=283,44210 ms
OpenAI → GeminiThroughput (req/s)47,238concurrency 64; n=283,44250 ms
OpenAI → GeminiThroughput (req/s)47,238concurrency 64; n=283,442100 ms
OpenAI → GeminiThroughput (req/s)47,238concurrency 64; n=283,442Unbounded
OpenAI Responses → GeminiAdded latency p50 (µs)148Valid · n=34,929 gateway / 262,960 direct—
OpenAI Responses → GeminiAdded latency p99 (µs)162Valid · n=34,929 gateway / 262,960 direct—
OpenAI Responses → GeminiGateway p99 at concurrency 1 (µs)190Valid · n=34,929—
OpenAI Responses → GeminiDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=262,960—
OpenAI Responses → GeminiCPU per request (µs)136.8Valid—
OpenAI Responses → GeminiRequests per CPU-second7,308.8Valid—
OpenAI Responses → GeminiCost window concurrency8Valid—
OpenAI Responses → GeminiCost window core utilisation0.981Valid—
OpenAI Responses → GeminiCost window successful requests172,561Valid—
OpenAI Responses → GeminiCost window throughput (req/s)28,757Valid—
OpenAI Responses → GeminiCost window threads10Valid—
OpenAI Responses → GeminiNon-voluntary context switches per request0Valid—
OpenAI Responses → GeminiCost window major page faults0Valid—
OpenAI Responses → GeminiAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → GeminiAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → GeminiAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → GeminiAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → GeminiStreams sustained (frames/s)—Untestable—
OpenAI Responses → GeminiStreams sustained—Untestable—
OpenAI Responses → GeminiIdle RSS (MiB)7.43Valid—
OpenAI Responses → GeminiSteady-state RSS (MiB)21.73Valid—
OpenAI Responses → GeminiRecovered RSS (MiB)15.08Valid—
OpenAI Responses → GeminiPeak RSS (MiB)22.41Valid—
OpenAI Responses → GeminiPeak RSS high-water mark (MiB)21.99Valid—
OpenAI Responses → GeminiTime to plateau (s)—Not measured—
OpenAI Responses → GeminiMemory growth (MiB/min)0.429Valid—
OpenAI Responses → GeminiMemory load duration (s)300Valid—
OpenAI Responses → GeminiMemory shape under load1Valid—
OpenAI Responses → GeminiIdle memory shape0Valid—
OpenAI Responses → GeminiIdle memory static0Valid—
OpenAI Responses → GeminiIdle memory growth (MiB/min)0.004Valid—
OpenAI Responses → GeminiThroughput (req/s)29,952concurrency 16; n=179,7341 ms
OpenAI Responses → GeminiThroughput (req/s)30,107concurrency 32; n=180,6645 ms
OpenAI Responses → GeminiThroughput (req/s)30,107concurrency 32; n=180,66410 ms
OpenAI Responses → GeminiThroughput (req/s)30,107concurrency 32; n=180,66450 ms
OpenAI Responses → GeminiThroughput (req/s)30,107concurrency 32; n=180,664100 ms
OpenAI Responses → GeminiThroughput (req/s)30,107concurrency 32; n=180,664Unbounded
Cohere → GeminiAdded latency p50 (µs)100Valid · n=49,012 gateway / 269,799 direct—
Cohere → GeminiAdded latency p99 (µs)112Valid · n=49,012 gateway / 269,799 direct—
Cohere → GeminiGateway p99 at concurrency 1 (µs)139Valid · n=49,012—
Cohere → GeminiDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=269,799—
Cohere → GeminiCPU per request (µs)88.31Valid—
Cohere → GeminiRequests per CPU-second11,323.5Valid—
Cohere → GeminiCost window concurrency8Valid—
Cohere → GeminiCost window core utilisation0.961Valid—
Cohere → GeminiCost window successful requests262,140Valid—
Cohere → GeminiCost window throughput (req/s)43,684Valid—
Cohere → GeminiCost window threads10Valid—
Cohere → GeminiNon-voluntary context switches per request0Valid—
Cohere → GeminiCost window major page faults0Valid—
Cohere → GeminiAdded time to first event p50 (µs)—Untestable—
Cohere → GeminiAdded time to first event p99 (µs)—Untestable—
Cohere → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Cohere → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Cohere → GeminiStreams sustained (frames/s)—Untestable—
Cohere → GeminiStreams sustained—Untestable—
Cohere → GeminiIdle RSS (MiB)7.42Valid—
Cohere → GeminiSteady-state RSS (MiB)20.26Valid—
Cohere → GeminiRecovered RSS (MiB)15.55Valid—
Cohere → GeminiPeak RSS (MiB)20.91Valid—
Cohere → GeminiPeak RSS high-water mark (MiB)20.59Valid—
Cohere → GeminiTime to plateau (s)—Not measured—
Cohere → GeminiMemory growth (MiB/min)0.71Valid—
Cohere → GeminiMemory load duration (s)300Valid—
Cohere → GeminiMemory shape under load1Valid—
Cohere → GeminiIdle memory shape0Valid—
Cohere → GeminiIdle memory static0Valid—
Cohere → GeminiIdle memory growth (MiB/min)0.004Valid—
Cohere → GeminiThroughput (req/s)46,982concurrency 16; n=281,9241 ms
Cohere → GeminiThroughput (req/s)47,826concurrency 64; n=286,9655 ms
Cohere → GeminiThroughput (req/s)47,826concurrency 64; n=286,96510 ms
Cohere → GeminiThroughput (req/s)47,826concurrency 64; n=286,96550 ms
Cohere → GeminiThroughput (req/s)47,826concurrency 64; n=286,965100 ms
Cohere → GeminiThroughput (req/s)47,826concurrency 64; n=286,965Unbounded

Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

Ingress → upstreamCell stateProbe HTTP statusRecorded reasonRecorded detailEvidence scopeResponse excerptTested build
Cohere → AnthropicServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI → AnthropicServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI Responses → AnthropicServed———Recorded run evidence—getbusbar/busbar:1.5.5
Anthropic → AnthropicServed———Recorded run evidence—getbusbar/busbar:1.5.5
Gemini → AnthropicServed———Recorded run evidence—getbusbar/busbar:1.5.5
Bedrock → AnthropicServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI Responses → BedrockServed———Recorded run evidence—getbusbar/busbar:1.5.5
Gemini → BedrockServed———Recorded run evidence—getbusbar/busbar:1.5.5
Cohere → BedrockServed———Recorded run evidence—getbusbar/busbar:1.5.5
Bedrock → BedrockServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI → BedrockServed———Recorded run evidence—getbusbar/busbar:1.5.5
Anthropic → BedrockServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI → OpenAIServed———Recorded run evidence—getbusbar/busbar:1.5.5
Cohere → OpenAIServed———Recorded run evidence—getbusbar/busbar:1.5.5
Anthropic → OpenAIServed———Recorded run evidence—getbusbar/busbar:1.5.5
Gemini → OpenAIServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI Responses → OpenAIServed———Recorded run evidence—getbusbar/busbar:1.5.5
Bedrock → OpenAIServed———Recorded run evidence—getbusbar/busbar:1.5.5
Anthropic → CohereServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI → CohereServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI Responses → CohereServed———Recorded run evidence—getbusbar/busbar:1.5.5
Gemini → CohereServed———Recorded run evidence—getbusbar/busbar:1.5.5
Cohere → CohereServed———Recorded run evidence—getbusbar/busbar:1.5.5
Bedrock → CohereServed———Recorded run evidence—getbusbar/busbar:1.5.5
Gemini → OpenAI ResponsesServed———Recorded run evidence—getbusbar/busbar:1.5.5
Anthropic → OpenAI ResponsesServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI → OpenAI ResponsesServed———Recorded run evidence—getbusbar/busbar:1.5.5
Cohere → OpenAI ResponsesServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI Responses → OpenAI ResponsesServed———Recorded run evidence—getbusbar/busbar:1.5.5
Bedrock → OpenAI ResponsesServed———Recorded run evidence—getbusbar/busbar:1.5.5
Anthropic → GeminiServed———Recorded run evidence—getbusbar/busbar:1.5.5
Bedrock → GeminiServed———Recorded run evidence—getbusbar/busbar:1.5.5
Gemini → GeminiServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI → GeminiServed———Recorded run evidence—getbusbar/busbar:1.5.5
OpenAI Responses → GeminiServed———Recorded run evidence—getbusbar/busbar:1.5.5
Cohere → GeminiServed———Recorded run evidence—getbusbar/busbar:1.5.5
Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2
Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8
Load generator SHA-256
not recorded
net.core.somaxconn
65,535
net.ipv4.tcp_max_syn_backlog
65,535

Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

Ingress → upstreamConcurrencyLanes by HTTP statusLanes with no response
OpenAI → Anthropic1HTTP 200: 10
OpenAI → Anthropic2HTTP 200: 20
OpenAI → Anthropic4HTTP 200: 40
OpenAI → Anthropic8HTTP 200: 80
OpenAI → Anthropic16HTTP 200: 160
OpenAI → Anthropic32HTTP 200: 320
OpenAI → Anthropic64HTTP 200: 640
OpenAI → Anthropic128HTTP 200: 1280
OpenAI → Anthropic256HTTP 200: 2560
OpenAI → Anthropic512HTTP 200: 5120
OpenAI → Anthropic1,024HTTP 200: 1,0240
OpenAI → Anthropic2,048HTTP 200: 2,0480
OpenAI → Anthropic1,536HTTP 200: 1,5360
OpenAI → Anthropic1,792HTTP 200: 1,7920
OpenAI → Anthropic1,664HTTP 200: 1,6640
OpenAI → Anthropic1,600HTTP 200: 1,6000
OpenAI → Anthropic1,568HTTP 200: 1,5680
OpenAI → Anthropic1,552HTTP 200: 1,5520
OpenAI → Anthropic1,544HTTP 200: 1,5440
OpenAI → Anthropic1,540HTTP 200: 1,5400
OpenAI → Anthropic1,538HTTP 200: 1,5380
OpenAI → Anthropic1,537HTTP 200: 1,5370
OpenAI → Anthropic1,536HTTP 200: 1,5360
OpenAI → Anthropic1,536HTTP 200: 1,5360
Anthropic → Anthropic1HTTP 200: 10
Anthropic → Anthropic2HTTP 200: 20
Anthropic → Anthropic4HTTP 200: 40
Anthropic → Anthropic8HTTP 200: 80
Anthropic → Anthropic16HTTP 200: 160
Anthropic → Anthropic32HTTP 200: 320
Anthropic → Anthropic64HTTP 200: 640
Anthropic → Anthropic128HTTP 200: 1280
Anthropic → Anthropic256HTTP 200: 2560
Anthropic → Anthropic512HTTP 200: 5120
Anthropic → Anthropic1,024HTTP 200: 1,0240
Anthropic → Anthropic2,048HTTP 200: 2,0480
Anthropic → Anthropic1,536HTTP 200: 1,5360
Anthropic → Anthropic1,792HTTP 200: 1,7920
Anthropic → Anthropic1,664HTTP 200: 1,6640
Anthropic → Anthropic1,728HTTP 200: 1,7280
Anthropic → Anthropic1,696HTTP 200: 1,6960
Anthropic → Anthropic1,712HTTP 200: 1,7120
Anthropic → Anthropic1,704HTTP 200: 1,7040
Anthropic → Anthropic1,700HTTP 200: 1,7000
Anthropic → Anthropic1,698HTTP 200: 1,6980
Anthropic → Anthropic1,699HTTP 200: 1,6990
Anthropic → Anthropic1,698HTTP 200: 1,6980
Anthropic → Anthropic1,698HTTP 200: 1,6980
OpenAI → OpenAI1HTTP 200: 10
OpenAI → OpenAI2HTTP 200: 20
OpenAI → OpenAI4HTTP 200: 40
OpenAI → OpenAI8HTTP 200: 80
OpenAI → OpenAI16HTTP 200: 160
OpenAI → OpenAI32HTTP 200: 320
OpenAI → OpenAI64HTTP 200: 640
OpenAI → OpenAI128HTTP 200: 1280
OpenAI → OpenAI256HTTP 200: 2560
OpenAI → OpenAI512HTTP 200: 5120
OpenAI → OpenAI1,024HTTP 200: 1,0240
OpenAI → OpenAI2,048HTTP 200: 2,0480
OpenAI → OpenAI1,536HTTP 200: 1,5360
OpenAI → OpenAI1,280HTTP 200: 1,2800
OpenAI → OpenAI1,408HTTP 200: 1,4080
OpenAI → OpenAI1,472HTTP 200: 1,4720
OpenAI → OpenAI1,440HTTP 200: 1,4400
OpenAI → OpenAI1,456HTTP 200: 1,4560
OpenAI → OpenAI1,448HTTP 200: 1,4480
OpenAI → OpenAI1,452HTTP 200: 1,4520
OpenAI → OpenAI1,454HTTP 200: 1,4540
OpenAI → OpenAI1,453HTTP 200: 1,4530
OpenAI → OpenAI1,452HTTP 200: 1,4520
OpenAI → OpenAI1,452HTTP 200: 1,4520
Anthropic → OpenAI1HTTP 200: 10
Anthropic → OpenAI2HTTP 200: 20
Anthropic → OpenAI4HTTP 200: 40
Anthropic → OpenAI8HTTP 200: 80
Anthropic → OpenAI16HTTP 200: 160
Anthropic → OpenAI32HTTP 200: 320
Anthropic → OpenAI64HTTP 200: 640
Anthropic → OpenAI128HTTP 200: 1280
Anthropic → OpenAI256HTTP 200: 2560
Anthropic → OpenAI512HTTP 200: 5120
Anthropic → OpenAI1,024HTTP 200: 1,0240
Anthropic → OpenAI2,048HTTP 200: 2,0480
Anthropic → OpenAI1,536HTTP 200: 1,5360
Anthropic → OpenAI1,792HTTP 200: 1,7920
Anthropic → OpenAI1,664HTTP 200: 1,6640
Anthropic → OpenAI1,600HTTP 200: 1,6000
Anthropic → OpenAI1,632HTTP 200: 1,6320
Anthropic → OpenAI1,616HTTP 200: 1,6160
Anthropic → OpenAI1,608HTTP 200: 1,6080
Anthropic → OpenAI1,612HTTP 200: 1,6120
Anthropic → OpenAI1,610HTTP 200: 1,6100
Anthropic → OpenAI1,611HTTP 200: 1,6110
Anthropic → OpenAI1,610HTTP 200: 1,6100
Anthropic → OpenAI1,610HTTP 200: 1,6100

Evidence for the current measurements

Configuration recorded by the run

# config.gen.yaml
listen: 127.0.0.1:8080
providers:
  mock-openai:
    api_key:
      env: BENCH_MOCK_KEY
  mock-responses:
    api_key:
      env: BENCH_MOCK_KEY
  mock-anthropic:
    api_key:
      env: BENCH_MOCK_KEY
  mock-gemini:
    api_key:
      env: BENCH_MOCK_KEY
  mock-cohere:
    api_key:
      env: BENCH_MOCK_KEY
  mock-bedrock:
    api_key:
      env: BENCH_BEDROCK_KEY
models:
  gpt-4o-mini:
    provider: mock-openai
  gpt-4o-mini-responses:
    provider: mock-responses
  gpt-4o-mini-anthropic:
    provider: mock-anthropic
  gpt-4o-mini-gemini:
    provider: mock-gemini
  gpt-4o-mini-cohere:
    provider: mock-cohere
  gpt-4o-mini-bedrock:
    provider: mock-bedrock


# providers.gen.yaml
mock-openai:
  protocol: openai
  base_url: http://127.0.0.1:8000
mock-responses:
  protocol: responses
  base_url: http://127.0.0.1:8000
mock-anthropic:
  protocol: anthropic
  base_url: http://127.0.0.1:8000
mock-gemini:
  protocol: gemini
  base_url: http://127.0.0.1:8000
mock-cohere:
  protocol: cohere
  base_url: http://127.0.0.1:8000
mock-bedrock:
  protocol: bedrock
  base_url: http://127.0.0.1:8000

Compare a shared protocol cell

Historical measurements