Bifrost benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Go · recorded build: maximhq/bifrost:v2.5.0. Measured: .

JSON · Markdown · CSV · Project source · Interactive dashboard

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

Published measurements

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
OpenAI → OpenAIAdded latency p50 (µs)658Valid · n=8,543 gateway / 269,310 direct—
OpenAI → OpenAIAdded latency p99 (µs)1,057Valid · n=8,543 gateway / 269,310 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)1,084Valid · n=8,543—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=269,310—
OpenAI → OpenAICPU per request (µs)865.1Valid—
OpenAI → OpenAIRequests per CPU-second1,155.9Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.991Valid—
OpenAI → OpenAICost window successful requests27,580Valid—
OpenAI → OpenAICost window throughput (req/s)4,595Valid—
OpenAI → OpenAICost window threads13Valid—
OpenAI → OpenAINon-voluntary context switches per request0.027Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)932Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)1,417Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)3Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)25,840.9Valid—
OpenAI → OpenAIStreams sustained580Valid—
OpenAI → OpenAIIdle RSS (MiB)160.3Valid—
OpenAI → OpenAISteady-state RSS (MiB)1,049.5Valid—
OpenAI → OpenAIRecovered RSS (MiB)1,256.6Valid—
OpenAI → OpenAIPeak RSS (MiB)1,257Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)1,137.4Valid—
OpenAI → OpenAITime to plateau (s)—Not measured—
OpenAI → OpenAIMemory growth (MiB/min)56.82Valid—
OpenAI → OpenAIMemory load duration (s)300Valid—
OpenAI → OpenAIMemory shape under load1Valid—
OpenAI → OpenAIIdle memory shape—Not measured—
OpenAI → OpenAIIdle memory static1Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)0.48Valid—
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → OpenAIThroughput (req/s)4,346concurrency 4; n=26,0815 ms
OpenAI → OpenAIThroughput (req/s)4,555concurrency 8; n=27,34010 ms
OpenAI → OpenAIThroughput (req/s)4,624concurrency 16; n=27,74850 ms
OpenAI → OpenAIThroughput (req/s)4,624concurrency 16; n=27,748100 ms
OpenAI → OpenAIThroughput (req/s)4,624concurrency 16; n=27,748Unbounded
OpenAI Responses → OpenAIAdded latency p50 (µs)586Valid · n=9,572 gateway / 263,867 direct—
OpenAI Responses → OpenAIAdded latency p99 (µs)944Valid · n=9,572 gateway / 263,867 direct—
OpenAI Responses → OpenAIGateway p99 at concurrency 1 (µs)971Valid · n=9,572—
OpenAI Responses → OpenAIDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=263,867—
OpenAI Responses → OpenAICPU per request (µs)842.7Valid—
OpenAI Responses → OpenAIRequests per CPU-second1,186.7Valid—
OpenAI Responses → OpenAICost window concurrency8Valid—
OpenAI Responses → OpenAICost window core utilisation0.993Valid—
OpenAI Responses → OpenAICost window successful requests28,386Valid—
OpenAI Responses → OpenAICost window throughput (req/s)4,730Valid—
OpenAI Responses → OpenAICost window threads13Valid—
OpenAI Responses → OpenAINon-voluntary context switches per request0.005Valid—
OpenAI Responses → OpenAICost window major page faults0Valid—
OpenAI Responses → OpenAIAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → OpenAIAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → OpenAIAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → OpenAIAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → OpenAIStreams sustained (frames/s)—Untestable—
OpenAI Responses → OpenAIStreams sustained—Untestable—
OpenAI Responses → OpenAIIdle RSS (MiB)161.2Valid—
OpenAI Responses → OpenAISteady-state RSS (MiB)1,095.2Valid—
OpenAI Responses → OpenAIRecovered RSS (MiB)1,073.7Valid—
OpenAI Responses → OpenAIPeak RSS (MiB)1,252.7Valid—
OpenAI Responses → OpenAIPeak RSS high-water mark (MiB)1,167.7Valid—
OpenAI Responses → OpenAITime to plateau (s)—Not measured—
OpenAI Responses → OpenAIMemory growth (MiB/min)73.11Valid—
OpenAI Responses → OpenAIMemory load duration (s)300Valid—
OpenAI Responses → OpenAIMemory shape under load1Valid—
OpenAI Responses → OpenAIIdle memory shape—Not measured—
OpenAI Responses → OpenAIIdle memory static1Valid—
OpenAI Responses → OpenAIIdle memory growth (MiB/min)0.355Valid—
OpenAI Responses → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI Responses → OpenAIThroughput (req/s)4,373concurrency 4; n=26,2475 ms
OpenAI Responses → OpenAIThroughput (req/s)4,576concurrency 8; n=27,46510 ms
OpenAI Responses → OpenAIThroughput (req/s)4,629concurrency 16; n=27,78250 ms
OpenAI Responses → OpenAIThroughput (req/s)4,629concurrency 16; n=27,782100 ms
OpenAI Responses → OpenAIThroughput (req/s)4,629concurrency 16; n=27,782Unbounded
Anthropic → OpenAIAdded latency p50 (µs)597Valid · n=9,413 gateway / 262,256 direct—
Anthropic → OpenAIAdded latency p99 (µs)968Valid · n=9,413 gateway / 262,256 direct—
Anthropic → OpenAIGateway p99 at concurrency 1 (µs)996Valid · n=9,413—
Anthropic → OpenAIDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=262,256—
Anthropic → OpenAICPU per request (µs)863.6Valid—
Anthropic → OpenAIRequests per CPU-second1,157.9Valid—
Anthropic → OpenAICost window concurrency8Valid—
Anthropic → OpenAICost window core utilisation0.991Valid—
Anthropic → OpenAICost window successful requests27,627Valid—
Anthropic → OpenAICost window throughput (req/s)4,603Valid—
Anthropic → OpenAICost window threads14Valid—
Anthropic → OpenAINon-voluntary context switches per request0Valid—
Anthropic → OpenAICost window major page faults0Valid—
Anthropic → OpenAIAdded time to first event p50 (µs)868Valid · n=100 gateway / 100 direct—
Anthropic → OpenAIAdded time to first event p99 (µs)1,061Valid · n=100 gateway / 100 direct—
Anthropic → OpenAIAdded inter-frame gap p50 (µs)4Valid · n=63 gateway / 63 direct—
Anthropic → OpenAIAdded inter-frame gap p99 (µs)28Valid · n=63 gateway / 63 direct—
Anthropic → OpenAIStreams sustained (frames/s)29,329.1Valid—
Anthropic → OpenAIStreams sustained678Valid—
Anthropic → OpenAIIdle RSS (MiB)161Valid—
Anthropic → OpenAISteady-state RSS (MiB)1,099.4Valid—
Anthropic → OpenAIRecovered RSS (MiB)1,073.3Valid—
Anthropic → OpenAIPeak RSS (MiB)1,300.5Valid—
Anthropic → OpenAIPeak RSS high-water mark (MiB)1,209.5Valid—
Anthropic → OpenAITime to plateau (s)—Not measured—
Anthropic → OpenAIMemory growth (MiB/min)88.45Valid—
Anthropic → OpenAIMemory load duration (s)300Valid—
Anthropic → OpenAIMemory shape under load1Valid—
Anthropic → OpenAIIdle memory shape—Not measured—
Anthropic → OpenAIIdle memory static1Valid—
Anthropic → OpenAIIdle memory growth (MiB/min)0.93Valid—
Anthropic → OpenAIThroughput (req/s)1,531concurrency 1; n=9,1921 ms
Anthropic → OpenAIThroughput (req/s)4,343concurrency 4; n=26,0625 ms
Anthropic → OpenAIThroughput (req/s)4,579concurrency 8; n=27,48110 ms
Anthropic → OpenAIThroughput (req/s)4,623concurrency 16; n=27,74350 ms
Anthropic → OpenAIThroughput (req/s)4,623concurrency 16; n=27,743100 ms
Anthropic → OpenAIThroughput (req/s)4,623concurrency 16; n=27,743Unbounded
Gemini → OpenAIAdded latency p50 (µs)588Valid · n=9,465 gateway / 259,786 direct—
Gemini → OpenAIAdded latency p99 (µs)972Valid · n=9,465 gateway / 259,786 direct—
Gemini → OpenAIGateway p99 at concurrency 1 (µs)1,000Valid · n=9,465—
Gemini → OpenAIDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=259,786—
Gemini → OpenAICPU per request (µs)846.2Valid—
Gemini → OpenAIRequests per CPU-second1,181.7Valid—
Gemini → OpenAICost window concurrency8Valid—
Gemini → OpenAICost window core utilisation0.989Valid—
Gemini → OpenAICost window successful requests28,137Valid—
Gemini → OpenAICost window throughput (req/s)4,688Valid—
Gemini → OpenAICost window threads13Valid—
Gemini → OpenAINon-voluntary context switches per request0.021Valid—
Gemini → OpenAICost window major page faults0Valid—
Gemini → OpenAIAdded time to first event p50 (µs)—Untestable—
Gemini → OpenAIAdded time to first event p99 (µs)—Untestable—
Gemini → OpenAIAdded inter-frame gap p50 (µs)—Untestable—
Gemini → OpenAIAdded inter-frame gap p99 (µs)—Untestable—
Gemini → OpenAIStreams sustained (frames/s)—Untestable—
Gemini → OpenAIStreams sustained—Untestable—
Gemini → OpenAIIdle RSS (MiB)162Valid—
Gemini → OpenAISteady-state RSS (MiB)1,091.2Valid—
Gemini → OpenAIRecovered RSS (MiB)1,074.7Valid—
Gemini → OpenAIPeak RSS (MiB)1,249.8Valid—
Gemini → OpenAIPeak RSS high-water mark (MiB)1,190.7Valid—
Gemini → OpenAITime to plateau (s)—Not measured—
Gemini → OpenAIMemory growth (MiB/min)81.08Valid—
Gemini → OpenAIMemory load duration (s)300Valid—
Gemini → OpenAIMemory shape under load1Valid—
Gemini → OpenAIIdle memory shape—Not measured—
Gemini → OpenAIIdle memory static1Valid—
Gemini → OpenAIIdle memory growth (MiB/min)0.263Valid—
Gemini → OpenAIThroughput (req/s)1,547concurrency 1; n=9,2871 ms
Gemini → OpenAIThroughput (req/s)4,450concurrency 4; n=26,7045 ms
Gemini → OpenAIThroughput (req/s)4,646concurrency 8; n=27,88410 ms
Gemini → OpenAIThroughput (req/s)4,724concurrency 16; n=28,35350 ms
Gemini → OpenAIThroughput (req/s)4,724concurrency 16; n=28,353100 ms
Gemini → OpenAIThroughput (req/s)4,724concurrency 16; n=28,353Unbounded
Cohere → OpenAIAdded latency p50 (µs)572Valid · n=9,654 gateway / 215,609 direct—
Cohere → OpenAIAdded latency p99 (µs)912Valid · n=9,654 gateway / 215,609 direct—
Cohere → OpenAIGateway p99 at concurrency 1 (µs)946Valid · n=9,654—
Cohere → OpenAIDirect-to-mock p99 at concurrency 1 (µs)34Valid · n=215,609—
Cohere → OpenAICPU per request (µs)813.6Valid—
Cohere → OpenAIRequests per CPU-second1,229.1Valid—
Cohere → OpenAICost window concurrency8Valid—
Cohere → OpenAICost window core utilisation0.988Valid—
Cohere → OpenAICost window successful requests29,229Valid—
Cohere → OpenAICost window throughput (req/s)4,870Valid—
Cohere → OpenAICost window threads13Valid—
Cohere → OpenAINon-voluntary context switches per request0.023Valid—
Cohere → OpenAICost window major page faults0Valid—
Cohere → OpenAIAdded time to first event p50 (µs)—Untestable—
Cohere → OpenAIAdded time to first event p99 (µs)—Untestable—
Cohere → OpenAIAdded inter-frame gap p50 (µs)—Untestable—
Cohere → OpenAIAdded inter-frame gap p99 (µs)—Untestable—
Cohere → OpenAIStreams sustained (frames/s)—Untestable—
Cohere → OpenAIStreams sustained—Untestable—
Cohere → OpenAIIdle RSS (MiB)158.4Valid—
Cohere → OpenAISteady-state RSS (MiB)1,049.9Valid—
Cohere → OpenAIRecovered RSS (MiB)1,259.3Valid—
Cohere → OpenAIPeak RSS (MiB)1,259.3Valid—
Cohere → OpenAIPeak RSS high-water mark (MiB)1,138.4Valid—
Cohere → OpenAITime to plateau (s)—Not measured—
Cohere → OpenAIMemory growth (MiB/min)78.13Valid—
Cohere → OpenAIMemory load duration (s)300Valid—
Cohere → OpenAIMemory shape under load1Valid—
Cohere → OpenAIIdle memory shape—Not measured—
Cohere → OpenAIIdle memory static1Valid—
Cohere → OpenAIIdle memory growth (MiB/min)0.933Valid—
Cohere → OpenAIThroughput (req/s)1,624concurrency 1; n=9,7481 ms
Cohere → OpenAIThroughput (req/s)4,853concurrency 8; n=29,1225 ms
Cohere → OpenAIThroughput (req/s)4,853concurrency 8; n=29,12210 ms
Cohere → OpenAIThroughput (req/s)4,898concurrency 16; n=29,39450 ms
Cohere → OpenAIThroughput (req/s)4,898concurrency 16; n=29,394100 ms
Cohere → OpenAIThroughput (req/s)4,898concurrency 16; n=29,394Unbounded
Bedrock → OpenAIAdded latency p50 (µs)632Valid · n=8,716 gateway / 262,374 direct—
Bedrock → OpenAIAdded latency p99 (µs)1,148Valid · n=8,716 gateway / 262,374 direct—
Bedrock → OpenAIGateway p99 at concurrency 1 (µs)1,176Valid · n=8,716—
Bedrock → OpenAIDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=262,374—
Bedrock → OpenAICPU per request (µs)909.3Valid—
Bedrock → OpenAIRequests per CPU-second1,099.8Valid—
Bedrock → OpenAICost window concurrency8Valid—
Bedrock → OpenAICost window core utilisation0.986Valid—
Bedrock → OpenAICost window successful requests26,109Valid—
Bedrock → OpenAICost window throughput (req/s)4,350Valid—
Bedrock → OpenAICost window threads13Valid—
Bedrock → OpenAINon-voluntary context switches per request0.037Valid—
Bedrock → OpenAICost window major page faults0Valid—
Bedrock → OpenAIAdded time to first event p50 (µs)—Untestable—
Bedrock → OpenAIAdded time to first event p99 (µs)—Untestable—
Bedrock → OpenAIAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → OpenAIAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → OpenAIStreams sustained (frames/s)—Untestable—
Bedrock → OpenAIStreams sustained—Untestable—
Bedrock → OpenAIIdle RSS (MiB)160.4Valid—
Bedrock → OpenAISteady-state RSS (MiB)1,042.7Valid—
Bedrock → OpenAIRecovered RSS (MiB)1,232Valid—
Bedrock → OpenAIPeak RSS (MiB)1,234.3Valid—
Bedrock → OpenAIPeak RSS high-water mark (MiB)1,146.6Valid—
Bedrock → OpenAITime to plateau (s)—Not measured—
Bedrock → OpenAIMemory growth (MiB/min)73.77Valid—
Bedrock → OpenAIMemory load duration (s)300Valid—
Bedrock → OpenAIMemory shape under load1Valid—
Bedrock → OpenAIIdle memory shape—Not measured—
Bedrock → OpenAIIdle memory static1Valid—
Bedrock → OpenAIIdle memory growth (MiB/min)0.168Valid—
Bedrock → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Bedrock → OpenAIThroughput (req/s)4,096concurrency 4; n=24,5825 ms
Bedrock → OpenAIThroughput (req/s)4,328concurrency 8; n=25,97310 ms
Bedrock → OpenAIThroughput (req/s)4,389concurrency 16; n=26,34050 ms
Bedrock → OpenAIThroughput (req/s)4,389concurrency 16; n=26,340100 ms
Bedrock → OpenAIThroughput (req/s)4,389concurrency 16; n=26,340Unbounded
OpenAI → OpenAI Responses——Not declared: recorded note 1—
OpenAI Responses → OpenAI ResponsesAdded latency p50 (µs)631Valid · n=8,878 gateway / 266,170 direct—
OpenAI Responses → OpenAI ResponsesAdded latency p99 (µs)1,001Valid · n=8,878 gateway / 266,170 direct—
OpenAI Responses → OpenAI ResponsesGateway p99 at concurrency 1 (µs)1,029Valid · n=8,878—
OpenAI Responses → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=266,170—
OpenAI Responses → OpenAI ResponsesCPU per request (µs)882.2Valid—
OpenAI Responses → OpenAI ResponsesRequests per CPU-second1,133.5Valid—
OpenAI Responses → OpenAI ResponsesCost window concurrency8Valid—
OpenAI Responses → OpenAI ResponsesCost window core utilisation0.99Valid—
OpenAI Responses → OpenAI ResponsesCost window successful requests27,011Valid—
OpenAI Responses → OpenAI ResponsesCost window throughput (req/s)4,501Valid—
OpenAI Responses → OpenAI ResponsesCost window threads15Valid—
OpenAI Responses → OpenAI ResponsesNon-voluntary context switches per request0.02Valid—
OpenAI Responses → OpenAI ResponsesCost window major page faults0Valid—
OpenAI Responses → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained—Untestable—
OpenAI Responses → OpenAI ResponsesIdle RSS (MiB)160.6Valid—
OpenAI Responses → OpenAI ResponsesSteady-state RSS (MiB)1,093.7Valid—
OpenAI Responses → OpenAI ResponsesRecovered RSS (MiB)1,288Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS (MiB)1,288.1Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS high-water mark (MiB)1,178.1Valid—
OpenAI Responses → OpenAI ResponsesTime to plateau (s)—Not measured—
OpenAI Responses → OpenAI ResponsesMemory growth (MiB/min)64.42Valid—
OpenAI Responses → OpenAI ResponsesMemory load duration (s)300Valid—
OpenAI Responses → OpenAI ResponsesMemory shape under load1Valid—
OpenAI Responses → OpenAI ResponsesIdle memory shape—Not measured—
OpenAI Responses → OpenAI ResponsesIdle memory static1Valid—
OpenAI Responses → OpenAI ResponsesIdle memory growth (MiB/min)0.224Valid—
OpenAI Responses → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)4,320concurrency 4; n=25,9245 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)4,443concurrency 8; n=26,66610 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)4,471concurrency 16; n=26,83150 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)4,471concurrency 16; n=26,831100 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)4,471concurrency 16; n=26,831Unbounded
Anthropic → OpenAI ResponsesAdded latency p50 (µs)641Valid · n=8,756 gateway / 262,163 direct—
Anthropic → OpenAI ResponsesAdded latency p99 (µs)1,011Valid · n=8,756 gateway / 262,163 direct—
Anthropic → OpenAI ResponsesGateway p99 at concurrency 1 (µs)1,038Valid · n=8,756—
Anthropic → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=262,163—
Anthropic → OpenAI ResponsesCPU per request (µs)886.3Valid—
Anthropic → OpenAI ResponsesRequests per CPU-second1,128.2Valid—
Anthropic → OpenAI ResponsesCost window concurrency8Valid—
Anthropic → OpenAI ResponsesCost window core utilisation0.989Valid—
Anthropic → OpenAI ResponsesCost window successful requests26,852Valid—
Anthropic → OpenAI ResponsesCost window throughput (req/s)4,474Valid—
Anthropic → OpenAI ResponsesCost window threads13Valid—
Anthropic → OpenAI ResponsesNon-voluntary context switches per request0.023Valid—
Anthropic → OpenAI ResponsesCost window major page faults0Valid—
Anthropic → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
Anthropic → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
Anthropic → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
Anthropic → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
Anthropic → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
Anthropic → OpenAI ResponsesStreams sustained—Untestable—
Anthropic → OpenAI ResponsesIdle RSS (MiB)164.2Valid—
Anthropic → OpenAI ResponsesSteady-state RSS (MiB)1,105.5Valid—
Anthropic → OpenAI ResponsesRecovered RSS (MiB)1,137.9Valid—
Anthropic → OpenAI ResponsesPeak RSS (MiB)1,237.3Valid—
Anthropic → OpenAI ResponsesPeak RSS high-water mark (MiB)1,209.7Valid—
Anthropic → OpenAI ResponsesTime to plateau (s)—Not measured—
Anthropic → OpenAI ResponsesMemory growth (MiB/min)83.21Valid—
Anthropic → OpenAI ResponsesMemory load duration (s)300Valid—
Anthropic → OpenAI ResponsesMemory shape under load1Valid—
Anthropic → OpenAI ResponsesIdle memory shape—Not measured—
Anthropic → OpenAI ResponsesIdle memory static1Valid—
Anthropic → OpenAI ResponsesIdle memory growth (MiB/min)0.97Valid—
Anthropic → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → OpenAI ResponsesThroughput (req/s)4,161concurrency 4; n=24,9725 ms
Anthropic → OpenAI ResponsesThroughput (req/s)4,330concurrency 8; n=25,98510 ms
Anthropic → OpenAI ResponsesThroughput (req/s)4,391concurrency 16; n=26,35750 ms
Anthropic → OpenAI ResponsesThroughput (req/s)4,391concurrency 16; n=26,357100 ms
Anthropic → OpenAI ResponsesThroughput (req/s)4,391concurrency 16; n=26,357Unbounded
Gemini → OpenAI ResponsesAdded latency p50 (µs)653Valid · n=8,613 gateway / 269,135 direct—
Gemini → OpenAI ResponsesAdded latency p99 (µs)1,008Valid · n=8,613 gateway / 269,135 direct—
Gemini → OpenAI ResponsesGateway p99 at concurrency 1 (µs)1,035Valid · n=8,613—
Gemini → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=269,135—
Gemini → OpenAI ResponsesCPU per request (µs)901.3Valid—
Gemini → OpenAI ResponsesRequests per CPU-second1,109.5Valid—
Gemini → OpenAI ResponsesCost window concurrency8Valid—
Gemini → OpenAI ResponsesCost window core utilisation0.989Valid—
Gemini → OpenAI ResponsesCost window successful requests26,417Valid—
Gemini → OpenAI ResponsesCost window throughput (req/s)4,402Valid—
Gemini → OpenAI ResponsesCost window threads14Valid—
Gemini → OpenAI ResponsesNon-voluntary context switches per request0Valid—
Gemini → OpenAI ResponsesCost window major page faults0Valid—
Gemini → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
Gemini → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
Gemini → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
Gemini → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
Gemini → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
Gemini → OpenAI ResponsesStreams sustained—Untestable—
Gemini → OpenAI ResponsesIdle RSS (MiB)159.7Valid—
Gemini → OpenAI ResponsesSteady-state RSS (MiB)1,083.4Valid—
Gemini → OpenAI ResponsesRecovered RSS (MiB)1,240Valid—
Gemini → OpenAI ResponsesPeak RSS (MiB)1,240Valid—
Gemini → OpenAI ResponsesPeak RSS high-water mark (MiB)1,183.2Valid—
Gemini → OpenAI ResponsesTime to plateau (s)—Not measured—
Gemini → OpenAI ResponsesMemory growth (MiB/min)66.32Valid—
Gemini → OpenAI ResponsesMemory load duration (s)300Valid—
Gemini → OpenAI ResponsesMemory shape under load1Valid—
Gemini → OpenAI ResponsesIdle memory shape—Not measured—
Gemini → OpenAI ResponsesIdle memory static1Valid—
Gemini → OpenAI ResponsesIdle memory growth (MiB/min)0.569Valid—
Gemini → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Gemini → OpenAI ResponsesThroughput (req/s)4,182concurrency 4; n=25,1015 ms
Gemini → OpenAI ResponsesThroughput (req/s)4,323concurrency 8; n=25,94410 ms
Gemini → OpenAI ResponsesThroughput (req/s)4,375concurrency 16; n=26,25450 ms
Gemini → OpenAI ResponsesThroughput (req/s)4,375concurrency 16; n=26,254100 ms
Gemini → OpenAI ResponsesThroughput (req/s)4,375concurrency 16; n=26,254Unbounded
Cohere → OpenAI Responses——Not declared: recorded note 1—
Bedrock → OpenAI ResponsesAdded latency p50 (µs)611Valid · n=9,156 gateway / 264,307 direct—
Bedrock → OpenAI ResponsesAdded latency p99 (µs)958Valid · n=9,156 gateway / 264,307 direct—
Bedrock → OpenAI ResponsesGateway p99 at concurrency 1 (µs)985Valid · n=9,156—
Bedrock → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=264,307—
Bedrock → OpenAI ResponsesCPU per request (µs)851.8Valid—
Bedrock → OpenAI ResponsesRequests per CPU-second1,174Valid—
Bedrock → OpenAI ResponsesCost window concurrency8Valid—
Bedrock → OpenAI ResponsesCost window core utilisation0.993Valid—
Bedrock → OpenAI ResponsesCost window successful requests28,082Valid—
Bedrock → OpenAI ResponsesCost window throughput (req/s)4,679Valid—
Bedrock → OpenAI ResponsesCost window threads14Valid—
Bedrock → OpenAI ResponsesNon-voluntary context switches per request0.044Valid—
Bedrock → OpenAI ResponsesCost window major page faults0Valid—
Bedrock → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
Bedrock → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
Bedrock → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
Bedrock → OpenAI ResponsesStreams sustained—Untestable—
Bedrock → OpenAI ResponsesIdle RSS (MiB)161.6Valid—
Bedrock → OpenAI ResponsesSteady-state RSS (MiB)1,083.2Valid—
Bedrock → OpenAI ResponsesRecovered RSS (MiB)1,295.6Valid—
Bedrock → OpenAI ResponsesPeak RSS (MiB)1,295.6Valid—
Bedrock → OpenAI ResponsesPeak RSS high-water mark (MiB)1,184Valid—
Bedrock → OpenAI ResponsesTime to plateau (s)—Not measured—
Bedrock → OpenAI ResponsesMemory growth (MiB/min)85.13Valid—
Bedrock → OpenAI ResponsesMemory load duration (s)300Valid—
Bedrock → OpenAI ResponsesMemory shape under load1Valid—
Bedrock → OpenAI ResponsesIdle memory shape—Not measured—
Bedrock → OpenAI ResponsesIdle memory static1Valid—
Bedrock → OpenAI ResponsesIdle memory growth (MiB/min)0.526Valid—
Bedrock → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Bedrock → OpenAI ResponsesThroughput (req/s)4,197concurrency 4; n=25,1885 ms
Bedrock → OpenAI ResponsesThroughput (req/s)4,442concurrency 8; n=26,65810 ms
Bedrock → OpenAI ResponsesThroughput (req/s)4,547concurrency 32; n=27,28550 ms
Bedrock → OpenAI ResponsesThroughput (req/s)4,547concurrency 32; n=27,285100 ms
Bedrock → OpenAI ResponsesThroughput (req/s)4,547concurrency 32; n=27,285Unbounded
OpenAI → AnthropicAdded latency p50 (µs)673Valid · n=8,297 gateway / 262,017 direct—
OpenAI → AnthropicAdded latency p99 (µs)1,094Valid · n=8,297 gateway / 262,017 direct—
OpenAI → AnthropicGateway p99 at concurrency 1 (µs)1,122Valid · n=8,297—
OpenAI → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=262,017—
OpenAI → AnthropicCPU per request (µs)931.3Valid—
OpenAI → AnthropicRequests per CPU-second1,073.7Valid—
OpenAI → AnthropicCost window concurrency8Valid—
OpenAI → AnthropicCost window core utilisation0.991Valid—
OpenAI → AnthropicCost window successful requests25,608Valid—
OpenAI → AnthropicCost window throughput (req/s)4,267Valid—
OpenAI → AnthropicCost window threads13Valid—
OpenAI → AnthropicNon-voluntary context switches per request0.022Valid—
OpenAI → AnthropicCost window major page faults0Valid—
OpenAI → AnthropicAdded time to first event p50 (µs)857Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded time to first event p99 (µs)1,039Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded inter-frame gap p50 (µs)7Valid · n=63 gateway / 63 direct—
OpenAI → AnthropicAdded inter-frame gap p99 (µs)25Valid · n=63 gateway / 63 direct—
OpenAI → AnthropicStreams sustained (frames/s)26,538Valid—
OpenAI → AnthropicStreams sustained624Valid—
OpenAI → AnthropicIdle RSS (MiB)162.3Valid—
OpenAI → AnthropicSteady-state RSS (MiB)1,067.1Valid—
OpenAI → AnthropicRecovered RSS (MiB)1,319.3Valid—
OpenAI → AnthropicPeak RSS (MiB)1,319.3Valid—
OpenAI → AnthropicPeak RSS high-water mark (MiB)1,199.9Valid—
OpenAI → AnthropicTime to plateau (s)—Not measured—
OpenAI → AnthropicMemory growth (MiB/min)63.29Valid—
OpenAI → AnthropicMemory load duration (s)300Valid—
OpenAI → AnthropicMemory shape under load1Valid—
OpenAI → AnthropicIdle memory shape—Not measured—
OpenAI → AnthropicIdle memory static1Valid—
OpenAI → AnthropicIdle memory growth (MiB/min)0.634Valid—
OpenAI → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → AnthropicThroughput (req/s)4,001concurrency 4; n=24,0145 ms
OpenAI → AnthropicThroughput (req/s)4,117concurrency 8; n=24,71010 ms
OpenAI → AnthropicThroughput (req/s)4,177concurrency 32; n=25,06750 ms
OpenAI → AnthropicThroughput (req/s)4,177concurrency 32; n=25,067100 ms
OpenAI → AnthropicThroughput (req/s)4,177concurrency 32; n=25,067Unbounded
OpenAI Responses → AnthropicAdded latency p50 (µs)770Valid · n=7,332 gateway / 263,386 direct—
OpenAI Responses → AnthropicAdded latency p99 (µs)1,259Valid · n=7,332 gateway / 263,386 direct—
OpenAI Responses → AnthropicGateway p99 at concurrency 1 (µs)1,287Valid · n=7,332—
OpenAI Responses → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=263,386—
OpenAI Responses → AnthropicCPU per request (µs)1,057.1Valid—
OpenAI Responses → AnthropicRequests per CPU-second946Valid—
OpenAI Responses → AnthropicCost window concurrency8Valid—
OpenAI Responses → AnthropicCost window core utilisation0.986Valid—
OpenAI Responses → AnthropicCost window successful requests22,448Valid—
OpenAI Responses → AnthropicCost window throughput (req/s)3,740Valid—
OpenAI Responses → AnthropicCost window threads13Valid—
OpenAI Responses → AnthropicNon-voluntary context switches per request0Valid—
OpenAI Responses → AnthropicCost window major page faults0Valid—
OpenAI Responses → AnthropicAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → AnthropicAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → AnthropicAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → AnthropicAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → AnthropicStreams sustained (frames/s)—Untestable—
OpenAI Responses → AnthropicStreams sustained—Untestable—
OpenAI Responses → AnthropicIdle RSS (MiB)159.1Valid—
OpenAI Responses → AnthropicSteady-state RSS (MiB)984.9Valid—
OpenAI Responses → AnthropicRecovered RSS (MiB)1,051Valid—
OpenAI Responses → AnthropicPeak RSS (MiB)1,220.6Valid—
OpenAI Responses → AnthropicPeak RSS high-water mark (MiB)1,174.6Valid—
OpenAI Responses → AnthropicTime to plateau (s)—Not measured—
OpenAI Responses → AnthropicMemory growth (MiB/min)168.5Valid—
OpenAI Responses → AnthropicMemory load duration (s)300Valid—
OpenAI Responses → AnthropicMemory shape under load1Valid—
OpenAI Responses → AnthropicIdle memory shape—Not measured—
OpenAI Responses → AnthropicIdle memory static1Valid—
OpenAI Responses → AnthropicIdle memory growth (MiB/min)0.47Valid—
OpenAI Responses → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI Responses → AnthropicThroughput (req/s)3,458concurrency 4; n=20,7605 ms
OpenAI Responses → AnthropicThroughput (req/s)3,703concurrency 8; n=22,22410 ms
OpenAI Responses → AnthropicThroughput (req/s)3,703concurrency 8; n=22,22450 ms
OpenAI Responses → AnthropicThroughput (req/s)3,703concurrency 8; n=22,224100 ms
OpenAI Responses → AnthropicThroughput (req/s)3,703concurrency 8; n=22,224Unbounded
Anthropic → AnthropicAdded latency p50 (µs)678Valid · n=8,283 gateway / 258,757 direct—
Anthropic → AnthropicAdded latency p99 (µs)1,101Valid · n=8,283 gateway / 258,757 direct—
Anthropic → AnthropicGateway p99 at concurrency 1 (µs)1,129Valid · n=8,283—
Anthropic → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=258,757—
Anthropic → AnthropicCPU per request (µs)1,001.9Valid—
Anthropic → AnthropicRequests per CPU-second998.1Valid—
Anthropic → AnthropicCost window concurrency8Valid—
Anthropic → AnthropicCost window core utilisation0.995Valid—
Anthropic → AnthropicCost window successful requests23,914Valid—
Anthropic → AnthropicCost window throughput (req/s)3,985Valid—
Anthropic → AnthropicCost window threads13Valid—
Anthropic → AnthropicNon-voluntary context switches per request0.037Valid—
Anthropic → AnthropicCost window major page faults0Valid—
Anthropic → AnthropicAdded time to first event p50 (µs)826Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded time to first event p99 (µs)924Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → AnthropicAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → AnthropicStreams sustained (frames/s)28,037.5Valid—
Anthropic → AnthropicStreams sustained683Valid—
Anthropic → AnthropicIdle RSS (MiB)161.9Valid—
Anthropic → AnthropicSteady-state RSS (MiB)1,059.2Valid—
Anthropic → AnthropicRecovered RSS (MiB)1,112.5Valid—
Anthropic → AnthropicPeak RSS (MiB)1,274.2Valid—
Anthropic → AnthropicPeak RSS high-water mark (MiB)1,215.4Valid—
Anthropic → AnthropicTime to plateau (s)—Not measured—
Anthropic → AnthropicMemory growth (MiB/min)80.73Valid—
Anthropic → AnthropicMemory load duration (s)300Valid—
Anthropic → AnthropicMemory shape under load1Valid—
Anthropic → AnthropicIdle memory shape—Not measured—
Anthropic → AnthropicIdle memory static1Valid—
Anthropic → AnthropicIdle memory growth (MiB/min)0.715Valid—
Anthropic → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → AnthropicThroughput (req/s)3,861concurrency 4; n=23,1745 ms
Anthropic → AnthropicThroughput (req/s)3,988concurrency 8; n=23,93110 ms
Anthropic → AnthropicThroughput (req/s)4,035concurrency 16; n=24,21650 ms
Anthropic → AnthropicThroughput (req/s)4,035concurrency 16; n=24,216100 ms
Anthropic → AnthropicThroughput (req/s)4,035concurrency 16; n=24,216Unbounded
Gemini → AnthropicAdded latency p50 (µs)693Valid · n=8,044 gateway / 259,231 direct—
Gemini → AnthropicAdded latency p99 (µs)1,153Valid · n=8,044 gateway / 259,231 direct—
Gemini → AnthropicGateway p99 at concurrency 1 (µs)1,181Valid · n=8,044—
Gemini → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=259,231—
Gemini → AnthropicCPU per request (µs)999Valid—
Gemini → AnthropicRequests per CPU-second1,001Valid—
Gemini → AnthropicCost window concurrency8Valid—
Gemini → AnthropicCost window core utilisation0.994Valid—
Gemini → AnthropicCost window successful requests23,954Valid—
Gemini → AnthropicCost window throughput (req/s)3,991Valid—
Gemini → AnthropicCost window threads16Valid—
Gemini → AnthropicNon-voluntary context switches per request0Valid—
Gemini → AnthropicCost window major page faults0Valid—
Gemini → AnthropicAdded time to first event p50 (µs)—Untestable—
Gemini → AnthropicAdded time to first event p99 (µs)—Untestable—
Gemini → AnthropicAdded inter-frame gap p50 (µs)—Untestable—
Gemini → AnthropicAdded inter-frame gap p99 (µs)—Untestable—
Gemini → AnthropicStreams sustained (frames/s)—Untestable—
Gemini → AnthropicStreams sustained—Untestable—
Gemini → AnthropicIdle RSS (MiB)161.1Valid—
Gemini → AnthropicSteady-state RSS (MiB)1,047.8Valid—
Gemini → AnthropicRecovered RSS (MiB)1,109.1Valid—
Gemini → AnthropicPeak RSS (MiB)1,249.8Valid—
Gemini → AnthropicPeak RSS high-water mark (MiB)1,215.3Valid—
Gemini → AnthropicTime to plateau (s)—Not measured—
Gemini → AnthropicMemory growth (MiB/min)96.98Valid—
Gemini → AnthropicMemory load duration (s)300Valid—
Gemini → AnthropicMemory shape under load1Valid—
Gemini → AnthropicIdle memory shape—Not measured—
Gemini → AnthropicIdle memory static1Valid—
Gemini → AnthropicIdle memory growth (MiB/min)0.212Valid—
Gemini → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Gemini → AnthropicThroughput (req/s)3,872concurrency 4; n=23,2395 ms
Gemini → AnthropicThroughput (req/s)3,995concurrency 8; n=23,97610 ms
Gemini → AnthropicThroughput (req/s)4,040concurrency 16; n=24,24950 ms
Gemini → AnthropicThroughput (req/s)4,040concurrency 16; n=24,249100 ms
Gemini → AnthropicThroughput (req/s)4,040concurrency 16; n=24,249Unbounded
Cohere → AnthropicAdded latency p50 (µs)685Valid · n=8,100 gateway / 245,712 direct—
Cohere → AnthropicAdded latency p99 (µs)1,201Valid · n=8,100 gateway / 245,712 direct—
Cohere → AnthropicGateway p99 at concurrency 1 (µs)1,230Valid · n=8,100—
Cohere → AnthropicDirect-to-mock p99 at concurrency 1 (µs)29Valid · n=245,712—
Cohere → AnthropicCPU per request (µs)946.2Valid—
Cohere → AnthropicRequests per CPU-second1,056.9Valid—
Cohere → AnthropicCost window concurrency8Valid—
Cohere → AnthropicCost window core utilisation0.991Valid—
Cohere → AnthropicCost window successful requests25,217Valid—
Cohere → AnthropicCost window throughput (req/s)4,202Valid—
Cohere → AnthropicCost window threads13Valid—
Cohere → AnthropicNon-voluntary context switches per request0.02Valid—
Cohere → AnthropicCost window major page faults0Valid—
Cohere → AnthropicAdded time to first event p50 (µs)—Untestable—
Cohere → AnthropicAdded time to first event p99 (µs)—Untestable—
Cohere → AnthropicAdded inter-frame gap p50 (µs)—Untestable—
Cohere → AnthropicAdded inter-frame gap p99 (µs)—Untestable—
Cohere → AnthropicStreams sustained (frames/s)—Untestable—
Cohere → AnthropicStreams sustained—Untestable—
Cohere → AnthropicIdle RSS (MiB)163Valid—
Cohere → AnthropicSteady-state RSS (MiB)1,019.6Valid—
Cohere → AnthropicRecovered RSS (MiB)961.4Valid—
Cohere → AnthropicPeak RSS (MiB)1,261.3Valid—
Cohere → AnthropicPeak RSS high-water mark (MiB)1,159Valid—
Cohere → AnthropicTime to plateau (s)—Not measured—
Cohere → AnthropicMemory growth (MiB/min)69.15Valid—
Cohere → AnthropicMemory load duration (s)300Valid—
Cohere → AnthropicMemory shape under load1Valid—
Cohere → AnthropicIdle memory shape—Not measured—
Cohere → AnthropicIdle memory static1Valid—
Cohere → AnthropicIdle memory growth (MiB/min)0.268Valid—
Cohere → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Cohere → AnthropicThroughput (req/s)3,987concurrency 4; n=23,9275 ms
Cohere → AnthropicThroughput (req/s)4,149concurrency 8; n=24,90210 ms
Cohere → AnthropicThroughput (req/s)4,224concurrency 16; n=25,34950 ms
Cohere → AnthropicThroughput (req/s)4,224concurrency 16; n=25,349100 ms
Cohere → AnthropicThroughput (req/s)4,224concurrency 16; n=25,349Unbounded
Bedrock → AnthropicAdded latency p50 (µs)620Valid · n=8,987 gateway / 265,983 direct—
Bedrock → AnthropicAdded latency p99 (µs)1,030Valid · n=8,987 gateway / 265,983 direct—
Bedrock → AnthropicGateway p99 at concurrency 1 (µs)1,057Valid · n=8,987—
Bedrock → AnthropicDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=265,983—
Bedrock → AnthropicCPU per request (µs)869.5Valid—
Bedrock → AnthropicRequests per CPU-second1,150.1Valid—
Bedrock → AnthropicCost window concurrency8Valid—
Bedrock → AnthropicCost window core utilisation0.991Valid—
Bedrock → AnthropicCost window successful requests27,453Valid—
Bedrock → AnthropicCost window throughput (req/s)4,574Valid—
Bedrock → AnthropicCost window threads13Valid—
Bedrock → AnthropicNon-voluntary context switches per request0Valid—
Bedrock → AnthropicCost window major page faults0Valid—
Bedrock → AnthropicAdded time to first event p50 (µs)—Untestable—
Bedrock → AnthropicAdded time to first event p99 (µs)—Untestable—
Bedrock → AnthropicAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → AnthropicAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → AnthropicStreams sustained (frames/s)—Untestable—
Bedrock → AnthropicStreams sustained—Untestable—
Bedrock → AnthropicIdle RSS (MiB)160.3Valid—
Bedrock → AnthropicSteady-state RSS (MiB)1,059.3Valid—
Bedrock → AnthropicRecovered RSS (MiB)1,284Valid—
Bedrock → AnthropicPeak RSS (MiB)1,284Valid—
Bedrock → AnthropicPeak RSS high-water mark (MiB)1,156Valid—
Bedrock → AnthropicTime to plateau (s)—Not measured—
Bedrock → AnthropicMemory growth (MiB/min)44.08Valid—
Bedrock → AnthropicMemory load duration (s)300Valid—
Bedrock → AnthropicMemory shape under load1Valid—
Bedrock → AnthropicIdle memory shape—Not measured—
Bedrock → AnthropicIdle memory static1Valid—
Bedrock → AnthropicIdle memory growth (MiB/min)0.56Valid—
Bedrock → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Bedrock → AnthropicThroughput (req/s)4,524concurrency 8; n=27,1505 ms
Bedrock → AnthropicThroughput (req/s)4,524concurrency 8; n=27,15010 ms
Bedrock → AnthropicThroughput (req/s)4,633concurrency 16; n=27,80750 ms
Bedrock → AnthropicThroughput (req/s)4,633concurrency 16; n=27,807100 ms
Bedrock → AnthropicThroughput (req/s)4,633concurrency 16; n=27,807Unbounded
OpenAI → GeminiAdded latency p50 (µs)626Valid · n=8,969 gateway / 262,364 direct—
OpenAI → GeminiAdded latency p99 (µs)1,018Valid · n=8,969 gateway / 262,364 direct—
OpenAI → GeminiGateway p99 at concurrency 1 (µs)1,046Valid · n=8,969—
OpenAI → GeminiDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=262,364—
OpenAI → GeminiCPU per request (µs)913.8Valid—
OpenAI → GeminiRequests per CPU-second1,094.3Valid—
OpenAI → GeminiCost window concurrency8Valid—
OpenAI → GeminiCost window core utilisation0.988Valid—
OpenAI → GeminiCost window successful requests26,034Valid—
OpenAI → GeminiCost window throughput (req/s)4,338Valid—
OpenAI → GeminiCost window threads14Valid—
OpenAI → GeminiNon-voluntary context switches per request0Valid—
OpenAI → GeminiCost window major page faults0Valid—
OpenAI → GeminiAdded time to first event p50 (µs)—Untestable—
OpenAI → GeminiAdded time to first event p99 (µs)—Untestable—
OpenAI → GeminiAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → GeminiAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → GeminiStreams sustained (frames/s)—Untestable—
OpenAI → GeminiStreams sustained—Untestable—
OpenAI → GeminiIdle RSS (MiB)162Valid—
OpenAI → GeminiSteady-state RSS (MiB)1,093.6Valid—
OpenAI → GeminiRecovered RSS (MiB)1,193.6Valid—
OpenAI → GeminiPeak RSS (MiB)1,199.4Valid—
OpenAI → GeminiPeak RSS high-water mark (MiB)1,200.1Valid—
OpenAI → GeminiTime to plateau (s)—Not measured—
OpenAI → GeminiMemory growth (MiB/min)73.3Valid—
OpenAI → GeminiMemory load duration (s)300Valid—
OpenAI → GeminiMemory shape under load1Valid—
OpenAI → GeminiIdle memory shape—Not measured—
OpenAI → GeminiIdle memory static1Valid—
OpenAI → GeminiIdle memory growth (MiB/min)0.381Valid—
OpenAI → GeminiThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → GeminiThroughput (req/s)4,251concurrency 4; n=25,5145 ms
OpenAI → GeminiThroughput (req/s)4,415concurrency 8; n=26,49710 ms
OpenAI → GeminiThroughput (req/s)4,467concurrency 16; n=26,80950 ms
OpenAI → GeminiThroughput (req/s)4,467concurrency 16; n=26,809100 ms
OpenAI → GeminiThroughput (req/s)4,467concurrency 16; n=26,809Unbounded
OpenAI Responses → GeminiAdded latency p50 (µs)667Valid · n=8,355 gateway / 244,434 direct—
OpenAI Responses → GeminiAdded latency p99 (µs)1,139Valid · n=8,355 gateway / 244,434 direct—
OpenAI Responses → GeminiGateway p99 at concurrency 1 (µs)1,169Valid · n=8,355—
OpenAI Responses → GeminiDirect-to-mock p99 at concurrency 1 (µs)30Valid · n=244,434—
OpenAI Responses → GeminiCPU per request (µs)958.6Valid—
OpenAI Responses → GeminiRequests per CPU-second1,043.2Valid—
OpenAI Responses → GeminiCost window concurrency8Valid—
OpenAI Responses → GeminiCost window core utilisation0.991Valid—
OpenAI Responses → GeminiCost window successful requests24,891Valid—
OpenAI Responses → GeminiCost window throughput (req/s)4,147Valid—
OpenAI Responses → GeminiCost window threads14Valid—
OpenAI Responses → GeminiNon-voluntary context switches per request0Valid—
OpenAI Responses → GeminiCost window major page faults0Valid—
OpenAI Responses → GeminiAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → GeminiAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → GeminiAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → GeminiAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → GeminiStreams sustained (frames/s)—Untestable—
OpenAI Responses → GeminiStreams sustained—Untestable—
OpenAI Responses → GeminiIdle RSS (MiB)161Valid—
OpenAI Responses → GeminiSteady-state RSS (MiB)1,065.3Valid—
OpenAI Responses → GeminiRecovered RSS (MiB)1,112.7Valid—
OpenAI Responses → GeminiPeak RSS (MiB)1,287.9Valid—
OpenAI Responses → GeminiPeak RSS high-water mark (MiB)1,203.4Valid—
OpenAI Responses → GeminiTime to plateau (s)—Not measured—
OpenAI Responses → GeminiMemory growth (MiB/min)57.62Valid—
OpenAI Responses → GeminiMemory load duration (s)300Valid—
OpenAI Responses → GeminiMemory shape under load1Valid—
OpenAI Responses → GeminiIdle memory shape—Not measured—
OpenAI Responses → GeminiIdle memory static1Valid—
OpenAI Responses → GeminiIdle memory growth (MiB/min)0.163Valid—
OpenAI Responses → GeminiThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI Responses → GeminiThroughput (req/s)4,033concurrency 4; n=24,2115 ms
OpenAI Responses → GeminiThroughput (req/s)4,174concurrency 8; n=25,04810 ms
OpenAI Responses → GeminiThroughput (req/s)4,222concurrency 16; n=25,33550 ms
OpenAI Responses → GeminiThroughput (req/s)4,222concurrency 16; n=25,335100 ms
OpenAI Responses → GeminiThroughput (req/s)4,222concurrency 16; n=25,335Unbounded
Anthropic → GeminiAdded latency p50 (µs)634Valid · n=8,607 gateway / 257,441 direct—
Anthropic → GeminiAdded latency p99 (µs)1,125Valid · n=8,607 gateway / 257,441 direct—
Anthropic → GeminiGateway p99 at concurrency 1 (µs)1,153Valid · n=8,607—
Anthropic → GeminiDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=257,441—
Anthropic → GeminiCPU per request (µs)926.3Valid—
Anthropic → GeminiRequests per CPU-second1,079.6Valid—
Anthropic → GeminiCost window concurrency8Valid—
Anthropic → GeminiCost window core utilisation0.986Valid—
Anthropic → GeminiCost window successful requests25,640Valid—
Anthropic → GeminiCost window throughput (req/s)4,272Valid—
Anthropic → GeminiCost window threads14Valid—
Anthropic → GeminiNon-voluntary context switches per request0.014Valid—
Anthropic → GeminiCost window major page faults0Valid—
Anthropic → GeminiAdded time to first event p50 (µs)—Untestable—
Anthropic → GeminiAdded time to first event p99 (µs)—Untestable—
Anthropic → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Anthropic → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Anthropic → GeminiStreams sustained (frames/s)—Untestable—
Anthropic → GeminiStreams sustained—Untestable—
Anthropic → GeminiIdle RSS (MiB)160.8Valid—
Anthropic → GeminiSteady-state RSS (MiB)1,083.5Valid—
Anthropic → GeminiRecovered RSS (MiB)1,179.1Valid—
Anthropic → GeminiPeak RSS (MiB)1,239.1Valid—
Anthropic → GeminiPeak RSS high-water mark (MiB)1,228Valid—
Anthropic → GeminiTime to plateau (s)—Not measured—
Anthropic → GeminiMemory growth (MiB/min)81.53Valid—
Anthropic → GeminiMemory load duration (s)300Valid—
Anthropic → GeminiMemory shape under load1Valid—
Anthropic → GeminiIdle memory shape—Not measured—
Anthropic → GeminiIdle memory static1Valid—
Anthropic → GeminiIdle memory growth (MiB/min)0.711Valid—
Anthropic → GeminiThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → GeminiThroughput (req/s)4,055concurrency 4; n=24,3345 ms
Anthropic → GeminiThroughput (req/s)4,182concurrency 8; n=25,10110 ms
Anthropic → GeminiThroughput (req/s)4,257concurrency 16; n=25,54750 ms
Anthropic → GeminiThroughput (req/s)4,257concurrency 16; n=25,547100 ms
Anthropic → GeminiThroughput (req/s)4,257concurrency 16; n=25,547Unbounded
Gemini → GeminiAdded latency p50 (µs)652Valid · n=8,589 gateway / 266,124 direct—
Gemini → GeminiAdded latency p99 (µs)1,098Valid · n=8,589 gateway / 266,124 direct—
Gemini → GeminiGateway p99 at concurrency 1 (µs)1,125Valid · n=8,589—
Gemini → GeminiDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=266,124—
Gemini → GeminiCPU per request (µs)954.9Valid—
Gemini → GeminiRequests per CPU-second1,047.2Valid—
Gemini → GeminiCost window concurrency8Valid—
Gemini → GeminiCost window core utilisation0.987Valid—
Gemini → GeminiCost window successful requests24,871Valid—
Gemini → GeminiCost window throughput (req/s)4,144Valid—
Gemini → GeminiCost window threads14Valid—
Gemini → GeminiNon-voluntary context switches per request0Valid—
Gemini → GeminiCost window major page faults0Valid—
Gemini → GeminiAdded time to first event p50 (µs)—Untestable—
Gemini → GeminiAdded time to first event p99 (µs)—Untestable—
Gemini → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Gemini → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Gemini → GeminiStreams sustained (frames/s)—Untestable—
Gemini → GeminiStreams sustained—Untestable—
Gemini → GeminiIdle RSS (MiB)161.5Valid—
Gemini → GeminiSteady-state RSS (MiB)1,066.1Valid—
Gemini → GeminiRecovered RSS (MiB)863.8Valid—
Gemini → GeminiPeak RSS (MiB)1,200.6Valid—
Gemini → GeminiPeak RSS high-water mark (MiB)1,200.7Valid—
Gemini → GeminiTime to plateau (s)—Not measured—
Gemini → GeminiMemory growth (MiB/min)86.41Valid—
Gemini → GeminiMemory load duration (s)300Valid—
Gemini → GeminiMemory shape under load1Valid—
Gemini → GeminiIdle memory shape—Not measured—
Gemini → GeminiIdle memory static1Valid—
Gemini → GeminiIdle memory growth (MiB/min)0.651Valid—
Gemini → GeminiThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Gemini → GeminiThroughput (req/s)3,959concurrency 4; n=23,7625 ms
Gemini → GeminiThroughput (req/s)4,109concurrency 8; n=24,66110 ms
Gemini → GeminiThroughput (req/s)4,162concurrency 16; n=24,97750 ms
Gemini → GeminiThroughput (req/s)4,162concurrency 16; n=24,977100 ms
Gemini → GeminiThroughput (req/s)4,162concurrency 16; n=24,977Unbounded
Cohere → GeminiAdded latency p50 (µs)658Valid · n=8,497 gateway / 243,412 direct—
Cohere → GeminiAdded latency p99 (µs)1,090Valid · n=8,497 gateway / 243,412 direct—
Cohere → GeminiGateway p99 at concurrency 1 (µs)1,120Valid · n=8,497—
Cohere → GeminiDirect-to-mock p99 at concurrency 1 (µs)30Valid · n=243,412—
Cohere → GeminiCPU per request (µs)953.7Valid—
Cohere → GeminiRequests per CPU-second1,048.6Valid—
Cohere → GeminiCost window concurrency8Valid—
Cohere → GeminiCost window core utilisation0.989Valid—
Cohere → GeminiCost window successful requests24,966Valid—
Cohere → GeminiCost window throughput (req/s)4,160Valid—
Cohere → GeminiCost window threads15Valid—
Cohere → GeminiNon-voluntary context switches per request0Valid—
Cohere → GeminiCost window major page faults0Valid—
Cohere → GeminiAdded time to first event p50 (µs)—Untestable—
Cohere → GeminiAdded time to first event p99 (µs)—Untestable—
Cohere → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Cohere → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Cohere → GeminiStreams sustained (frames/s)—Untestable—
Cohere → GeminiStreams sustained—Untestable—
Cohere → GeminiIdle RSS (MiB)160.1Valid—
Cohere → GeminiSteady-state RSS (MiB)986.2Valid—
Cohere → GeminiRecovered RSS (MiB)1,046.5Valid—
Cohere → GeminiPeak RSS (MiB)1,179.5Valid—
Cohere → GeminiPeak RSS high-water mark (MiB)1,142.3Valid—
Cohere → GeminiTime to plateau (s)—Not measured—
Cohere → GeminiMemory growth (MiB/min)69.08Valid—
Cohere → GeminiMemory load duration (s)300Valid—
Cohere → GeminiMemory shape under load1Valid—
Cohere → GeminiIdle memory shape—Not measured—
Cohere → GeminiIdle memory static1Valid—
Cohere → GeminiIdle memory growth (MiB/min)0.439Valid—
Cohere → GeminiThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Cohere → GeminiThroughput (req/s)4,011concurrency 4; n=24,0725 ms
Cohere → GeminiThroughput (req/s)4,196concurrency 8; n=25,18110 ms
Cohere → GeminiThroughput (req/s)4,233concurrency 16; n=25,40050 ms
Cohere → GeminiThroughput (req/s)4,233concurrency 16; n=25,400100 ms
Cohere → GeminiThroughput (req/s)4,233concurrency 16; n=25,400Unbounded
Bedrock → GeminiAdded latency p50 (µs)607Valid · n=9,211 gateway / 248,434 direct—
Bedrock → GeminiAdded latency p99 (µs)1,009Valid · n=9,211 gateway / 248,434 direct—
Bedrock → GeminiGateway p99 at concurrency 1 (µs)1,038Valid · n=9,211—
Bedrock → GeminiDirect-to-mock p99 at concurrency 1 (µs)29Valid · n=248,434—
Bedrock → GeminiCPU per request (µs)898Valid—
Bedrock → GeminiRequests per CPU-second1,113.6Valid—
Bedrock → GeminiCost window concurrency8Valid—
Bedrock → GeminiCost window core utilisation0.993Valid—
Bedrock → GeminiCost window successful requests26,614Valid—
Bedrock → GeminiCost window throughput (req/s)4,435Valid—
Bedrock → GeminiCost window threads13Valid—
Bedrock → GeminiNon-voluntary context switches per request0Valid—
Bedrock → GeminiCost window major page faults0Valid—
Bedrock → GeminiAdded time to first event p50 (µs)—Untestable—
Bedrock → GeminiAdded time to first event p99 (µs)—Untestable—
Bedrock → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → GeminiStreams sustained (frames/s)—Untestable—
Bedrock → GeminiStreams sustained—Untestable—
Bedrock → GeminiIdle RSS (MiB)160.7Valid—
Bedrock → GeminiSteady-state RSS (MiB)1,057.9Valid—
Bedrock → GeminiRecovered RSS (MiB)928.5Valid—
Bedrock → GeminiPeak RSS (MiB)1,270.6Valid—
Bedrock → GeminiPeak RSS high-water mark (MiB)1,157.7Valid—
Bedrock → GeminiTime to plateau (s)—Not measured—
Bedrock → GeminiMemory growth (MiB/min)64.94Valid—
Bedrock → GeminiMemory load duration (s)300Valid—
Bedrock → GeminiMemory shape under load1Valid—
Bedrock → GeminiIdle memory shape—Not measured—
Bedrock → GeminiIdle memory static1Valid—
Bedrock → GeminiIdle memory growth (MiB/min)0.335Valid—
Bedrock → GeminiThroughput (req/s)1,551concurrency 1; n=9,3101 ms
Bedrock → GeminiThroughput (req/s)4,311concurrency 4; n=25,8715 ms
Bedrock → GeminiThroughput (req/s)4,413concurrency 8; n=26,48210 ms
Bedrock → GeminiThroughput (req/s)4,603concurrency 16; n=27,62450 ms
Bedrock → GeminiThroughput (req/s)4,603concurrency 16; n=27,624100 ms
Bedrock → GeminiThroughput (req/s)4,603concurrency 16; n=27,624Unbounded
OpenAI → CohereAdded latency p50 (µs)641Valid · n=8,677 gateway / 263,654 direct—
OpenAI → CohereAdded latency p99 (µs)1,090Valid · n=8,677 gateway / 263,654 direct—
OpenAI → CohereGateway p99 at concurrency 1 (µs)1,118Valid · n=8,677—
OpenAI → CohereDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=263,654—
OpenAI → CohereCPU per request (µs)900.1Valid—
OpenAI → CohereRequests per CPU-second1,111Valid—
OpenAI → CohereCost window concurrency8Valid—
OpenAI → CohereCost window core utilisation0.991Valid—
OpenAI → CohereCost window successful requests26,498Valid—
OpenAI → CohereCost window throughput (req/s)4,415Valid—
OpenAI → CohereCost window threads14Valid—
OpenAI → CohereNon-voluntary context switches per request0Valid—
OpenAI → CohereCost window major page faults0Valid—
OpenAI → CohereAdded time to first event p50 (µs)—Untestable—
OpenAI → CohereAdded time to first event p99 (µs)—Untestable—
OpenAI → CohereAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → CohereAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → CohereStreams sustained (frames/s)—Untestable—
OpenAI → CohereStreams sustained—Untestable—
OpenAI → CohereIdle RSS (MiB)161.4Valid—
OpenAI → CohereSteady-state RSS (MiB)1,119.1Valid—
OpenAI → CohereRecovered RSS (MiB)1,071.8Valid—
OpenAI → CoherePeak RSS (MiB)1,254.9Valid—
OpenAI → CoherePeak RSS high-water mark (MiB)1,211.2Valid—
OpenAI → CohereTime to plateau (s)—Not measured—
OpenAI → CohereMemory growth (MiB/min)74.02Valid—
OpenAI → CohereMemory load duration (s)300Valid—
OpenAI → CohereMemory shape under load1Valid—
OpenAI → CohereIdle memory shape—Not measured—
OpenAI → CohereIdle memory static1Valid—
OpenAI → CohereIdle memory growth (MiB/min)0.156Valid—
OpenAI → CohereThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → CohereThroughput (req/s)4,074concurrency 4; n=24,4535 ms
OpenAI → CohereThroughput (req/s)4,339concurrency 8; n=26,03910 ms
OpenAI → CohereThroughput (req/s)4,507concurrency 32; n=27,04850 ms
OpenAI → CohereThroughput (req/s)4,507concurrency 32; n=27,048100 ms
OpenAI → CohereThroughput (req/s)4,507concurrency 32; n=27,048Unbounded
OpenAI Responses → CohereAdded latency p50 (µs)630Valid · n=8,900 gateway / 251,031 direct—
OpenAI Responses → CohereAdded latency p99 (µs)1,032Valid · n=8,900 gateway / 251,031 direct—
OpenAI Responses → CohereGateway p99 at concurrency 1 (µs)1,061Valid · n=8,900—
OpenAI Responses → CohereDirect-to-mock p99 at concurrency 1 (µs)29Valid · n=251,031—
OpenAI Responses → CohereCPU per request (µs)907.9Valid—
OpenAI Responses → CohereRequests per CPU-second1,101.4Valid—
OpenAI Responses → CohereCost window concurrency8Valid—
OpenAI Responses → CohereCost window core utilisation0.987Valid—
OpenAI Responses → CohereCost window successful requests26,169Valid—
OpenAI Responses → CohereCost window throughput (req/s)4,360Valid—
OpenAI Responses → CohereCost window threads14Valid—
OpenAI Responses → CohereNon-voluntary context switches per request0.037Valid—
OpenAI Responses → CohereCost window major page faults0Valid—
OpenAI Responses → CohereAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → CohereAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → CohereAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → CohereAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → CohereStreams sustained (frames/s)—Untestable—
OpenAI Responses → CohereStreams sustained—Untestable—
OpenAI Responses → CohereIdle RSS (MiB)161.4Valid—
OpenAI Responses → CohereSteady-state RSS (MiB)1,074.9Valid—
OpenAI Responses → CohereRecovered RSS (MiB)1,037.1Valid—
OpenAI Responses → CoherePeak RSS (MiB)1,256.9Valid—
OpenAI Responses → CoherePeak RSS high-water mark (MiB)1,184.6Valid—
OpenAI Responses → CohereTime to plateau (s)—Not measured—
OpenAI Responses → CohereMemory growth (MiB/min)79.79Valid—
OpenAI Responses → CohereMemory load duration (s)300Valid—
OpenAI Responses → CohereMemory shape under load1Valid—
OpenAI Responses → CohereIdle memory shape—Not measured—
OpenAI Responses → CohereIdle memory static1Valid—
OpenAI Responses → CohereIdle memory growth (MiB/min)0.51Valid—
OpenAI Responses → CohereThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI Responses → CohereThroughput (req/s)4,135concurrency 4; n=24,8165 ms
OpenAI Responses → CohereThroughput (req/s)4,290concurrency 8; n=25,75110 ms
OpenAI Responses → CohereThroughput (req/s)4,330concurrency 16; n=25,98850 ms
OpenAI Responses → CohereThroughput (req/s)4,330concurrency 16; n=25,988100 ms
OpenAI Responses → CohereThroughput (req/s)4,330concurrency 16; n=25,988Unbounded
Anthropic → CohereAdded latency p50 (µs)608Valid · n=9,106 gateway / 260,492 direct—
Anthropic → CohereAdded latency p99 (µs)1,061Valid · n=9,106 gateway / 260,492 direct—
Anthropic → CohereGateway p99 at concurrency 1 (µs)1,089Valid · n=9,106—
Anthropic → CohereDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=260,492—
Anthropic → CohereCPU per request (µs)892.7Valid—
Anthropic → CohereRequests per CPU-second1,120.2Valid—
Anthropic → CohereCost window concurrency8Valid—
Anthropic → CohereCost window core utilisation0.991Valid—
Anthropic → CohereCost window successful requests26,751Valid—
Anthropic → CohereCost window throughput (req/s)4,457Valid—
Anthropic → CohereCost window threads14Valid—
Anthropic → CohereNon-voluntary context switches per request0.018Valid—
Anthropic → CohereCost window major page faults0Valid—
Anthropic → CohereAdded time to first event p50 (µs)—Untestable—
Anthropic → CohereAdded time to first event p99 (µs)—Untestable—
Anthropic → CohereAdded inter-frame gap p50 (µs)—Untestable—
Anthropic → CohereAdded inter-frame gap p99 (µs)—Untestable—
Anthropic → CohereStreams sustained (frames/s)—Untestable—
Anthropic → CohereStreams sustained—Untestable—
Anthropic → CohereIdle RSS (MiB)161.3Valid—
Anthropic → CohereSteady-state RSS (MiB)1,118.1Valid—
Anthropic → CohereRecovered RSS (MiB)1,140.1Valid—
Anthropic → CoherePeak RSS (MiB)1,222.5Valid—
Anthropic → CoherePeak RSS high-water mark (MiB)1,216.4Valid—
Anthropic → CohereTime to plateau (s)—Not measured—
Anthropic → CohereMemory growth (MiB/min)68.59Valid—
Anthropic → CohereMemory load duration (s)300Valid—
Anthropic → CohereMemory shape under load1Valid—
Anthropic → CohereIdle memory shape—Not measured—
Anthropic → CohereIdle memory static1Valid—
Anthropic → CohereIdle memory growth (MiB/min)0.167Valid—
Anthropic → CohereThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → CohereThroughput (req/s)4,182concurrency 4; n=25,0975 ms
Anthropic → CohereThroughput (req/s)4,364concurrency 8; n=26,18810 ms
Anthropic → CohereThroughput (req/s)4,404concurrency 16; n=26,42950 ms
Anthropic → CohereThroughput (req/s)4,404concurrency 16; n=26,429100 ms
Anthropic → CohereThroughput (req/s)4,404concurrency 16; n=26,429Unbounded
Gemini → CohereAdded latency p50 (µs)670Valid · n=8,377 gateway / 268,876 direct—
Gemini → CohereAdded latency p99 (µs)1,111Valid · n=8,377 gateway / 268,876 direct—
Gemini → CohereGateway p99 at concurrency 1 (µs)1,138Valid · n=8,377—
Gemini → CohereDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=268,876—
Gemini → CohereCPU per request (µs)944.4Valid—
Gemini → CohereRequests per CPU-second1,058.8Valid—
Gemini → CohereCost window concurrency8Valid—
Gemini → CohereCost window core utilisation0.989Valid—
Gemini → CohereCost window successful requests25,211Valid—
Gemini → CohereCost window throughput (req/s)4,200Valid—
Gemini → CohereCost window threads14Valid—
Gemini → CohereNon-voluntary context switches per request0.046Valid—
Gemini → CohereCost window major page faults0Valid—
Gemini → CohereAdded time to first event p50 (µs)—Untestable—
Gemini → CohereAdded time to first event p99 (µs)—Untestable—
Gemini → CohereAdded inter-frame gap p50 (µs)—Untestable—
Gemini → CohereAdded inter-frame gap p99 (µs)—Untestable—
Gemini → CohereStreams sustained (frames/s)—Untestable—
Gemini → CohereStreams sustained—Untestable—
Gemini → CohereIdle RSS (MiB)162.3Valid—
Gemini → CohereSteady-state RSS (MiB)1,063Valid—
Gemini → CohereRecovered RSS (MiB)1,089.3Valid—
Gemini → CoherePeak RSS (MiB)1,247.8Valid—
Gemini → CoherePeak RSS high-water mark (MiB)1,208.6Valid—
Gemini → CohereTime to plateau (s)—Not measured—
Gemini → CohereMemory growth (MiB/min)89.15Valid—
Gemini → CohereMemory load duration (s)300Valid—
Gemini → CohereMemory shape under load1Valid—
Gemini → CohereIdle memory shape—Not measured—
Gemini → CohereIdle memory static1Valid—
Gemini → CohereIdle memory growth (MiB/min)0.289Valid—
Gemini → CohereThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Gemini → CohereThroughput (req/s)4,019concurrency 4; n=24,1185 ms
Gemini → CohereThroughput (req/s)4,163concurrency 8; n=24,98710 ms
Gemini → CohereThroughput (req/s)4,191concurrency 16; n=25,15550 ms
Gemini → CohereThroughput (req/s)4,191concurrency 16; n=25,155100 ms
Gemini → CohereThroughput (req/s)4,191concurrency 16; n=25,155Unbounded
Cohere → CohereAdded latency p50 (µs)609Valid · n=9,094 gateway / 264,951 direct—
Cohere → CohereAdded latency p99 (µs)1,062Valid · n=9,094 gateway / 264,951 direct—
Cohere → CohereGateway p99 at concurrency 1 (µs)1,090Valid · n=9,094—
Cohere → CohereDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=264,951—
Cohere → CohereCPU per request (µs)888.4Valid—
Cohere → CohereRequests per CPU-second1,125.7Valid—
Cohere → CohereCost window concurrency8Valid—
Cohere → CohereCost window core utilisation0.992Valid—
Cohere → CohereCost window successful requests26,881Valid—
Cohere → CohereCost window throughput (req/s)4,479Valid—
Cohere → CohereCost window threads14Valid—
Cohere → CohereNon-voluntary context switches per request0.019Valid—
Cohere → CohereCost window major page faults0Valid—
Cohere → CohereAdded time to first event p50 (µs)—Untestable—
Cohere → CohereAdded time to first event p99 (µs)—Untestable—
Cohere → CohereAdded inter-frame gap p50 (µs)—Untestable—
Cohere → CohereAdded inter-frame gap p99 (µs)—Untestable—
Cohere → CohereStreams sustained (frames/s)—Untestable—
Cohere → CohereStreams sustained—Untestable—
Cohere → CohereIdle RSS (MiB)160.7Valid—
Cohere → CohereSteady-state RSS (MiB)1,074.5Valid—
Cohere → CohereRecovered RSS (MiB)1,027.6Valid—
Cohere → CoherePeak RSS (MiB)1,245.7Valid—
Cohere → CoherePeak RSS high-water mark (MiB)1,159Valid—
Cohere → CohereTime to plateau (s)—Not measured—
Cohere → CohereMemory growth (MiB/min)76.4Valid—
Cohere → CohereMemory load duration (s)300Valid—
Cohere → CohereMemory shape under load1Valid—
Cohere → CohereIdle memory shape—Not measured—
Cohere → CohereIdle memory static1Valid—
Cohere → CohereIdle memory growth (MiB/min)0.628Valid—
Cohere → CohereThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Cohere → CohereThroughput (req/s)4,379concurrency 4; n=26,2845 ms
Cohere → CohereThroughput (req/s)4,498concurrency 8; n=26,99210 ms
Cohere → CohereThroughput (req/s)4,544concurrency 16; n=27,26750 ms
Cohere → CohereThroughput (req/s)4,544concurrency 16; n=27,267100 ms
Cohere → CohereThroughput (req/s)4,544concurrency 16; n=27,267Unbounded
Bedrock → CohereAdded latency p50 (µs)553Valid · n=10,081 gateway / 263,966 direct—
Bedrock → CohereAdded latency p99 (µs)938Valid · n=10,081 gateway / 263,966 direct—
Bedrock → CohereGateway p99 at concurrency 1 (µs)966Valid · n=10,081—
Bedrock → CohereDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=263,966—
Bedrock → CohereCPU per request (µs)808.9Valid—
Bedrock → CohereRequests per CPU-second1,236.3Valid—
Bedrock → CohereCost window concurrency8Valid—
Bedrock → CohereCost window core utilisation0.99Valid—
Bedrock → CohereCost window successful requests29,461Valid—
Bedrock → CohereCost window throughput (req/s)4,909Valid—
Bedrock → CohereCost window threads14Valid—
Bedrock → CohereNon-voluntary context switches per request0.016Valid—
Bedrock → CohereCost window major page faults0Valid—
Bedrock → CohereAdded time to first event p50 (µs)—Untestable—
Bedrock → CohereAdded time to first event p99 (µs)—Untestable—
Bedrock → CohereAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → CohereAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → CohereStreams sustained (frames/s)—Untestable—
Bedrock → CohereStreams sustained—Untestable—
Bedrock → CohereIdle RSS (MiB)161.3Valid—
Bedrock → CohereSteady-state RSS (MiB)1,081.5Valid—
Bedrock → CohereRecovered RSS (MiB)1,024Valid—
Bedrock → CoherePeak RSS (MiB)1,293.4Valid—
Bedrock → CoherePeak RSS high-water mark (MiB)1,176.1Valid—
Bedrock → CohereTime to plateau (s)—Not measured—
Bedrock → CohereMemory growth (MiB/min)70.1Valid—
Bedrock → CohereMemory load duration (s)300Valid—
Bedrock → CohereMemory shape under load1Valid—
Bedrock → CohereIdle memory shape—Not measured—
Bedrock → CohereIdle memory static1Valid—
Bedrock → CohereIdle memory growth (MiB/min)0.335Valid—
Bedrock → CohereThroughput (req/s)1,683concurrency 1; n=10,1011 ms
Bedrock → CohereThroughput (req/s)4,732concurrency 4; n=28,4015 ms
Bedrock → CohereThroughput (req/s)4,965concurrency 8; n=29,79510 ms
Bedrock → CohereThroughput (req/s)5,048concurrency 16; n=30,29250 ms
Bedrock → CohereThroughput (req/s)5,048concurrency 16; n=30,292100 ms
Bedrock → CohereThroughput (req/s)5,048concurrency 16; n=30,292Unbounded
OpenAI → Bedrock (upstream over HTTPS)Added latency p50 (µs)708Valid · n=7,893 gateway / 260,423 direct—
OpenAI → Bedrock (upstream over HTTPS)Added latency p99 (µs)1,098Valid · n=7,893 gateway / 260,423 direct—
OpenAI → Bedrock (upstream over HTTPS)Gateway p99 at concurrency 1 (µs)1,126Valid · n=7,893—
OpenAI → Bedrock (upstream over HTTPS)Direct-to-mock p99 at concurrency 1 (µs)28Valid · n=260,423—
OpenAI → Bedrock (upstream over HTTPS)CPU per request (µs)955.8Valid—
OpenAI → Bedrock (upstream over HTTPS)Requests per CPU-second1,046.2Valid—
OpenAI → Bedrock (upstream over HTTPS)Cost window concurrency8Valid—
OpenAI → Bedrock (upstream over HTTPS)Cost window core utilisation0.989Valid—
OpenAI → Bedrock (upstream over HTTPS)Cost window successful requests24,910Valid—
OpenAI → Bedrock (upstream over HTTPS)Cost window throughput (req/s)4,151Valid—
OpenAI → Bedrock (upstream over HTTPS)Cost window threads17Valid—
OpenAI → Bedrock (upstream over HTTPS)Non-voluntary context switches per request0Valid—
OpenAI → Bedrock (upstream over HTTPS)Cost window major page faults0Valid—
OpenAI → Bedrock (upstream over HTTPS)Added time to first event p50 (µs)—Untestable—
OpenAI → Bedrock (upstream over HTTPS)Added time to first event p99 (µs)—Untestable—
OpenAI → Bedrock (upstream over HTTPS)Added inter-frame gap p50 (µs)—Untestable—
OpenAI → Bedrock (upstream over HTTPS)Added inter-frame gap p99 (µs)—Untestable—
OpenAI → Bedrock (upstream over HTTPS)Streams sustained (frames/s)—Untestable—
OpenAI → Bedrock (upstream over HTTPS)Streams sustained—Untestable—
OpenAI → Bedrock (upstream over HTTPS)Idle RSS (MiB)160Valid—
OpenAI → Bedrock (upstream over HTTPS)Steady-state RSS (MiB)981.9Valid—
OpenAI → Bedrock (upstream over HTTPS)Recovered RSS (MiB)1,138Valid—
OpenAI → Bedrock (upstream over HTTPS)Peak RSS (MiB)1,138.1Valid—
OpenAI → Bedrock (upstream over HTTPS)Peak RSS high-water mark (MiB)1,131Valid—
OpenAI → Bedrock (upstream over HTTPS)Time to plateau (s)—Not measured—
OpenAI → Bedrock (upstream over HTTPS)Memory growth (MiB/min)92.38Valid—
OpenAI → Bedrock (upstream over HTTPS)Memory load duration (s)300Valid—
OpenAI → Bedrock (upstream over HTTPS)Memory shape under load1Valid—
OpenAI → Bedrock (upstream over HTTPS)Idle memory shape—Not measured—
OpenAI → Bedrock (upstream over HTTPS)Idle memory static1Valid—
OpenAI → Bedrock (upstream over HTTPS)Idle memory growth (MiB/min)0.746Valid—
OpenAI → Bedrock (upstream over HTTPS)Throughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → Bedrock (upstream over HTTPS)Throughput (req/s)3,880concurrency 4; n=23,2915 ms
OpenAI → Bedrock (upstream over HTTPS)Throughput (req/s)4,031concurrency 8; n=24,19410 ms
OpenAI → Bedrock (upstream over HTTPS)Throughput (req/s)4,078concurrency 32; n=24,47550 ms
OpenAI → Bedrock (upstream over HTTPS)Throughput (req/s)4,078concurrency 32; n=24,475100 ms
OpenAI → Bedrock (upstream over HTTPS)Throughput (req/s)4,078concurrency 32; n=24,475Unbounded
OpenAI Responses → Bedrock (upstream over HTTPS)Added latency p50 (µs)728Valid · n=7,776 gateway / 260,884 direct—
OpenAI Responses → Bedrock (upstream over HTTPS)Added latency p99 (µs)1,097Valid · n=7,776 gateway / 260,884 direct—
OpenAI Responses → Bedrock (upstream over HTTPS)Gateway p99 at concurrency 1 (µs)1,125Valid · n=7,776—
OpenAI Responses → Bedrock (upstream over HTTPS)Direct-to-mock p99 at concurrency 1 (µs)28Valid · n=260,884—
OpenAI Responses → Bedrock (upstream over HTTPS)CPU per request (µs)995.5Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Requests per CPU-second1,004.6Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Cost window concurrency8Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Cost window core utilisation0.984Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Cost window successful requests23,808Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Cost window throughput (req/s)3,967Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Cost window threads14Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Non-voluntary context switches per request0Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Cost window major page faults0Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Added time to first event p50 (µs)—Untestable—
OpenAI Responses → Bedrock (upstream over HTTPS)Added time to first event p99 (µs)—Untestable—
OpenAI Responses → Bedrock (upstream over HTTPS)Added inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → Bedrock (upstream over HTTPS)Added inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → Bedrock (upstream over HTTPS)Streams sustained (frames/s)—Untestable—
OpenAI Responses → Bedrock (upstream over HTTPS)Streams sustained—Untestable—
OpenAI Responses → Bedrock (upstream over HTTPS)Idle RSS (MiB)161.1Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Steady-state RSS (MiB)922.3Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Recovered RSS (MiB)1,033Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Peak RSS (MiB)1,145.9Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Peak RSS high-water mark (MiB)1,102.1Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Time to plateau (s)—Not measured—
OpenAI Responses → Bedrock (upstream over HTTPS)Memory growth (MiB/min)104.2Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Memory load duration (s)300Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Memory shape under load1Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Idle memory shape—Not measured—
OpenAI Responses → Bedrock (upstream over HTTPS)Idle memory static1Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Idle memory growth (MiB/min)0.265Valid—
OpenAI Responses → Bedrock (upstream over HTTPS)Throughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI Responses → Bedrock (upstream over HTTPS)Throughput (req/s)3,740concurrency 4; n=22,4455 ms
OpenAI Responses → Bedrock (upstream over HTTPS)Throughput (req/s)3,865concurrency 8; n=23,20110 ms
OpenAI Responses → Bedrock (upstream over HTTPS)Throughput (req/s)3,918concurrency 32; n=23,51150 ms
OpenAI Responses → Bedrock (upstream over HTTPS)Throughput (req/s)3,918concurrency 32; n=23,511100 ms
OpenAI Responses → Bedrock (upstream over HTTPS)Throughput (req/s)3,918concurrency 32; n=23,511Unbounded
Anthropic → Bedrock (upstream over HTTPS)Added latency p50 (µs)726Valid · n=7,777 gateway / 244,991 direct—
Anthropic → Bedrock (upstream over HTTPS)Added latency p99 (µs)1,099Valid · n=7,777 gateway / 244,991 direct—
Anthropic → Bedrock (upstream over HTTPS)Gateway p99 at concurrency 1 (µs)1,129Valid · n=7,777—
Anthropic → Bedrock (upstream over HTTPS)Direct-to-mock p99 at concurrency 1 (µs)30Valid · n=244,991—
Anthropic → Bedrock (upstream over HTTPS)CPU per request (µs)1,001.5Valid—
Anthropic → Bedrock (upstream over HTTPS)Requests per CPU-second998.5Valid—
Anthropic → Bedrock (upstream over HTTPS)Cost window concurrency8Valid—
Anthropic → Bedrock (upstream over HTTPS)Cost window core utilisation0.99Valid—
Anthropic → Bedrock (upstream over HTTPS)Cost window successful requests23,804Valid—
Anthropic → Bedrock (upstream over HTTPS)Cost window throughput (req/s)3,966Valid—
Anthropic → Bedrock (upstream over HTTPS)Cost window threads20Valid—
Anthropic → Bedrock (upstream over HTTPS)Non-voluntary context switches per request0.033Valid—
Anthropic → Bedrock (upstream over HTTPS)Cost window major page faults0Valid—
Anthropic → Bedrock (upstream over HTTPS)Added time to first event p50 (µs)—Untestable—
Anthropic → Bedrock (upstream over HTTPS)Added time to first event p99 (µs)—Untestable—
Anthropic → Bedrock (upstream over HTTPS)Added inter-frame gap p50 (µs)—Untestable—
Anthropic → Bedrock (upstream over HTTPS)Added inter-frame gap p99 (µs)—Untestable—
Anthropic → Bedrock (upstream over HTTPS)Streams sustained (frames/s)—Untestable—
Anthropic → Bedrock (upstream over HTTPS)Streams sustained—Untestable—
Anthropic → Bedrock (upstream over HTTPS)Idle RSS (MiB)160.8Valid—
Anthropic → Bedrock (upstream over HTTPS)Steady-state RSS (MiB)950.2Valid—
Anthropic → Bedrock (upstream over HTTPS)Recovered RSS (MiB)1,054Valid—
Anthropic → Bedrock (upstream over HTTPS)Peak RSS (MiB)1,191.1Valid—
Anthropic → Bedrock (upstream over HTTPS)Peak RSS high-water mark (MiB)1,138.2Valid—
Anthropic → Bedrock (upstream over HTTPS)Time to plateau (s)—Not measured—
Anthropic → Bedrock (upstream over HTTPS)Memory growth (MiB/min)99.69Valid—
Anthropic → Bedrock (upstream over HTTPS)Memory load duration (s)300Valid—
Anthropic → Bedrock (upstream over HTTPS)Memory shape under load1Valid—
Anthropic → Bedrock (upstream over HTTPS)Idle memory shape—Not measured—
Anthropic → Bedrock (upstream over HTTPS)Idle memory static1Valid—
Anthropic → Bedrock (upstream over HTTPS)Idle memory growth (MiB/min)0.353Valid—
Anthropic → Bedrock (upstream over HTTPS)Throughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → Bedrock (upstream over HTTPS)Throughput (req/s)3,426concurrency 4; n=20,5655 ms
Anthropic → Bedrock (upstream over HTTPS)Throughput (req/s)3,624concurrency 8; n=21,75210 ms
Anthropic → Bedrock (upstream over HTTPS)Throughput (req/s)3,706concurrency 32; n=22,24150 ms
Anthropic → Bedrock (upstream over HTTPS)Throughput (req/s)3,706concurrency 32; n=22,241100 ms
Anthropic → Bedrock (upstream over HTTPS)Throughput (req/s)3,706concurrency 32; n=22,241Unbounded
Gemini → Bedrock (upstream over HTTPS)Added latency p50 (µs)732Valid · n=7,747 gateway / 257,445 direct—
Gemini → Bedrock (upstream over HTTPS)Added latency p99 (µs)1,096Valid · n=7,747 gateway / 257,445 direct—
Gemini → Bedrock (upstream over HTTPS)Gateway p99 at concurrency 1 (µs)1,125Valid · n=7,747—
Gemini → Bedrock (upstream over HTTPS)Direct-to-mock p99 at concurrency 1 (µs)29Valid · n=257,445—
Gemini → Bedrock (upstream over HTTPS)CPU per request (µs)1,022.9Valid—
Gemini → Bedrock (upstream over HTTPS)Requests per CPU-second977.6Valid—
Gemini → Bedrock (upstream over HTTPS)Cost window concurrency8Valid—
Gemini → Bedrock (upstream over HTTPS)Cost window core utilisation0.986Valid—
Gemini → Bedrock (upstream over HTTPS)Cost window successful requests23,209Valid—
Gemini → Bedrock (upstream over HTTPS)Cost window throughput (req/s)3,867Valid—
Gemini → Bedrock (upstream over HTTPS)Cost window threads13Valid—
Gemini → Bedrock (upstream over HTTPS)Non-voluntary context switches per request0.019Valid—
Gemini → Bedrock (upstream over HTTPS)Cost window major page faults0Valid—
Gemini → Bedrock (upstream over HTTPS)Added time to first event p50 (µs)—Untestable—
Gemini → Bedrock (upstream over HTTPS)Added time to first event p99 (µs)—Untestable—
Gemini → Bedrock (upstream over HTTPS)Added inter-frame gap p50 (µs)—Untestable—
Gemini → Bedrock (upstream over HTTPS)Added inter-frame gap p99 (µs)—Untestable—
Gemini → Bedrock (upstream over HTTPS)Streams sustained (frames/s)—Untestable—
Gemini → Bedrock (upstream over HTTPS)Streams sustained—Untestable—
Gemini → Bedrock (upstream over HTTPS)Idle RSS (MiB)159.6Valid—
Gemini → Bedrock (upstream over HTTPS)Steady-state RSS (MiB)951.7Valid—
Gemini → Bedrock (upstream over HTTPS)Recovered RSS (MiB)1,257.6Valid—
Gemini → Bedrock (upstream over HTTPS)Peak RSS (MiB)1,257.6Valid—
Gemini → Bedrock (upstream over HTTPS)Peak RSS high-water mark (MiB)1,150.5Valid—
Gemini → Bedrock (upstream over HTTPS)Time to plateau (s)—Not measured—
Gemini → Bedrock (upstream over HTTPS)Memory growth (MiB/min)113.8Valid—
Gemini → Bedrock (upstream over HTTPS)Memory load duration (s)300Valid—
Gemini → Bedrock (upstream over HTTPS)Memory shape under load1Valid—
Gemini → Bedrock (upstream over HTTPS)Idle memory shape—Not measured—
Gemini → Bedrock (upstream over HTTPS)Idle memory static1Valid—
Gemini → Bedrock (upstream over HTTPS)Idle memory growth (MiB/min)0.389Valid—
Gemini → Bedrock (upstream over HTTPS)Throughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Gemini → Bedrock (upstream over HTTPS)Throughput (req/s)3,839concurrency 4; n=23,0385 ms
Gemini → Bedrock (upstream over HTTPS)Throughput (req/s)3,970concurrency 8; n=23,82910 ms
Gemini → Bedrock (upstream over HTTPS)Throughput (req/s)4,018concurrency 32; n=24,11750 ms
Gemini → Bedrock (upstream over HTTPS)Throughput (req/s)4,018concurrency 32; n=24,117100 ms
Gemini → Bedrock (upstream over HTTPS)Throughput (req/s)4,018concurrency 32; n=24,117Unbounded
Cohere → Bedrock (upstream over HTTPS)Added latency p50 (µs)692Valid · n=8,190 gateway / 262,997 direct—
Cohere → Bedrock (upstream over HTTPS)Added latency p99 (µs)1,061Valid · n=8,190 gateway / 262,997 direct—
Cohere → Bedrock (upstream over HTTPS)Gateway p99 at concurrency 1 (µs)1,090Valid · n=8,190—
Cohere → Bedrock (upstream over HTTPS)Direct-to-mock p99 at concurrency 1 (µs)29Valid · n=262,997—
Cohere → Bedrock (upstream over HTTPS)CPU per request (µs)938.6Valid—
Cohere → Bedrock (upstream over HTTPS)Requests per CPU-second1,065.4Valid—
Cohere → Bedrock (upstream over HTTPS)Cost window concurrency8Valid—
Cohere → Bedrock (upstream over HTTPS)Cost window core utilisation0.981Valid—
Cohere → Bedrock (upstream over HTTPS)Cost window successful requests25,165Valid—
Cohere → Bedrock (upstream over HTTPS)Cost window throughput (req/s)4,193Valid—
Cohere → Bedrock (upstream over HTTPS)Cost window threads13Valid—
Cohere → Bedrock (upstream over HTTPS)Non-voluntary context switches per request0Valid—
Cohere → Bedrock (upstream over HTTPS)Cost window major page faults0Valid—
Cohere → Bedrock (upstream over HTTPS)Added time to first event p50 (µs)—Untestable—
Cohere → Bedrock (upstream over HTTPS)Added time to first event p99 (µs)—Untestable—
Cohere → Bedrock (upstream over HTTPS)Added inter-frame gap p50 (µs)—Untestable—
Cohere → Bedrock (upstream over HTTPS)Added inter-frame gap p99 (µs)—Untestable—
Cohere → Bedrock (upstream over HTTPS)Streams sustained (frames/s)—Untestable—
Cohere → Bedrock (upstream over HTTPS)Streams sustained—Untestable—
Cohere → Bedrock (upstream over HTTPS)Idle RSS (MiB)162.1Valid—
Cohere → Bedrock (upstream over HTTPS)Steady-state RSS (MiB)1,001.6Valid—
Cohere → Bedrock (upstream over HTTPS)Recovered RSS (MiB)1,041Valid—
Cohere → Bedrock (upstream over HTTPS)Peak RSS (MiB)1,135.6Valid—
Cohere → Bedrock (upstream over HTTPS)Peak RSS high-water mark (MiB)1,107.1Valid—
Cohere → Bedrock (upstream over HTTPS)Time to plateau (s)—Not measured—
Cohere → Bedrock (upstream over HTTPS)Memory growth (MiB/min)66.49Valid—
Cohere → Bedrock (upstream over HTTPS)Memory load duration (s)300Valid—
Cohere → Bedrock (upstream over HTTPS)Memory shape under load1Valid—
Cohere → Bedrock (upstream over HTTPS)Idle memory shape—Not measured—
Cohere → Bedrock (upstream over HTTPS)Idle memory static1Valid—
Cohere → Bedrock (upstream over HTTPS)Idle memory growth (MiB/min)0.799Valid—
Cohere → Bedrock (upstream over HTTPS)Throughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Cohere → Bedrock (upstream over HTTPS)Throughput (req/s)4,119concurrency 4; n=24,7195 ms
Cohere → Bedrock (upstream over HTTPS)Throughput (req/s)4,263concurrency 8; n=25,58110 ms
Cohere → Bedrock (upstream over HTTPS)Throughput (req/s)4,275concurrency 32; n=25,65850 ms
Cohere → Bedrock (upstream over HTTPS)Throughput (req/s)4,275concurrency 32; n=25,658100 ms
Cohere → Bedrock (upstream over HTTPS)Throughput (req/s)4,275concurrency 32; n=25,658Unbounded
Bedrock → Bedrock (upstream over HTTPS)Added latency p50 (µs)694Valid · n=8,133 gateway / 263,582 direct—
Bedrock → Bedrock (upstream over HTTPS)Added latency p99 (µs)1,052Valid · n=8,133 gateway / 263,582 direct—
Bedrock → Bedrock (upstream over HTTPS)Gateway p99 at concurrency 1 (µs)1,079Valid · n=8,133—
Bedrock → Bedrock (upstream over HTTPS)Direct-to-mock p99 at concurrency 1 (µs)27Valid · n=263,582—
Bedrock → Bedrock (upstream over HTTPS)CPU per request (µs)956.8Valid—
Bedrock → Bedrock (upstream over HTTPS)Requests per CPU-second1,045.2Valid—
Bedrock → Bedrock (upstream over HTTPS)Cost window concurrency8Valid—
Bedrock → Bedrock (upstream over HTTPS)Cost window core utilisation0.989Valid—
Bedrock → Bedrock (upstream over HTTPS)Cost window successful requests24,886Valid—
Bedrock → Bedrock (upstream over HTTPS)Cost window throughput (req/s)4,147Valid—
Bedrock → Bedrock (upstream over HTTPS)Cost window threads14Valid—
Bedrock → Bedrock (upstream over HTTPS)Non-voluntary context switches per request0.034Valid—
Bedrock → Bedrock (upstream over HTTPS)Cost window major page faults0Valid—
Bedrock → Bedrock (upstream over HTTPS)Added time to first event p50 (µs)—Untestable—
Bedrock → Bedrock (upstream over HTTPS)Added time to first event p99 (µs)—Untestable—
Bedrock → Bedrock (upstream over HTTPS)Added inter-frame gap p50 (µs)—Untestable—
Bedrock → Bedrock (upstream over HTTPS)Added inter-frame gap p99 (µs)—Untestable—
Bedrock → Bedrock (upstream over HTTPS)Streams sustained (frames/s)—Untestable—
Bedrock → Bedrock (upstream over HTTPS)Streams sustained—Untestable—
Bedrock → Bedrock (upstream over HTTPS)Idle RSS (MiB)160Valid—
Bedrock → Bedrock (upstream over HTTPS)Steady-state RSS (MiB)964.1Valid—
Bedrock → Bedrock (upstream over HTTPS)Recovered RSS (MiB)1,242.1Valid—
Bedrock → Bedrock (upstream over HTTPS)Peak RSS (MiB)1,242.1Valid—
Bedrock → Bedrock (upstream over HTTPS)Peak RSS high-water mark (MiB)1,119.4Valid—
Bedrock → Bedrock (upstream over HTTPS)Time to plateau (s)—Not measured—
Bedrock → Bedrock (upstream over HTTPS)Memory growth (MiB/min)61.4Valid—
Bedrock → Bedrock (upstream over HTTPS)Memory load duration (s)300Valid—
Bedrock → Bedrock (upstream over HTTPS)Memory shape under load1Valid—
Bedrock → Bedrock (upstream over HTTPS)Idle memory shape—Not measured—
Bedrock → Bedrock (upstream over HTTPS)Idle memory static1Valid—
Bedrock → Bedrock (upstream over HTTPS)Idle memory growth (MiB/min)0.783Valid—
Bedrock → Bedrock (upstream over HTTPS)Throughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Bedrock → Bedrock (upstream over HTTPS)Throughput (req/s)3,832concurrency 4; n=22,9995 ms
Bedrock → Bedrock (upstream over HTTPS)Throughput (req/s)3,972concurrency 8; n=23,83510 ms
Bedrock → Bedrock (upstream over HTTPS)Throughput (req/s)4,035concurrency 32; n=24,21450 ms
Bedrock → Bedrock (upstream over HTTPS)Throughput (req/s)4,035concurrency 32; n=24,214100 ms
Bedrock → Bedrock (upstream over HTTPS)Throughput (req/s)4,035concurrency 32; n=24,214Unbounded

Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

Ingress → upstreamCell stateProbe HTTP statusRecorded reasonRecorded detailEvidence scopeResponse excerptTested build
OpenAI → OpenAIServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI Responses → OpenAIServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Anthropic → OpenAIServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Gemini → OpenAIServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Cohere → OpenAIServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Bedrock → OpenAIServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—maximhq/bifrost:v2.5.0
OpenAI Responses → OpenAI ResponsesServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Anthropic → OpenAI ResponsesServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Gemini → OpenAI ResponsesServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Cohere → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—maximhq/bifrost:v2.5.0
Bedrock → OpenAI ResponsesServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI → AnthropicServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI Responses → AnthropicServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Anthropic → AnthropicServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Gemini → AnthropicServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Cohere → AnthropicServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Bedrock → AnthropicServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI → GeminiServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI Responses → GeminiServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Anthropic → GeminiServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Gemini → GeminiServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Cohere → GeminiServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Bedrock → GeminiServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI → CohereServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI Responses → CohereServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Anthropic → CohereServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Gemini → CohereServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Cohere → CohereServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Bedrock → CohereServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI → BedrockServed———Recorded run evidence—maximhq/bifrost:v2.5.0
OpenAI Responses → BedrockServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Anthropic → BedrockServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Gemini → BedrockServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Cohere → BedrockServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Bedrock → BedrockServed———Recorded run evidence—maximhq/bifrost:v2.5.0
Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2
Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8
Load generator SHA-256
not recorded
net.core.somaxconn
65,535
net.ipv4.tcp_max_syn_backlog
65,535

Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

Ingress → upstreamConcurrencyLanes by HTTP statusLanes with no response
OpenAI → OpenAI1HTTP 200: 10
OpenAI → OpenAI2HTTP 200: 20
OpenAI → OpenAI4HTTP 200: 40
OpenAI → OpenAI8HTTP 200: 80
OpenAI → OpenAI16HTTP 200: 160
OpenAI → OpenAI32HTTP 200: 320
OpenAI → OpenAI64HTTP 200: 640
OpenAI → OpenAI128HTTP 200: 1280
OpenAI → OpenAI256HTTP 200: 2560
OpenAI → OpenAI512HTTP 200: 5120
OpenAI → OpenAI1,024HTTP 200: 1,0240
OpenAI → OpenAI768HTTP 200: 7680
OpenAI → OpenAI640HTTP 200: 6400
OpenAI → OpenAI576HTTP 200: 5760
OpenAI → OpenAI608HTTP 200: 6080
OpenAI → OpenAI592HTTP 200: 5920
OpenAI → OpenAI584HTTP 200: 5840
OpenAI → OpenAI580HTTP 200: 5800
OpenAI → OpenAI582HTTP 200: 5820
OpenAI → OpenAI581HTTP 200: 5810
OpenAI → OpenAI580HTTP 200: 5800
OpenAI → OpenAI580HTTP 200: 5800
Anthropic → OpenAI1HTTP 200: 10
Anthropic → OpenAI2HTTP 200: 20
Anthropic → OpenAI4HTTP 200: 40
Anthropic → OpenAI8HTTP 200: 80
Anthropic → OpenAI16HTTP 200: 160
Anthropic → OpenAI32HTTP 200: 320
Anthropic → OpenAI64HTTP 200: 640
Anthropic → OpenAI128HTTP 200: 1280
Anthropic → OpenAI256HTTP 200: 2560
Anthropic → OpenAI512HTTP 200: 5120
Anthropic → OpenAI1,024HTTP 200: 1,0240
Anthropic → OpenAI768HTTP 200: 7680
Anthropic → OpenAI640HTTP 200: 6400
Anthropic → OpenAI704HTTP 200: 7040
Anthropic → OpenAI672HTTP 200: 6720
Anthropic → OpenAI688HTTP 200: 6880
Anthropic → OpenAI680HTTP 200: 6800
Anthropic → OpenAI676HTTP 200: 6760
Anthropic → OpenAI678HTTP 200: 6780
Anthropic → OpenAI679HTTP 200: 6790
Anthropic → OpenAI678HTTP 200: 6780
Anthropic → OpenAI678HTTP 200: 6780
OpenAI → Anthropic1HTTP 200: 10
OpenAI → Anthropic2HTTP 200: 20
OpenAI → Anthropic4HTTP 200: 40
OpenAI → Anthropic8HTTP 200: 80
OpenAI → Anthropic16HTTP 200: 160
OpenAI → Anthropic32HTTP 200: 320
OpenAI → Anthropic64HTTP 200: 640
OpenAI → Anthropic128HTTP 200: 1280
OpenAI → Anthropic256HTTP 200: 2560
OpenAI → Anthropic512HTTP 200: 5120
OpenAI → Anthropic1,024HTTP 200: 1,0240
OpenAI → Anthropic768HTTP 200: 7680
OpenAI → Anthropic640HTTP 200: 6400
OpenAI → Anthropic576HTTP 200: 5760
OpenAI → Anthropic608HTTP 200: 6080
OpenAI → Anthropic624HTTP 200: 6240
OpenAI → Anthropic632HTTP 200: 6320
OpenAI → Anthropic628HTTP 200: 6280
OpenAI → Anthropic626HTTP 200: 6260
OpenAI → Anthropic625HTTP 200: 6250
OpenAI → Anthropic624HTTP 200: 6240
OpenAI → Anthropic624HTTP 200: 6240
Anthropic → Anthropic1HTTP 200: 10
Anthropic → Anthropic2HTTP 200: 20
Anthropic → Anthropic4HTTP 200: 40
Anthropic → Anthropic8HTTP 200: 80
Anthropic → Anthropic16HTTP 200: 160
Anthropic → Anthropic32HTTP 200: 320
Anthropic → Anthropic64HTTP 200: 640
Anthropic → Anthropic128HTTP 200: 1280
Anthropic → Anthropic256HTTP 200: 2560
Anthropic → Anthropic512HTTP 200: 5120
Anthropic → Anthropic1,024HTTP 200: 1,0240
Anthropic → Anthropic768HTTP 200: 7680
Anthropic → Anthropic896HTTP 200: 8960
Anthropic → Anthropic832HTTP 200: 8320
Anthropic → Anthropic864HTTP 200: 8640
Anthropic → Anthropic848HTTP 200: 8480
Anthropic → Anthropic856HTTP 200: 8560
Anthropic → Anthropic852HTTP 200: 8520
Anthropic → Anthropic854HTTP 200: 8540
Anthropic → Anthropic855HTTP 200: 8550
Anthropic → Anthropic855HTTP 200: 8550
Anthropic → Anthropic855HTTP 200: 8550
Anthropic → Anthropic683HTTP 200: 6830
Anthropic → Anthropic683HTTP 200: 6830
Anthropic → Anthropic683HTTP 200: 6830

Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

  1. Bifrost v2.5.0 source 69f4d56d18b9ba00ac5f9cf969b4501a882b9967. Every upstream except Bedrock uses the field's common model gpt-4o-mini; Bedrock uses a Claude model id (anthropic.claude-3-5-sonnet-20240620-v1:0) so Bifrost stays on the bedrock-runtime Converse surface rather than routing an OpenAI model family to its OpenAI-compatible Mantle surface (core/providers/bedrock/bedrock.go resolveSurface). OpenAI Chat and Cohere ingress into the Responses-only upstream are not enabled: Bifrost only converts them when the model catalog marks the model Responses-only (plugins/compat/main.go::markForConversion), and selecting a different model (o3-pro) to trigger that path also triggers the compat plugin's default parameter dropping (it removed max_completion_tokens in the preflight), which the benchmark's out-of-the-box rule excludes. The Bedrock upstream uses Bifrost's default HTTPS runtime host (bedrock-runtime.us-east-1.amazonaws.com), which the engine v1.2 catch-all DNS resolves to the mock; its six cells are probed and carry upstream_transport "tls".
  2. Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

Evidence for the current measurements

Configuration recorded by the run

{
  "framework": {
    "pricing": {
      "pricing_url": "file:///app/data/pricing.json",
      "pricing_sync_interval": 86400,
      "model_parameters_url": "file:///app/data/model-parameters.json"
    }
  },
  "governance": {
    "virtual_keys": [
      {
        "id": "bench",
        "name": "benchmark",
        "value": "sk-bf-benchmark-dummy",
        "is_active": true,
        "allow_all_providers": true
      }
    ]
  },
  "providers": {
    "anthropic": {
      "keys": [
        {
          "value": "sk-dummy",
          "models": [
            "claude-3-5-sonnet-20241022"
          ],
          "weight": 1
        }
      ],
      "network_config": {
        "allow_private_network": true,
        "base_url": "http://127.0.0.1:8000"
      }
    },
    "gemini": {
      "keys": [
        {
          "value": "sk-dummy",
          "models": [
            "gemini-1.5-pro"
          ],
          "weight": 1
        }
      ],
      "network_config": {
        "allow_private_network": true,
        "base_url": "http://127.0.0.1:8000"
      }
    },
    "cohere": {
      "keys": [
        {
          "value": "sk-dummy",
          "models": [
            "command-r-plus"
          ],
          "weight": 1
        }
      ],
      "network_config": {
        "allow_private_network": true,
        "base_url": "http://127.0.0.1:8000"
      }
    },
    "bedrock": {
      "keys": [
        {
          "value": "",
          "models": [
            "anthropic.claude-3-5-sonnet-20240620-v1:0"
          ],
          "weight": 1,
          "bedrock_key_config": {
            "access_key": "AKIAMOCKACCESSKEY",
            "secret_key": "mock-secret-access-key",
            "region": "us-east-1"
          }
        }
      ],
      "network_config": {
        "allow_private_network": true
      }
    },
    "bench-chat": {
      "keys": [
        {
          "value": "sk-dummy",
          "models": [
            "gpt-4o-mini"
          ],
          "weight": 1
        }
      ],
      "network_config": {
        "allow_private_network": true,
        "base_url": "http://127.0.0.1:8000"
      },
      "custom_provider_config": {
        "base_provider_type": "openai",
        "allowed_requests": {
          "list_models": true,
          "chat_completion": true,
          "chat_completion_stream": true
        }
      }
    },
    "bench-responses": {
      "keys": [
        {
          "value": "sk-dummy",
          "models": [
            "gpt-4o-mini"
          ],
          "weight": 1
        }
      ],
      "network_config": {
        "allow_private_network": true,
        "base_url": "http://127.0.0.1:8000"
      },
      "custom_provider_config": {
        "base_provider_type": "openai",
        "allowed_requests": {
          "list_models": true,
          "responses": true,
          "responses_stream": true
        }
      }
    }
  }
}

Compare a shared protocol cell

Historical measurements