Portkey benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Node · recorded build: portkeyai/gateway:1.15.2. Measured: .

JSON · Markdown · CSV · Project source · Interactive dashboard

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

Published measurements

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
Bedrock → OpenAI——Not declared: recorded note 1—
Gemini → OpenAI——Not declared: recorded note 1—
OpenAI → OpenAIAdded latency p50 (µs)1,196Valid · n=4,304 gateway / 246,357 direct—
OpenAI → OpenAIAdded latency p99 (µs)3,902Valid · n=4,304 gateway / 246,357 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)3,938Valid · n=4,304—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)36Valid · n=246,357—
OpenAI → OpenAICPU per request (µs)1,528.4Valid—
OpenAI → OpenAIRequests per CPU-second654.3Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.295Valid—
OpenAI → OpenAICost window successful requests4,652Valid—
OpenAI → OpenAICost window throughput (req/s)775Valid—
OpenAI → OpenAICost window threads22Valid—
OpenAI → OpenAINon-voluntary context switches per request0.007Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)28,114Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)29,752Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)0Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)440Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)4,996.2Valid—
OpenAI → OpenAIStreams sustained1,519Valid—
OpenAI → OpenAIIdle RSS (MiB)125.1Valid—
OpenAI → OpenAISteady-state RSS (MiB)245.2Valid—
OpenAI → OpenAIRecovered RSS (MiB)142.1Valid—
OpenAI → OpenAIPeak RSS (MiB)250Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)264Valid—
OpenAI → OpenAITime to plateau (s)—Not measured—
OpenAI → OpenAIMemory growth (MiB/min)0.202Valid—
OpenAI → OpenAIMemory load duration (s)301Valid—
OpenAI → OpenAIMemory shape under load0Valid—
OpenAI → OpenAIIdle memory shape-1Valid—
OpenAI → OpenAIIdle memory static0Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)-13.96Valid—
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → OpenAIThroughput (req/s)831concurrency 1; n=4,9925 ms
OpenAI → OpenAIThroughput (req/s)835concurrency 2; n=5,01210 ms
OpenAI → OpenAIThroughput (req/s)836concurrency 8; n=5,01850 ms
OpenAI → OpenAIThroughput (req/s)836concurrency 8; n=5,018100 ms
OpenAI → OpenAIThroughput (req/s)836concurrency 8; n=5,018Unbounded
OpenAI Responses → OpenAI——Not declared: recorded note 1—
Anthropic → OpenAI——Not declared: recorded note 1—
Cohere → OpenAI——Not declared: recorded note 1—
Gemini → Cohere——Not declared: recorded note 1—
Cohere → CohereAdded latency p50 (µs)1,128Valid · n=4,519 gateway / 248,527 direct—
Cohere → CohereAdded latency p99 (µs)3,714Valid · n=4,519 gateway / 248,527 direct—
Cohere → CohereGateway p99 at concurrency 1 (µs)3,750Valid · n=4,519—
Cohere → CohereDirect-to-mock p99 at concurrency 1 (µs)36Valid · n=248,527—
Cohere → CohereCPU per request (µs)1,331.2Valid—
Cohere → CohereRequests per CPU-second751.2Valid—
Cohere → CohereCost window concurrency8Valid—
Cohere → CohereCost window core utilisation0.28Valid—
Cohere → CohereCost window successful requests5,063Valid—
Cohere → CohereCost window throughput (req/s)843Valid—
Cohere → CohereCost window threads22Valid—
Cohere → CohereNon-voluntary context switches per request0.005Valid—
Cohere → CohereCost window major page faults0Valid—
Cohere → CohereAdded time to first event p50 (µs)—Untestable—
Cohere → CohereAdded time to first event p99 (µs)—Untestable—
Cohere → CohereAdded inter-frame gap p50 (µs)—Untestable—
Cohere → CohereAdded inter-frame gap p99 (µs)—Untestable—
Cohere → CohereStreams sustained (frames/s)—Untestable—
Cohere → CohereStreams sustained—Untestable—
Cohere → CohereIdle RSS (MiB)125Valid—
Cohere → CohereSteady-state RSS (MiB)245.9Valid—
Cohere → CohereRecovered RSS (MiB)141.5Valid—
Cohere → CoherePeak RSS (MiB)249.9Valid—
Cohere → CoherePeak RSS high-water mark (MiB)264Valid—
Cohere → CohereTime to plateau (s)—Not measured—
Cohere → CohereMemory growth (MiB/min)0.806Valid—
Cohere → CohereMemory load duration (s)301Valid—
Cohere → CohereMemory shape under load0Valid—
Cohere → CohereIdle memory shape-1Valid—
Cohere → CohereIdle memory static0Valid—
Cohere → CohereIdle memory growth (MiB/min)-14.17Valid—
Cohere → CohereThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Cohere → CohereThroughput (req/s)860concurrency 2; n=5,1615 ms
Cohere → CohereThroughput (req/s)860concurrency 2; n=5,16110 ms
Cohere → CohereThroughput (req/s)860concurrency 2; n=5,16150 ms
Cohere → CohereThroughput (req/s)860concurrency 2; n=5,161100 ms
Cohere → CohereThroughput (req/s)860concurrency 2; n=5,161Unbounded
Bedrock → Cohere——Not declared: recorded note 1—
OpenAI → CohereAdded latency p50 (µs)1,204Valid · n=4,255 gateway / 267,416 direct—
OpenAI → CohereAdded latency p99 (µs)3,792Valid · n=4,255 gateway / 267,416 direct—
OpenAI → CohereGateway p99 at concurrency 1 (µs)3,819Valid · n=4,255—
OpenAI → CohereDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=267,416—
OpenAI → CohereCPU per request (µs)1,383.8Valid—
OpenAI → CohereRequests per CPU-second722.6Valid—
OpenAI → CohereCost window concurrency8Valid—
OpenAI → CohereCost window core utilisation0.28Valid—
OpenAI → CohereCost window successful requests4,885Valid—
OpenAI → CohereCost window throughput (req/s)813Valid—
OpenAI → CohereCost window threads22Valid—
OpenAI → CohereNon-voluntary context switches per request0.006Valid—
OpenAI → CohereCost window major page faults0Valid—
OpenAI → CohereAdded time to first event p50 (µs)—Untestable—
OpenAI → CohereAdded time to first event p99 (µs)—Untestable—
OpenAI → CohereAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → CohereAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → CohereStreams sustained (frames/s)—Untestable—
OpenAI → CohereStreams sustained—Untestable—
OpenAI → CohereIdle RSS (MiB)124.9Valid—
OpenAI → CohereSteady-state RSS (MiB)244.8Valid—
OpenAI → CohereRecovered RSS (MiB)141.5Valid—
OpenAI → CoherePeak RSS (MiB)250.2Valid—
OpenAI → CoherePeak RSS high-water mark (MiB)264.8Valid—
OpenAI → CohereTime to plateau (s)—Not measured—
OpenAI → CohereMemory growth (MiB/min)0.676Valid—
OpenAI → CohereMemory load duration (s)301Valid—
OpenAI → CohereMemory shape under load0Valid—
OpenAI → CohereIdle memory shape-1Valid—
OpenAI → CohereIdle memory static0Valid—
OpenAI → CohereIdle memory growth (MiB/min)-14.25Valid—
OpenAI → CohereThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → CohereThroughput (req/s)821concurrency 2; n=4,9295 ms
OpenAI → CohereThroughput (req/s)828concurrency 4; n=4,97510 ms
OpenAI → CohereThroughput (req/s)828concurrency 4; n=4,97550 ms
OpenAI → CohereThroughput (req/s)828concurrency 4; n=4,975100 ms
OpenAI → CohereThroughput (req/s)828concurrency 4; n=4,975Unbounded
OpenAI Responses → Cohere——Not declared: recorded note 1—
Anthropic → Cohere——Not declared: recorded note 1—
Gemini → Anthropic——Not declared: recorded note 1—
Cohere → Anthropic——Not declared: recorded note 1—
Bedrock → Anthropic——Not declared: recorded note 1—
Anthropic → AnthropicAdded latency p50 (µs)1,196Valid · n=4,332 gateway / 243,420 direct—
Anthropic → AnthropicAdded latency p99 (µs)3,745Valid · n=4,332 gateway / 243,420 direct—
Anthropic → AnthropicGateway p99 at concurrency 1 (µs)3,781Valid · n=4,332—
Anthropic → AnthropicDirect-to-mock p99 at concurrency 1 (µs)36Valid · n=243,420—
Anthropic → AnthropicCPU per request (µs)1,549.2Valid—
Anthropic → AnthropicRequests per CPU-second645.5Valid—
Anthropic → AnthropicCost window concurrency8Valid—
Anthropic → AnthropicCost window core utilisation0.3Valid—
Anthropic → AnthropicCost window successful requests4,667Valid—
Anthropic → AnthropicCost window throughput (req/s)777Valid—
Anthropic → AnthropicCost window threads22Valid—
Anthropic → AnthropicNon-voluntary context switches per request0.007Valid—
Anthropic → AnthropicCost window major page faults0Valid—
Anthropic → AnthropicAdded time to first event p50 (µs)28,106Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded time to first event p99 (µs)29,379Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded inter-frame gap p50 (µs)7Valid · n=63 gateway / 63 direct—
Anthropic → AnthropicAdded inter-frame gap p99 (µs)400Valid · n=63 gateway / 63 direct—
Anthropic → AnthropicStreams sustained (frames/s)4,968.8Valid—
Anthropic → AnthropicStreams sustained1,087Valid—
Anthropic → AnthropicIdle RSS (MiB)125Valid—
Anthropic → AnthropicSteady-state RSS (MiB)245.3Valid—
Anthropic → AnthropicRecovered RSS (MiB)141.7Valid—
Anthropic → AnthropicPeak RSS (MiB)253.7Valid—
Anthropic → AnthropicPeak RSS high-water mark (MiB)268.4Valid—
Anthropic → AnthropicTime to plateau (s)—Not measured—
Anthropic → AnthropicMemory growth (MiB/min)-1.46Valid—
Anthropic → AnthropicMemory load duration (s)301Valid—
Anthropic → AnthropicMemory shape under load0Valid—
Anthropic → AnthropicIdle memory shape-1Valid—
Anthropic → AnthropicIdle memory static0Valid—
Anthropic → AnthropicIdle memory growth (MiB/min)-14.09Valid—
Anthropic → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → AnthropicThroughput (req/s)823concurrency 2; n=4,9415 ms
Anthropic → AnthropicThroughput (req/s)823concurrency 2; n=4,94110 ms
Anthropic → AnthropicThroughput (req/s)823concurrency 2; n=4,94150 ms
Anthropic → AnthropicThroughput (req/s)823concurrency 2; n=4,941100 ms
Anthropic → AnthropicThroughput (req/s)823concurrency 2; n=4,941Unbounded
OpenAI → AnthropicAdded latency p50 (µs)1,238Valid · n=4,169 gateway / 245,970 direct—
OpenAI → AnthropicAdded latency p99 (µs)3,814Valid · n=4,169 gateway / 245,970 direct—
OpenAI → AnthropicGateway p99 at concurrency 1 (µs)3,850Valid · n=4,169—
OpenAI → AnthropicDirect-to-mock p99 at concurrency 1 (µs)36Valid · n=245,970—
OpenAI → AnthropicCPU per request (µs)1,583.7Valid—
OpenAI → AnthropicRequests per CPU-second631.4Valid—
OpenAI → AnthropicCost window concurrency8Valid—
OpenAI → AnthropicCost window core utilisation0.297Valid—
OpenAI → AnthropicCost window successful requests4,521Valid—
OpenAI → AnthropicCost window throughput (req/s)753Valid—
OpenAI → AnthropicCost window threads22Valid—
OpenAI → AnthropicNon-voluntary context switches per request0.007Valid—
OpenAI → AnthropicCost window major page faults0Valid—
OpenAI → AnthropicAdded time to first event p50 (µs)28,321Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded time to first event p99 (µs)29,941Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded inter-frame gap p50 (µs)2Valid · n=63 gateway / 63 direct—
OpenAI → AnthropicAdded inter-frame gap p99 (µs)363Valid · n=63 gateway / 63 direct—
OpenAI → AnthropicStreams sustained (frames/s)4,586.9Valid—
OpenAI → AnthropicStreams sustained1,055Valid—
OpenAI → AnthropicIdle RSS (MiB)125.2Valid—
OpenAI → AnthropicSteady-state RSS (MiB)246.1Valid—
OpenAI → AnthropicRecovered RSS (MiB)141.9Valid—
OpenAI → AnthropicPeak RSS (MiB)250.6Valid—
OpenAI → AnthropicPeak RSS high-water mark (MiB)265.1Valid—
OpenAI → AnthropicTime to plateau (s)—Not measured—
OpenAI → AnthropicMemory growth (MiB/min)1.69Valid—
OpenAI → AnthropicMemory load duration (s)301Valid—
OpenAI → AnthropicMemory shape under load0Valid—
OpenAI → AnthropicIdle memory shape-1Valid—
OpenAI → AnthropicIdle memory static0Valid—
OpenAI → AnthropicIdle memory growth (MiB/min)-13.91Valid—
OpenAI → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → AnthropicThroughput (req/s)824concurrency 2; n=4,9485 ms
OpenAI → AnthropicThroughput (req/s)824concurrency 2; n=4,94810 ms
OpenAI → AnthropicThroughput (req/s)824concurrency 2; n=4,94850 ms
OpenAI → AnthropicThroughput (req/s)824concurrency 2; n=4,948100 ms
OpenAI → AnthropicThroughput (req/s)824concurrency 2; n=4,948Unbounded
OpenAI Responses → Anthropic——Not declared: recorded note 1—
Gemini → Bedrock——Not declared: recorded note 1—
Bedrock → BedrockAdded latency p50 (µs)1,316Valid · n=3,896 gateway / 249,657 direct—
Bedrock → BedrockAdded latency p99 (µs)4,150Valid · n=3,896 gateway / 249,657 direct—
Bedrock → BedrockGateway p99 at concurrency 1 (µs)4,186Valid · n=3,896—
Bedrock → BedrockDirect-to-mock p99 at concurrency 1 (µs)36Valid · n=249,657—
Bedrock → BedrockCPU per request (µs)1,534.8Valid—
Bedrock → BedrockRequests per CPU-second651.5Valid—
Bedrock → BedrockCost window concurrency8Valid—
Bedrock → BedrockCost window core utilisation0.283Valid—
Bedrock → BedrockCost window successful requests4,450Valid—
Bedrock → BedrockCost window throughput (req/s)741Valid—
Bedrock → BedrockCost window threads22Valid—
Bedrock → BedrockNon-voluntary context switches per request0.007Valid—
Bedrock → BedrockCost window major page faults0Valid—
Bedrock → BedrockAdded time to first event p50 (µs)—Untestable—
Bedrock → BedrockAdded time to first event p99 (µs)—Untestable—
Bedrock → BedrockAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → BedrockAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → BedrockStreams sustained (frames/s)—Untestable—
Bedrock → BedrockStreams sustained—Untestable—
Bedrock → BedrockIdle RSS (MiB)125Valid—
Bedrock → BedrockSteady-state RSS (MiB)248.5Valid—
Bedrock → BedrockRecovered RSS (MiB)142.3Valid—
Bedrock → BedrockPeak RSS (MiB)253.1Valid—
Bedrock → BedrockPeak RSS high-water mark (MiB)268.1Valid—
Bedrock → BedrockTime to plateau (s)—Not measured—
Bedrock → BedrockMemory growth (MiB/min)0.327Valid—
Bedrock → BedrockMemory load duration (s)301Valid—
Bedrock → BedrockMemory shape under load0Valid—
Bedrock → BedrockIdle memory shape-1Valid—
Bedrock → BedrockIdle memory static0Valid—
Bedrock → BedrockIdle memory growth (MiB/min)-14.19Valid—
Bedrock → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Bedrock → BedrockThroughput (req/s)757concurrency 2; n=4,5475 ms
Bedrock → BedrockThroughput (req/s)757concurrency 2; n=4,54710 ms
Bedrock → BedrockThroughput (req/s)757concurrency 2; n=4,54750 ms
Bedrock → BedrockThroughput (req/s)757concurrency 2; n=4,547100 ms
Bedrock → BedrockThroughput (req/s)757concurrency 2; n=4,547Unbounded
OpenAI Responses → Bedrock——Not declared: recorded note 1—
Anthropic → BedrockAdded latency p50 (µs)1,438Valid · n=3,579 gateway / 262,193 direct—
Anthropic → BedrockAdded latency p99 (µs)4,467Valid · n=3,579 gateway / 262,193 direct—
Anthropic → BedrockGateway p99 at concurrency 1 (µs)4,494Valid · n=3,579—
Anthropic → BedrockDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=262,193—
Anthropic → BedrockCPU per request (µs)1,632.6Valid—
Anthropic → BedrockRequests per CPU-second612.5Valid—
Anthropic → BedrockCost window concurrency8Valid—
Anthropic → BedrockCost window core utilisation0.282Valid—
Anthropic → BedrockCost window successful requests4,159Valid—
Anthropic → BedrockCost window throughput (req/s)693Valid—
Anthropic → BedrockCost window threads22Valid—
Anthropic → BedrockNon-voluntary context switches per request0.007Valid—
Anthropic → BedrockCost window major page faults0Valid—
Anthropic → BedrockAdded time to first event p50 (µs)—Untestable—
Anthropic → BedrockAdded time to first event p99 (µs)—Untestable—
Anthropic → BedrockAdded inter-frame gap p50 (µs)—Untestable—
Anthropic → BedrockAdded inter-frame gap p99 (µs)—Untestable—
Anthropic → BedrockStreams sustained (frames/s)—Untestable—
Anthropic → BedrockStreams sustained—Untestable—
Anthropic → BedrockIdle RSS (MiB)125.1Valid—
Anthropic → BedrockSteady-state RSS (MiB)250.3Valid—
Anthropic → BedrockRecovered RSS (MiB)142.5Valid—
Anthropic → BedrockPeak RSS (MiB)254.4Valid—
Anthropic → BedrockPeak RSS high-water mark (MiB)268.3Valid—
Anthropic → BedrockTime to plateau (s)—Not measured—
Anthropic → BedrockMemory growth (MiB/min)-2.66Valid—
Anthropic → BedrockMemory load duration (s)301Valid—
Anthropic → BedrockMemory shape under load0Valid—
Anthropic → BedrockIdle memory shape-1Valid—
Anthropic → BedrockIdle memory static0Valid—
Anthropic → BedrockIdle memory growth (MiB/min)-13.93Valid—
Anthropic → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → BedrockThroughput (req/s)703concurrency 1; n=4,2235 ms
Anthropic → BedrockThroughput (req/s)706concurrency 2; n=4,23710 ms
Anthropic → BedrockThroughput (req/s)706concurrency 2; n=4,23750 ms
Anthropic → BedrockThroughput (req/s)706concurrency 2; n=4,237100 ms
Anthropic → BedrockThroughput (req/s)706concurrency 2; n=4,237Unbounded
Cohere → Bedrock——Not declared: recorded note 1—
OpenAI → BedrockAdded latency p50 (µs)1,430Valid · n=3,563 gateway / 269,629 direct—
OpenAI → BedrockAdded latency p99 (µs)4,577Valid · n=3,563 gateway / 269,629 direct—
OpenAI → BedrockGateway p99 at concurrency 1 (µs)4,604Valid · n=3,563—
OpenAI → BedrockDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=269,629—
OpenAI → BedrockCPU per request (µs)1,664.2Valid—
OpenAI → BedrockRequests per CPU-second600.9Valid—
OpenAI → BedrockCost window concurrency8Valid—
OpenAI → BedrockCost window core utilisation0.283Valid—
OpenAI → BedrockCost window successful requests4,098Valid—
OpenAI → BedrockCost window throughput (req/s)682Valid—
OpenAI → BedrockCost window threads22Valid—
OpenAI → BedrockNon-voluntary context switches per request0.008Valid—
OpenAI → BedrockCost window major page faults0Valid—
OpenAI → BedrockAdded time to first event p50 (µs)—Untestable—
OpenAI → BedrockAdded time to first event p99 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → BedrockStreams sustained (frames/s)—Untestable—
OpenAI → BedrockStreams sustained—Untestable—
OpenAI → BedrockIdle RSS (MiB)125.2Valid—
OpenAI → BedrockSteady-state RSS (MiB)248.4Valid—
OpenAI → BedrockRecovered RSS (MiB)142.8Valid—
OpenAI → BedrockPeak RSS (MiB)253.9Valid—
OpenAI → BedrockPeak RSS high-water mark (MiB)268.4Valid—
OpenAI → BedrockTime to plateau (s)—Not measured—
OpenAI → BedrockMemory growth (MiB/min)-1.45Valid—
OpenAI → BedrockMemory load duration (s)301Valid—
OpenAI → BedrockMemory shape under load0Valid—
OpenAI → BedrockIdle memory shape-1Valid—
OpenAI → BedrockIdle memory static0Valid—
OpenAI → BedrockIdle memory growth (MiB/min)-14.02Valid—
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → BedrockThroughput (req/s)685concurrency 1; n=4,1145 ms
OpenAI → BedrockThroughput (req/s)696concurrency 2; n=4,17710 ms
OpenAI → BedrockThroughput (req/s)696concurrency 2; n=4,17750 ms
OpenAI → BedrockThroughput (req/s)696concurrency 2; n=4,177100 ms
OpenAI → BedrockThroughput (req/s)696concurrency 2; n=4,177Unbounded
OpenAI Responses → Gemini——Not declared: recorded note 1—
Anthropic → Gemini——Not declared: recorded note 1—
Gemini → GeminiAdded latency p50 (µs)1,145Valid · n=4,389 gateway / 249,981 direct—
Gemini → GeminiAdded latency p99 (µs)4,018Valid · n=4,389 gateway / 249,981 direct—
Gemini → GeminiGateway p99 at concurrency 1 (µs)4,054Valid · n=4,389—
Gemini → GeminiDirect-to-mock p99 at concurrency 1 (µs)36Valid · n=249,981—
Gemini → GeminiCPU per request (µs)1,361.8Valid—
Gemini → GeminiRequests per CPU-second734.3Valid—
Gemini → GeminiCost window concurrency8Valid—
Gemini → GeminiCost window core utilisation0.286Valid—
Gemini → GeminiCost window successful requests5,052Valid—
Gemini → GeminiCost window throughput (req/s)841Valid—
Gemini → GeminiCost window threads22Valid—
Gemini → GeminiNon-voluntary context switches per request0.008Valid—
Gemini → GeminiCost window major page faults0Valid—
Gemini → GeminiAdded time to first event p50 (µs)—Untestable—
Gemini → GeminiAdded time to first event p99 (µs)—Untestable—
Gemini → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Gemini → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Gemini → GeminiStreams sustained (frames/s)—Untestable—
Gemini → GeminiStreams sustained—Untestable—
Gemini → GeminiIdle RSS (MiB)124.1Valid—
Gemini → GeminiSteady-state RSS (MiB)245.7Valid—
Gemini → GeminiRecovered RSS (MiB)140Valid—
Gemini → GeminiPeak RSS (MiB)249.9Valid—
Gemini → GeminiPeak RSS high-water mark (MiB)263.9Valid—
Gemini → GeminiTime to plateau (s)—Not measured—
Gemini → GeminiMemory growth (MiB/min)-0.813Valid—
Gemini → GeminiMemory load duration (s)301Valid—
Gemini → GeminiMemory shape under load0Valid—
Gemini → GeminiIdle memory shape-1Valid—
Gemini → GeminiIdle memory static0Valid—
Gemini → GeminiIdle memory growth (MiB/min)-14.01Valid—
Gemini → GeminiThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Gemini → GeminiThroughput (req/s)865concurrency 2; n=5,1935 ms
Gemini → GeminiThroughput (req/s)868concurrency 4; n=5,21210 ms
Gemini → GeminiThroughput (req/s)868concurrency 4; n=5,21250 ms
Gemini → GeminiThroughput (req/s)868concurrency 4; n=5,212100 ms
Gemini → GeminiThroughput (req/s)868concurrency 4; n=5,212Unbounded
OpenAI → GeminiAdded latency p50 (µs)1,199Valid · n=4,241 gateway / 249,376 direct—
OpenAI → GeminiAdded latency p99 (µs)3,969Valid · n=4,241 gateway / 249,376 direct—
OpenAI → GeminiGateway p99 at concurrency 1 (µs)4,005Valid · n=4,241—
OpenAI → GeminiDirect-to-mock p99 at concurrency 1 (µs)36Valid · n=249,376—
OpenAI → GeminiCPU per request (µs)1,420.5Valid—
OpenAI → GeminiRequests per CPU-second704Valid—
OpenAI → GeminiCost window concurrency8Valid—
OpenAI → GeminiCost window core utilisation0.283Valid—
OpenAI → GeminiCost window successful requests4,801Valid—
OpenAI → GeminiCost window throughput (req/s)800Valid—
OpenAI → GeminiCost window threads22Valid—
OpenAI → GeminiNon-voluntary context switches per request0.006Valid—
OpenAI → GeminiCost window major page faults0Valid—
OpenAI → GeminiAdded time to first event p50 (µs)—Untestable—
OpenAI → GeminiAdded time to first event p99 (µs)—Untestable—
OpenAI → GeminiAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → GeminiAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → GeminiStreams sustained (frames/s)—Untestable—
OpenAI → GeminiStreams sustained—Untestable—
OpenAI → GeminiIdle RSS (MiB)125.2Valid—
OpenAI → GeminiSteady-state RSS (MiB)247Valid—
OpenAI → GeminiRecovered RSS (MiB)142.2Valid—
OpenAI → GeminiPeak RSS (MiB)250.6Valid—
OpenAI → GeminiPeak RSS high-water mark (MiB)265.4Valid—
OpenAI → GeminiTime to plateau (s)—Not measured—
OpenAI → GeminiMemory growth (MiB/min)-1.36Valid—
OpenAI → GeminiMemory load duration (s)301Valid—
OpenAI → GeminiMemory shape under load0Valid—
OpenAI → GeminiIdle memory shape-1Valid—
OpenAI → GeminiIdle memory static0Valid—
OpenAI → GeminiIdle memory growth (MiB/min)-14.12Valid—
OpenAI → GeminiThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → GeminiThroughput (req/s)800concurrency 1; n=4,8055 ms
OpenAI → GeminiThroughput (req/s)809concurrency 2; n=4,85710 ms
OpenAI → GeminiThroughput (req/s)809concurrency 2; n=4,85750 ms
OpenAI → GeminiThroughput (req/s)809concurrency 2; n=4,857100 ms
OpenAI → GeminiThroughput (req/s)809concurrency 2; n=4,857Unbounded
Bedrock → Gemini——Not declared: recorded note 1—
Cohere → Gemini——Not declared: recorded note 1—
Anthropic → OpenAI Responses——Not declared: recorded note 1—
OpenAI Responses → OpenAI ResponsesAdded latency p50 (µs)1,238Valid · n=4,169 gateway / 265,251 direct—
OpenAI Responses → OpenAI ResponsesAdded latency p99 (µs)3,854Valid · n=4,169 gateway / 265,251 direct—
OpenAI Responses → OpenAI ResponsesGateway p99 at concurrency 1 (µs)3,881Valid · n=4,169—
OpenAI Responses → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=265,251—
OpenAI Responses → OpenAI ResponsesCPU per request (µs)1,427.7Valid—
OpenAI Responses → OpenAI ResponsesRequests per CPU-second700.4Valid—
OpenAI Responses → OpenAI ResponsesCost window concurrency8Valid—
OpenAI Responses → OpenAI ResponsesCost window core utilisation0.285Valid—
OpenAI Responses → OpenAI ResponsesCost window successful requests4,805Valid—
OpenAI Responses → OpenAI ResponsesCost window throughput (req/s)800Valid—
OpenAI Responses → OpenAI ResponsesCost window threads22Valid—
OpenAI Responses → OpenAI ResponsesNon-voluntary context switches per request0.004Valid—
OpenAI Responses → OpenAI ResponsesCost window major page faults0Valid—
OpenAI Responses → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained—Untestable—
OpenAI Responses → OpenAI ResponsesIdle RSS (MiB)125Valid—
OpenAI Responses → OpenAI ResponsesSteady-state RSS (MiB)243.7Valid—
OpenAI Responses → OpenAI ResponsesRecovered RSS (MiB)142Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS (MiB)249.5Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS high-water mark (MiB)264.3Valid—
OpenAI Responses → OpenAI ResponsesTime to plateau (s)—Not measured—
OpenAI Responses → OpenAI ResponsesMemory growth (MiB/min)-1.94Valid—
OpenAI Responses → OpenAI ResponsesMemory load duration (s)301Valid—
OpenAI Responses → OpenAI ResponsesMemory shape under load0Valid—
OpenAI Responses → OpenAI ResponsesIdle memory shape-1Valid—
OpenAI Responses → OpenAI ResponsesIdle memory static0Valid—
OpenAI Responses → OpenAI ResponsesIdle memory growth (MiB/min)-14.08Valid—
OpenAI Responses → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)809concurrency 2; n=4,8555 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)809concurrency 2; n=4,85510 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)809concurrency 2; n=4,85550 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)809concurrency 2; n=4,855100 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)809concurrency 2; n=4,855Unbounded
OpenAI → OpenAI Responses——Not declared: recorded note 1—
Gemini → OpenAI Responses——Not declared: recorded note 1—
Bedrock → OpenAI Responses——Not declared: recorded note 1—
Cohere → OpenAI Responses——Not declared: recorded note 1—

Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

Ingress → upstreamCell stateProbe HTTP statusRecorded reasonRecorded detailEvidence scopeResponse excerptTested build
Bedrock → OpenAINot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Gemini → OpenAINot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
OpenAI → OpenAIServed———Recorded run evidence—portkeyai/gateway:1.15.2
OpenAI Responses → OpenAINot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Anthropic → OpenAINot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Cohere → OpenAINot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Gemini → CohereNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Cohere → CohereServed———Recorded run evidence—portkeyai/gateway:1.15.2
Bedrock → CohereNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
OpenAI → CohereServed———Recorded run evidence—portkeyai/gateway:1.15.2
OpenAI Responses → CohereNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Anthropic → CohereNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Gemini → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Cohere → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Bedrock → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Anthropic → AnthropicServed———Recorded run evidence—portkeyai/gateway:1.15.2
OpenAI → AnthropicServed———Recorded run evidence—portkeyai/gateway:1.15.2
OpenAI Responses → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Gemini → BedrockNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Bedrock → BedrockServed———Recorded run evidence—portkeyai/gateway:1.15.2
OpenAI Responses → BedrockNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Anthropic → BedrockServed———Recorded run evidence—portkeyai/gateway:1.15.2
Cohere → BedrockNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
OpenAI → BedrockServed———Recorded run evidence—portkeyai/gateway:1.15.2
OpenAI Responses → GeminiNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Anthropic → GeminiNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Gemini → GeminiServed———Recorded run evidence—portkeyai/gateway:1.15.2
OpenAI → GeminiServed———Recorded run evidence—portkeyai/gateway:1.15.2
Bedrock → GeminiNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Cohere → GeminiNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Anthropic → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
OpenAI Responses → OpenAI ResponsesServed———Recorded run evidence—portkeyai/gateway:1.15.2
OpenAI → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Gemini → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Bedrock → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Cohere → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—portkeyai/gateway:1.15.2
Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2
Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8
Load generator SHA-256
not recorded
net.core.somaxconn
65,535
net.ipv4.tcp_max_syn_backlog
65,535

Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

Ingress → upstreamConcurrencyLanes by HTTP statusLanes with no response
OpenAI → OpenAI1HTTP 200: 10
OpenAI → OpenAI2HTTP 200: 20
OpenAI → OpenAI4HTTP 200: 40
OpenAI → OpenAI8HTTP 200: 80
OpenAI → OpenAI16HTTP 200: 160
OpenAI → OpenAI32HTTP 200: 320
OpenAI → OpenAI64HTTP 200: 640
OpenAI → OpenAI128HTTP 200: 1280
OpenAI → OpenAI256HTTP 200: 2560
OpenAI → OpenAI512HTTP 200: 5120
OpenAI → OpenAI1,024HTTP 200: 1,0240
OpenAI → OpenAI2,048HTTP 200: 1,810238
OpenAI → OpenAI1,536HTTP 200: 703833
OpenAI → OpenAI1,280HTTP 200: 1,2800
OpenAI → OpenAI1,408HTTP 200: 1,4080
OpenAI → OpenAI1,472HTTP 200: 1,4720
OpenAI → OpenAI1,504HTTP 200: 1,5040
OpenAI → OpenAI1,520HTTP 200: 1,50713
OpenAI → OpenAI1,512HTTP 200: 1,5120
OpenAI → OpenAI1,516HTTP 200: 1,5160
OpenAI → OpenAI1,518HTTP 200: 1,5180
OpenAI → OpenAI1,519HTTP 200: 1,5190
OpenAI → OpenAI1,519HTTP 200: 1,44574
OpenAI → OpenAI1,519HTTP 200: 1,5190
Anthropic → Anthropic1HTTP 200: 10
Anthropic → Anthropic2HTTP 200: 20
Anthropic → Anthropic4HTTP 200: 40
Anthropic → Anthropic8HTTP 200: 80
Anthropic → Anthropic16HTTP 200: 160
Anthropic → Anthropic32HTTP 200: 320
Anthropic → Anthropic64HTTP 200: 640
Anthropic → Anthropic128HTTP 200: 1280
Anthropic → Anthropic256HTTP 200: 2560
Anthropic → Anthropic512HTTP 200: 5120
Anthropic → Anthropic1,024HTTP 200: 1,0240
Anthropic → Anthropic2,048HTTP 200: 1,798250
Anthropic → Anthropic1,536HTTP 200: 1,151385
Anthropic → Anthropic1,280HTTP 200: 918362
Anthropic → Anthropic1,152HTTP 200: 919233
Anthropic → Anthropic1,088HTTP 200: 1,0880
Anthropic → Anthropic1,056HTTP 200: 1,0560
Anthropic → Anthropic1,072HTTP 200: 1,0720
Anthropic → Anthropic1,080HTTP 200: 1,0800
Anthropic → Anthropic1,084HTTP 200: 1,0840
Anthropic → Anthropic1,086HTTP 200: 1,0860
Anthropic → Anthropic1,087HTTP 200: 1,0870
Anthropic → Anthropic1,087HTTP 200: 1,0870
Anthropic → Anthropic1,087HTTP 200: 1,0870
OpenAI → Anthropic1HTTP 200: 10
OpenAI → Anthropic2HTTP 200: 20
OpenAI → Anthropic4HTTP 200: 40
OpenAI → Anthropic8HTTP 200: 80
OpenAI → Anthropic16HTTP 200: 160
OpenAI → Anthropic32HTTP 200: 320
OpenAI → Anthropic64HTTP 200: 640
OpenAI → Anthropic128HTTP 200: 1280
OpenAI → Anthropic256HTTP 200: 2560
OpenAI → Anthropic512HTTP 200: 5120
OpenAI → Anthropic1,024HTTP 200: 1,0240
OpenAI → Anthropic2,048HTTP 200: 1,626422
OpenAI → Anthropic1,536HTTP 200: 1,211325
OpenAI → Anthropic1,280HTTP 200: 854426
OpenAI → Anthropic1,152HTTP 200: 950202
OpenAI → Anthropic1,088HTTP 200: 724364
OpenAI → Anthropic1,056HTTP 200: 1,0506
OpenAI → Anthropic1,040HTTP 200: 1,0400
OpenAI → Anthropic1,048HTTP 200: 1,0480
OpenAI → Anthropic1,052HTTP 200: 1,0520
OpenAI → Anthropic1,054HTTP 200: 1,0540
OpenAI → Anthropic1,055HTTP 200: 1,0550
OpenAI → Anthropic1,055HTTP 200: 1,0550
OpenAI → Anthropic1,055HTTP 200: 1,0550

Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

  1. Pinned v1.15.2 source ca7712947ec8c0887803990b746740d36011c697: src/index.ts registers chat-completions, messages, responses and raw proxy routes; provider operation tables determine translations. Recorder preflight proves 11 exact protocol pairs. Raw proxy passes native requests through without cross-protocol conversion, so only its matching diagonal is enabled. OpenAI Chat Completions and Responses share a provider but remain distinct upstream operations. Evidence: results/diagnostics/refresh-configuration/portkey-protocol-preflight.json
  2. Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

Evidence for the current measurements

Configuration recorded by the run

Not recorded.

Compare a shared protocol cell

Historical measurements