Kong benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Other · recorded build: kong:3.9.4. Measured: .

JSON · Markdown · CSV · Project source · Interactive dashboard

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

Published measurements

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
Cohere → OpenAI——Not declared: recorded note 1—
Bedrock → OpenAI——Not declared: recorded note 1—
Anthropic → OpenAI——Not declared: recorded note 1—
OpenAI → OpenAIAdded latency p50 (µs)213Valid · n=23,861 gateway / 246,038 direct—
OpenAI → OpenAIAdded latency p99 (µs)410Valid · n=23,861 gateway / 246,038 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)440Valid · n=23,861—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)30Valid · n=246,038—
OpenAI → OpenAICPU per request (µs)220.7Valid—
OpenAI → OpenAIRequests per CPU-second4,531.3Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.787Valid—
OpenAI → OpenAICost window successful requests85,868Valid—
OpenAI → OpenAICost window throughput (req/s)14,309Valid—
OpenAI → OpenAICost window threads5Valid—
OpenAI → OpenAINon-voluntary context switches per request0.000734Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)106,009Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)106,433Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)168,648Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)3,731.9Valid—
OpenAI → OpenAIStreams sustained78Valid—
OpenAI → OpenAIIdle RSS (MiB)406.2Valid—
OpenAI → OpenAISteady-state RSS (MiB)617.4Valid—
OpenAI → OpenAIRecovered RSS (MiB)617.5Valid—
OpenAI → OpenAIPeak RSS (MiB)617.7Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)618.6Valid—
OpenAI → OpenAITime to plateau (s)66Valid—
OpenAI → OpenAIMemory growth (MiB/min)0.105Valid—
OpenAI → OpenAIMemory load duration (s)300Valid—
OpenAI → OpenAIMemory shape under load—Not measured—
OpenAI → OpenAIIdle memory shape—Not measured—
OpenAI → OpenAIIdle memory static1Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)0.241Valid—
OpenAI → OpenAIThroughput (req/s)13,315concurrency 4; n=79,9051 ms
OpenAI → OpenAIThroughput (req/s)19,190concurrency 8; n=115,1575 ms
OpenAI → OpenAIThroughput (req/s)21,287concurrency 64; n=127,72510 ms
OpenAI → OpenAIThroughput (req/s)22,171concurrency 128; n=133,04650 ms
OpenAI → OpenAIThroughput (req/s)22,171concurrency 128; n=133,046100 ms
OpenAI → OpenAIThroughput (req/s)22,171concurrency 128; n=133,046Unbounded
Gemini → OpenAI——Not declared: recorded note 1—
OpenAI Responses → OpenAI——Not declared: recorded note 1—
Anthropic → Anthropic——Not declared: recorded note 1—
Cohere → Anthropic——Not declared: recorded note 1—
OpenAI → AnthropicAdded latency p50 (µs)211Valid · n=24,037 gateway / 241,342 direct—
OpenAI → AnthropicAdded latency p99 (µs)420Valid · n=24,037 gateway / 241,342 direct—
OpenAI → AnthropicGateway p99 at concurrency 1 (µs)450Valid · n=24,037—
OpenAI → AnthropicDirect-to-mock p99 at concurrency 1 (µs)30Valid · n=241,342—
OpenAI → AnthropicCPU per request (µs)221.7Valid—
OpenAI → AnthropicRequests per CPU-second4,509.8Valid—
OpenAI → AnthropicCost window concurrency8Valid—
OpenAI → AnthropicCost window core utilisation0.972Valid—
OpenAI → AnthropicCost window successful requests105,485Valid—
OpenAI → AnthropicCost window throughput (req/s)17,578Valid—
OpenAI → AnthropicCost window threads5Valid—
OpenAI → AnthropicNon-voluntary context switches per request0.002Valid—
OpenAI → AnthropicCost window major page faults0Valid—
OpenAI → AnthropicAdded time to first event p50 (µs)84,892Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded time to first event p99 (µs)85,699Valid · n=100 gateway / 100 direct—
OpenAI → AnthropicAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → AnthropicAdded inter-frame gap p99 (µs)147,628Valid · n=63 gateway / 63 direct—
OpenAI → AnthropicStreams sustained (frames/s)12,509.3Valid—
OpenAI → AnthropicStreams sustained264Valid—
OpenAI → AnthropicIdle RSS (MiB)406.1Valid—
OpenAI → AnthropicSteady-state RSS (MiB)614.6Valid—
OpenAI → AnthropicRecovered RSS (MiB)614.7Valid—
OpenAI → AnthropicPeak RSS (MiB)614.9Valid—
OpenAI → AnthropicPeak RSS high-water mark (MiB)616Valid—
OpenAI → AnthropicTime to plateau (s)78Valid—
OpenAI → AnthropicMemory growth (MiB/min)0.111Valid—
OpenAI → AnthropicMemory load duration (s)300Valid—
OpenAI → AnthropicMemory shape under load—Not measured—
OpenAI → AnthropicIdle memory shape—Not measured—
OpenAI → AnthropicIdle memory static1Valid—
OpenAI → AnthropicIdle memory growth (MiB/min)0.329Valid—
OpenAI → AnthropicThroughput (req/s)9,859concurrency 4; n=59,1651 ms
OpenAI → AnthropicThroughput (req/s)16,975concurrency 8; n=101,8655 ms
OpenAI → AnthropicThroughput (req/s)18,516concurrency 64; n=111,10010 ms
OpenAI → AnthropicThroughput (req/s)19,147concurrency 128; n=114,89650 ms
OpenAI → AnthropicThroughput (req/s)19,147concurrency 128; n=114,896100 ms
OpenAI → AnthropicThroughput (req/s)19,147concurrency 128; n=114,896Unbounded
OpenAI Responses → Anthropic——Not declared: recorded note 1—
Gemini → Anthropic——Not declared: recorded note 1—
Bedrock → Anthropic——Not declared: recorded note 1—
OpenAI → CohereAdded latency p50 (µs)207Valid · n=24,758 gateway / 238,237 direct—
OpenAI → CohereAdded latency p99 (µs)423Valid · n=24,758 gateway / 238,237 direct—
OpenAI → CohereGateway p99 at concurrency 1 (µs)454Valid · n=24,758—
OpenAI → CohereDirect-to-mock p99 at concurrency 1 (µs)31Valid · n=238,237—
OpenAI → CohereCPU per request (µs)216.6Valid—
OpenAI → CohereRequests per CPU-second4,616.9Valid—
OpenAI → CohereCost window concurrency8Valid—
OpenAI → CohereCost window core utilisation0.952Valid—
OpenAI → CohereCost window successful requests105,820Valid—
OpenAI → CohereCost window throughput (req/s)17,634Valid—
OpenAI → CohereCost window threads5Valid—
OpenAI → CohereNon-voluntary context switches per request0.001Valid—
OpenAI → CohereCost window major page faults0Valid—
OpenAI → CohereAdded time to first event p50 (µs)—Untestable—
OpenAI → CohereAdded time to first event p99 (µs)—Untestable—
OpenAI → CohereAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → CohereAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → CohereStreams sustained (frames/s)—Untestable—
OpenAI → CohereStreams sustained—Untestable—
OpenAI → CohereIdle RSS (MiB)408.1Valid—
OpenAI → CohereSteady-state RSS (MiB)620.3Valid—
OpenAI → CohereRecovered RSS (MiB)620.7Valid—
OpenAI → CoherePeak RSS (MiB)620.8Valid—
OpenAI → CoherePeak RSS high-water mark (MiB)622.1Valid—
OpenAI → CohereTime to plateau (s)66Valid—
OpenAI → CohereMemory growth (MiB/min)0.409Valid—
OpenAI → CohereMemory load duration (s)300Valid—
OpenAI → CohereMemory shape under load—Not measured—
OpenAI → CohereIdle memory shape—Not measured—
OpenAI → CohereIdle memory static1Valid—
OpenAI → CohereIdle memory growth (MiB/min)0.256Valid—
OpenAI → CohereThroughput (req/s)12,570concurrency 4; n=75,4301 ms
OpenAI → CohereThroughput (req/s)18,172concurrency 16; n=109,0455 ms
OpenAI → CohereThroughput (req/s)18,962concurrency 32; n=113,77710 ms
OpenAI → CohereThroughput (req/s)20,178concurrency 128; n=121,08850 ms
OpenAI → CohereThroughput (req/s)20,178concurrency 128; n=121,088100 ms
OpenAI → CohereThroughput (req/s)20,178concurrency 128; n=121,088Unbounded
Cohere → Cohere——Not declared: recorded note 1—
OpenAI Responses → Cohere——Not declared: recorded note 1—
Gemini → Cohere——Not declared: recorded note 1—
Bedrock → Cohere——Not declared: recorded note 1—
Anthropic → Cohere——Not declared: recorded note 1—
Cohere → OpenAI Responses——Not declared: recorded note 1—
OpenAI → OpenAI Responses——Not declared: recorded note 1—
Anthropic → OpenAI Responses——Not declared: recorded note 1—
Gemini → OpenAI Responses——Not declared: recorded note 1—
Bedrock → OpenAI Responses——Not declared: recorded note 1—
OpenAI Responses → OpenAI Responses——Not declared: recorded note 1—
OpenAI → GeminiAdded latency p50 (µs)214Valid · n=23,733 gateway / 242,637 direct—
OpenAI → GeminiAdded latency p99 (µs)438Valid · n=23,733 gateway / 242,637 direct—
OpenAI → GeminiGateway p99 at concurrency 1 (µs)468Valid · n=23,733—
OpenAI → GeminiDirect-to-mock p99 at concurrency 1 (µs)30Valid · n=242,637—
OpenAI → GeminiCPU per request (µs)214.8Valid—
OpenAI → GeminiRequests per CPU-second4,656Valid—
OpenAI → GeminiCost window concurrency8Valid—
OpenAI → GeminiCost window core utilisation0.971Valid—
OpenAI → GeminiCost window successful requests108,858Valid—
OpenAI → GeminiCost window throughput (req/s)18,140Valid—
OpenAI → GeminiCost window threads5Valid—
OpenAI → GeminiNon-voluntary context switches per request0.001Valid—
OpenAI → GeminiCost window major page faults0Valid—
OpenAI → GeminiAdded time to first event p50 (µs)—Untestable—
OpenAI → GeminiAdded time to first event p99 (µs)—Untestable—
OpenAI → GeminiAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → GeminiAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → GeminiStreams sustained (frames/s)—Untestable—
OpenAI → GeminiStreams sustained—Untestable—
OpenAI → GeminiIdle RSS (MiB)408.6Valid—
OpenAI → GeminiSteady-state RSS (MiB)622.8Valid—
OpenAI → GeminiRecovered RSS (MiB)623Valid—
OpenAI → GeminiPeak RSS (MiB)623.1Valid—
OpenAI → GeminiPeak RSS high-water mark (MiB)624.2Valid—
OpenAI → GeminiTime to plateau (s)66Valid—
OpenAI → GeminiMemory growth (MiB/min)0.261Valid—
OpenAI → GeminiMemory load duration (s)300Valid—
OpenAI → GeminiMemory shape under load—Not measured—
OpenAI → GeminiIdle memory shape—Not measured—
OpenAI → GeminiIdle memory static1Valid—
OpenAI → GeminiIdle memory growth (MiB/min)0.387Valid—
OpenAI → GeminiThroughput (req/s)18,455concurrency 8; n=110,7481 ms
OpenAI → GeminiThroughput (req/s)18,455concurrency 8; n=110,7485 ms
OpenAI → GeminiThroughput (req/s)19,059concurrency 32; n=114,36810 ms
OpenAI → GeminiThroughput (req/s)19,713concurrency 128; n=118,29650 ms
OpenAI → GeminiThroughput (req/s)19,713concurrency 128; n=118,296100 ms
OpenAI → GeminiThroughput (req/s)19,713concurrency 128; n=118,296Unbounded
OpenAI Responses → Gemini——Not declared: recorded note 1—
Cohere → Gemini——Not declared: recorded note 1—
Anthropic → Gemini——Not declared: recorded note 1—
Gemini → GeminiAdded latency p50 (µs)193Valid · n=26,272 gateway / 244,214 direct—
Gemini → GeminiAdded latency p99 (µs)402Valid · n=26,272 gateway / 244,214 direct—
Gemini → GeminiGateway p99 at concurrency 1 (µs)432Valid · n=26,272—
Gemini → GeminiDirect-to-mock p99 at concurrency 1 (µs)30Valid · n=244,214—
Gemini → GeminiCPU per request (µs)195.5Valid—
Gemini → GeminiRequests per CPU-second5,115.7Valid—
Gemini → GeminiCost window concurrency8Valid—
Gemini → GeminiCost window core utilisation0.984Valid—
Gemini → GeminiCost window successful requests121,242Valid—
Gemini → GeminiCost window throughput (req/s)20,204Valid—
Gemini → GeminiCost window threads5Valid—
Gemini → GeminiNon-voluntary context switches per request0.000759Valid—
Gemini → GeminiCost window major page faults0Valid—
Gemini → GeminiAdded time to first event p50 (µs)—Untestable—
Gemini → GeminiAdded time to first event p99 (µs)—Untestable—
Gemini → GeminiAdded inter-frame gap p50 (µs)—Untestable—
Gemini → GeminiAdded inter-frame gap p99 (µs)—Untestable—
Gemini → GeminiStreams sustained (frames/s)—Untestable—
Gemini → GeminiStreams sustained—Untestable—
Gemini → GeminiIdle RSS (MiB)405.8Valid—
Gemini → GeminiSteady-state RSS (MiB)619.2Valid—
Gemini → GeminiRecovered RSS (MiB)619.6Valid—
Gemini → GeminiPeak RSS (MiB)619.8Valid—
Gemini → GeminiPeak RSS high-water mark (MiB)621Valid—
Gemini → GeminiTime to plateau (s)66Valid—
Gemini → GeminiMemory growth (MiB/min)0.356Valid—
Gemini → GeminiMemory load duration (s)300Valid—
Gemini → GeminiMemory shape under load—Not measured—
Gemini → GeminiIdle memory shape—Not measured—
Gemini → GeminiIdle memory static1Valid—
Gemini → GeminiIdle memory growth (MiB/min)0.467Valid—
Gemini → GeminiThroughput (req/s)13,349concurrency 4; n=80,1041 ms
Gemini → GeminiThroughput (req/s)20,568concurrency 32; n=123,4125 ms
Gemini → GeminiThroughput (req/s)21,659concurrency 64; n=129,96410 ms
Gemini → GeminiThroughput (req/s)21,659concurrency 64; n=129,96450 ms
Gemini → GeminiThroughput (req/s)21,659concurrency 64; n=129,964100 ms
Gemini → GeminiThroughput (req/s)21,659concurrency 64; n=129,964Unbounded
Bedrock → Gemini——Not declared: recorded note 1—
Anthropic → Bedrock——Not declared: recorded note 1—
OpenAI Responses → Bedrock——Not declared: recorded note 1—
Cohere → Bedrock——Not declared: recorded note 1—
OpenAI → BedrockAdded latency p50 (µs)1,038Valid · n=5,167 gateway / 237,310 direct—
OpenAI → BedrockAdded latency p99 (µs)2,133Valid · n=5,167 gateway / 237,310 direct—
OpenAI → BedrockGateway p99 at concurrency 1 (µs)2,164Valid · n=5,167—
OpenAI → BedrockDirect-to-mock p99 at concurrency 1 (µs)31Valid · n=237,310—
OpenAI → BedrockCPU per request (µs)1,041.3Valid—
OpenAI → BedrockRequests per CPU-second960.3Valid—
OpenAI → BedrockCost window concurrency8Valid—
OpenAI → BedrockCost window core utilisation0.985Valid—
OpenAI → BedrockCost window successful requests22,779Valid—
OpenAI → BedrockCost window throughput (req/s)3,796Valid—
OpenAI → BedrockCost window threads5Valid—
OpenAI → BedrockNon-voluntary context switches per request0.004Valid—
OpenAI → BedrockCost window major page faults0Valid—
OpenAI → BedrockAdded time to first event p50 (µs)—Untestable—
OpenAI → BedrockAdded time to first event p99 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → BedrockStreams sustained (frames/s)—Untestable—
OpenAI → BedrockStreams sustained—Untestable—
OpenAI → BedrockIdle RSS (MiB)407.8Valid—
OpenAI → BedrockSteady-state RSS (MiB)3,411.8Valid—
OpenAI → BedrockRecovered RSS (MiB)3,448.5Valid—
OpenAI → BedrockPeak RSS (MiB)3,466.1Valid—
OpenAI → BedrockPeak RSS high-water mark (MiB)3,477Valid—
OpenAI → BedrockTime to plateau (s)—Not measured—
OpenAI → BedrockMemory growth (MiB/min)10.9Valid—
OpenAI → BedrockMemory load duration (s)300Valid—
OpenAI → BedrockMemory shape under load0Valid—
OpenAI → BedrockIdle memory shape—Not measured—
OpenAI → BedrockIdle memory static1Valid—
OpenAI → BedrockIdle memory growth (MiB/min)0.341Valid—
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → BedrockThroughput (req/s)3,961concurrency 8; n=23,7725 ms
OpenAI → BedrockThroughput (req/s)3,961concurrency 8; n=23,77210 ms
OpenAI → BedrockThroughput (req/s)3,961concurrency 8; n=23,77250 ms
OpenAI → BedrockThroughput (req/s)3,961concurrency 8; n=23,772100 ms
OpenAI → BedrockThroughput (req/s)3,961concurrency 8; n=23,772Unbounded
Gemini → Bedrock——Not declared: recorded note 1—
Bedrock → BedrockAdded latency p50 (µs)957Valid · n=5,584 gateway / 241,921 direct—
Bedrock → BedrockAdded latency p99 (µs)1,988Valid · n=5,584 gateway / 241,921 direct—
Bedrock → BedrockGateway p99 at concurrency 1 (µs)2,018Valid · n=5,584—
Bedrock → BedrockDirect-to-mock p99 at concurrency 1 (µs)30Valid · n=241,921—
Bedrock → BedrockCPU per request (µs)1,094.7Valid—
Bedrock → BedrockRequests per CPU-second913.5Valid—
Bedrock → BedrockCost window concurrency8Valid—
Bedrock → BedrockCost window core utilisation0.989Valid—
Bedrock → BedrockCost window successful requests21,741Valid—
Bedrock → BedrockCost window throughput (req/s)3,623Valid—
Bedrock → BedrockCost window threads5Valid—
Bedrock → BedrockNon-voluntary context switches per request0.007Valid—
Bedrock → BedrockCost window major page faults0Valid—
Bedrock → BedrockAdded time to first event p50 (µs)—Untestable—
Bedrock → BedrockAdded time to first event p99 (µs)—Untestable—
Bedrock → BedrockAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → BedrockAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → BedrockStreams sustained (frames/s)—Untestable—
Bedrock → BedrockStreams sustained—Untestable—
Bedrock → BedrockIdle RSS (MiB)407.7Valid—
Bedrock → BedrockSteady-state RSS (MiB)3,782.2Valid—
Bedrock → BedrockRecovered RSS (MiB)3,821.5Valid—
Bedrock → BedrockPeak RSS (MiB)3,840.8Valid—
Bedrock → BedrockPeak RSS high-water mark (MiB)3,862.1Valid—
Bedrock → BedrockTime to plateau (s)—Not measured—
Bedrock → BedrockMemory growth (MiB/min)46.31Valid—
Bedrock → BedrockMemory load duration (s)300Valid—
Bedrock → BedrockMemory shape under load0Valid—
Bedrock → BedrockIdle memory shape—Not measured—
Bedrock → BedrockIdle memory static1Valid—
Bedrock → BedrockIdle memory growth (MiB/min)0.326Valid—
Bedrock → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Bedrock → BedrockThroughput (req/s)3,922concurrency 8; n=23,5395 ms
Bedrock → BedrockThroughput (req/s)3,922concurrency 8; n=23,53910 ms
Bedrock → BedrockThroughput (req/s)4,006concurrency 64; n=24,04250 ms
Bedrock → BedrockThroughput (req/s)4,006concurrency 64; n=24,042100 ms
Bedrock → BedrockThroughput (req/s)4,006concurrency 64; n=24,042Unbounded

Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

Ingress → upstreamCell stateProbe HTTP statusRecorded reasonRecorded detailEvidence scopeResponse excerptTested build
Cohere → OpenAINot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Bedrock → OpenAINot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Anthropic → OpenAINot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI → OpenAIServed———Recorded run evidence—kong:3.9.4
Gemini → OpenAINot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI Responses → OpenAINot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Anthropic → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Cohere → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI → AnthropicServed———Recorded run evidence—kong:3.9.4
OpenAI Responses → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Gemini → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Bedrock → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI → CohereServed———Recorded run evidence—kong:3.9.4
Cohere → CohereNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI Responses → CohereNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Gemini → CohereNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Bedrock → CohereNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Anthropic → CohereNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Cohere → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Anthropic → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Gemini → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Bedrock → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI Responses → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI → GeminiServed———Recorded run evidence—kong:3.9.4
OpenAI Responses → GeminiNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Cohere → GeminiNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Anthropic → GeminiNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Gemini → GeminiServed———Recorded run evidence—kong:3.9.4
Bedrock → GeminiNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Anthropic → BedrockNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI Responses → BedrockNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Cohere → BedrockNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
OpenAI → BedrockServed———Recorded run evidence—kong:3.9.4
Gemini → BedrockNot declared—recorded note 1recorded note 1recorded note 2—kong:3.9.4
Bedrock → BedrockServed———Recorded run evidence—kong:3.9.4
Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2
Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8
Load generator SHA-256
not recorded
net.core.somaxconn
65,535
net.ipv4.tcp_max_syn_backlog
65,535

Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

Ingress → upstreamConcurrencyLanes by HTTP statusLanes with no response
OpenAI → OpenAI1HTTP 200: 10
OpenAI → OpenAI2HTTP 200: 20
OpenAI → OpenAI4HTTP 200: 40
OpenAI → OpenAI8HTTP 200: 80
OpenAI → OpenAI16HTTP 200: 160
OpenAI → OpenAI32HTTP 200: 320
OpenAI → OpenAI64HTTP 200: 640
OpenAI → OpenAI128HTTP 200: 1280
OpenAI → OpenAI96HTTP 200: 960
OpenAI → OpenAI80HTTP 200: 800
OpenAI → OpenAI72HTTP 200: 720
OpenAI → OpenAI76HTTP 200: 760
OpenAI → OpenAI78HTTP 200: 780
OpenAI → OpenAI79HTTP 200: 790
OpenAI → OpenAI78HTTP 200: 780
OpenAI → OpenAI78HTTP 200: 780
OpenAI → Anthropic1HTTP 200: 10
OpenAI → Anthropic2HTTP 200: 20
OpenAI → Anthropic4HTTP 200: 40
OpenAI → Anthropic8HTTP 200: 80
OpenAI → Anthropic16HTTP 200: 160
OpenAI → Anthropic32HTTP 200: 320
OpenAI → Anthropic64HTTP 200: 640
OpenAI → Anthropic128HTTP 200: 1280
OpenAI → Anthropic256HTTP 200: 2560
OpenAI → Anthropic512HTTP 200: 5120
OpenAI → Anthropic384HTTP 200: 3840
OpenAI → Anthropic320HTTP 200: 3200
OpenAI → Anthropic288HTTP 200: 2880
OpenAI → Anthropic272HTTP 200: 2720
OpenAI → Anthropic280HTTP 200: 2800
OpenAI → Anthropic276HTTP 200: 2760
OpenAI → Anthropic274HTTP 200: 2740
OpenAI → Anthropic273HTTP 200: 2730
OpenAI → Anthropic272HTTP 200: 2720
OpenAI → Anthropic272HTTP 200: 2720
OpenAI → Anthropic264HTTP 200: 2640
OpenAI → Anthropic264HTTP 200: 2640
OpenAI → Anthropic264HTTP 200: 2640

Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

  1. Pinned Kong 3.9.4 ai-proxy schema supports OpenAI ingress plus native Gemini and Bedrock llm_format, with native formats restricted to their matching provider. OpenAI ingress is configured for five providers including Cohere v1. Seven exact pairs passed local response and upstream-recorder preflight. OpenAI Responses and native Anthropic/Cohere ingress are not exposed by this ai-proxy schema. Evidence: results/diagnostics/refresh-configuration/kong-protocol-preflight.json
  2. Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

Evidence for the current measurements

Configuration recorded by the run

_format_version: "3.0"
# ONE static config, five upstream providers and two native ingress routes. The client picks the upstream with a request header on
# the SAME uniform OpenAI path: \`x-llm-provider: anthropic|gemini|bedrock\`; no header = openai.
# Route priority puts every header-matched route above the header-less openai fallback
# (kong/router/transform.lua get_priority: match_weight occupies the top 3 bits).
services:
  - name: llm
    # Placeholder: ai-proxy overrides host/port/scheme/path per plugin instance via
    # kong.service.set_target() + set_path() from model.options.upstream_url.
    url: http://127.0.0.1:1
    routes:
      - name: chat-anthropic
        paths: ["/v1/chat/completions"]
        headers:
          x-llm-provider: ["anthropic"]
        strip_path: false
        plugins:
          - name: ai-proxy
            config:
              route_type: llm/v1/chat
              auth:
                header_name: Authorization
                header_value: "Bearer dummy"
              model:
                provider: anthropic
                name: gpt-4o-mini
                options:
                  anthropic_version: "2023-06-01"
                  upstream_url: "http://127.0.0.1:8000/v1/messages"
      - name: chat-gemini
        paths: ["/v1/chat/completions"]
        headers:
          x-llm-provider: ["gemini"]
        strip_path: false
        plugins:
          - name: ai-proxy
            config:
              route_type: llm/v1/chat
              auth:
                header_name: Authorization
                header_value: "Bearer dummy"
              model:
                provider: gemini
                name: gpt-4o-mini
                options:
                  upstream_url: "http://127.0.0.1:8000/v1beta/models/gpt-4o-mini:generateContent"
      - name: chat-bedrock
        paths: ["/v1/chat/completions"]
        headers:
          x-llm-provider: ["bedrock"]
        strip_path: false
        plugins:
          - name: ai-proxy
            config:
              route_type: llm/v1/chat
              auth:
                aws_access_key_id: "AKIAMOCKACCESSKEY"
                aws_secret_access_key: "mock-secret-access-key"
              model:
                provider: bedrock
                name: gpt-4o-mini
                options:
                  bedrock:
                    aws_region: "us-east-1"
                  upstream_url: "http://127.0.0.1:8000/model/gpt-4o-mini/converse"
      - name: chat-cohere
        paths: ["/v1/chat/completions"]
        headers:
          x-llm-provider: ["cohere"]
        strip_path: false
        plugins:
          - name: ai-proxy
            config:
              route_type: llm/v1/chat
              auth:
                header_name: Authorization
                header_value: "Bearer dummy"
              model:
                provider: cohere
                name: gpt-4o-mini
                options:
                  upstream_url: "http://127.0.0.1:8000/v1/chat"
      - name: native-gemini
        paths: ["/v1beta/models/gpt-4o-mini:generateContent"]
        strip_path: false
        plugins:
          - name: ai-proxy
            config:
              route_type: llm/v1/chat
              llm_format: gemini
              auth:
                header_name: Authorization
                header_value: "Bearer dummy"
              model:
                provider: gemini
                name: gpt-4o-mini
                options:
                  upstream_url: "http://127.0.0.1:8000/v1beta/models/gpt-4o-mini:generateContent"
      - name: native-bedrock
        paths: ["/model/gpt-4o-mini/converse"]
        strip_path: false
        plugins:
          - name: ai-proxy
            config:
              route_type: llm/v1/chat
              llm_format: bedrock
              auth:
                aws_access_key_id: "AKIAMOCKACCESSKEY"
                aws_secret_access_key: "mock-secret-access-key"
              model:
                provider: bedrock
                name: gpt-4o-mini
                options:
                  bedrock:
                    aws_region: "us-east-1"
                  upstream_url: "http://127.0.0.1:8000/model/gpt-4o-mini/converse"
      # Header-less fallback: the OOTB OpenAI client experience (lowest route priority).
      - name: chat
        paths: ["/v1/chat/completions"]
        strip_path: false
        plugins:
          - name: ai-proxy
            config:
              route_type: llm/v1/chat
              auth:
                header_name: Authorization
                header_value: "Bearer dummy"
              model:
                provider: openai
                name: gpt-4o-mini
                options:
                  upstream_url: "http://127.0.0.1:8000/v1/chat/completions"

Compare a shared protocol cell

Historical measurements