Kong benchmark measurements
Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.
Other · recorded build: kong:3.9.4. Measured: .
JSON · Markdown · CSV · Project source · Interactive dashboard
Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.
Published measurements
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| Cohere → OpenAI | — | — | Not declared: recorded note 1 | — |
| Bedrock → OpenAI | — | — | Not declared: recorded note 1 | — |
| Anthropic → OpenAI | — | — | Not declared: recorded note 1 | — |
| OpenAI → OpenAI | Added latency p50 (µs) | 213 | Valid · n=23,861 gateway / 246,038 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 410 | Valid · n=23,861 gateway / 246,038 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 440 | Valid · n=23,861 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 30 | Valid · n=246,038 | — |
| OpenAI → OpenAI | CPU per request (µs) | 220.7 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 4,531.3 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.787 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 85,868 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 14,309 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 5 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 0.000734 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | 106,009 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | 106,433 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | 168,648 | Valid · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 3,731.9 | Valid | — |
| OpenAI → OpenAI | Streams sustained | 78 | Valid | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 406.2 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 617.4 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 617.5 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 617.7 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 618.6 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | 66 | Valid | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | 0.105 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 300 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | — | Not measured | — |
| OpenAI → OpenAI | Idle memory shape | — | Not measured | — |
| OpenAI → OpenAI | Idle memory static | 1 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | 0.241 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | 13,315 | concurrency 4; n=79,905 | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | 19,190 | concurrency 8; n=115,157 | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | 21,287 | concurrency 64; n=127,725 | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | 22,171 | concurrency 128; n=133,046 | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | 22,171 | concurrency 128; n=133,046 | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 22,171 | concurrency 128; n=133,046 | Unbounded |
| Gemini → OpenAI | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → OpenAI | — | — | Not declared: recorded note 1 | — |
| Anthropic → Anthropic | — | — | Not declared: recorded note 1 | — |
| Cohere → Anthropic | — | — | Not declared: recorded note 1 | — |
| OpenAI → Anthropic | Added latency p50 (µs) | 211 | Valid · n=24,037 gateway / 241,342 direct | — |
| OpenAI → Anthropic | Added latency p99 (µs) | 420 | Valid · n=24,037 gateway / 241,342 direct | — |
| OpenAI → Anthropic | Gateway p99 at concurrency 1 (µs) | 450 | Valid · n=24,037 | — |
| OpenAI → Anthropic | Direct-to-mock p99 at concurrency 1 (µs) | 30 | Valid · n=241,342 | — |
| OpenAI → Anthropic | CPU per request (µs) | 221.7 | Valid | — |
| OpenAI → Anthropic | Requests per CPU-second | 4,509.8 | Valid | — |
| OpenAI → Anthropic | Cost window concurrency | 8 | Valid | — |
| OpenAI → Anthropic | Cost window core utilisation | 0.972 | Valid | — |
| OpenAI → Anthropic | Cost window successful requests | 105,485 | Valid | — |
| OpenAI → Anthropic | Cost window throughput (req/s) | 17,578 | Valid | — |
| OpenAI → Anthropic | Cost window threads | 5 | Valid | — |
| OpenAI → Anthropic | Non-voluntary context switches per request | 0.002 | Valid | — |
| OpenAI → Anthropic | Cost window major page faults | 0 | Valid | — |
| OpenAI → Anthropic | Added time to first event p50 (µs) | 84,892 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → Anthropic | Added time to first event p99 (µs) | 85,699 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → Anthropic | Added inter-frame gap p50 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → Anthropic | Added inter-frame gap p99 (µs) | 147,628 | Valid · n=63 gateway / 63 direct | — |
| OpenAI → Anthropic | Streams sustained (frames/s) | 12,509.3 | Valid | — |
| OpenAI → Anthropic | Streams sustained | 264 | Valid | — |
| OpenAI → Anthropic | Idle RSS (MiB) | 406.1 | Valid | — |
| OpenAI → Anthropic | Steady-state RSS (MiB) | 614.6 | Valid | — |
| OpenAI → Anthropic | Recovered RSS (MiB) | 614.7 | Valid | — |
| OpenAI → Anthropic | Peak RSS (MiB) | 614.9 | Valid | — |
| OpenAI → Anthropic | Peak RSS high-water mark (MiB) | 616 | Valid | — |
| OpenAI → Anthropic | Time to plateau (s) | 78 | Valid | — |
| OpenAI → Anthropic | Memory growth (MiB/min) | 0.111 | Valid | — |
| OpenAI → Anthropic | Memory load duration (s) | 300 | Valid | — |
| OpenAI → Anthropic | Memory shape under load | — | Not measured | — |
| OpenAI → Anthropic | Idle memory shape | — | Not measured | — |
| OpenAI → Anthropic | Idle memory static | 1 | Valid | — |
| OpenAI → Anthropic | Idle memory growth (MiB/min) | 0.329 | Valid | — |
| OpenAI → Anthropic | Throughput (req/s) | 9,859 | concurrency 4; n=59,165 | 1 ms |
| OpenAI → Anthropic | Throughput (req/s) | 16,975 | concurrency 8; n=101,865 | 5 ms |
| OpenAI → Anthropic | Throughput (req/s) | 18,516 | concurrency 64; n=111,100 | 10 ms |
| OpenAI → Anthropic | Throughput (req/s) | 19,147 | concurrency 128; n=114,896 | 50 ms |
| OpenAI → Anthropic | Throughput (req/s) | 19,147 | concurrency 128; n=114,896 | 100 ms |
| OpenAI → Anthropic | Throughput (req/s) | 19,147 | concurrency 128; n=114,896 | Unbounded |
| OpenAI Responses → Anthropic | — | — | Not declared: recorded note 1 | — |
| Gemini → Anthropic | — | — | Not declared: recorded note 1 | — |
| Bedrock → Anthropic | — | — | Not declared: recorded note 1 | — |
| OpenAI → Cohere | Added latency p50 (µs) | 207 | Valid · n=24,758 gateway / 238,237 direct | — |
| OpenAI → Cohere | Added latency p99 (µs) | 423 | Valid · n=24,758 gateway / 238,237 direct | — |
| OpenAI → Cohere | Gateway p99 at concurrency 1 (µs) | 454 | Valid · n=24,758 | — |
| OpenAI → Cohere | Direct-to-mock p99 at concurrency 1 (µs) | 31 | Valid · n=238,237 | — |
| OpenAI → Cohere | CPU per request (µs) | 216.6 | Valid | — |
| OpenAI → Cohere | Requests per CPU-second | 4,616.9 | Valid | — |
| OpenAI → Cohere | Cost window concurrency | 8 | Valid | — |
| OpenAI → Cohere | Cost window core utilisation | 0.952 | Valid | — |
| OpenAI → Cohere | Cost window successful requests | 105,820 | Valid | — |
| OpenAI → Cohere | Cost window throughput (req/s) | 17,634 | Valid | — |
| OpenAI → Cohere | Cost window threads | 5 | Valid | — |
| OpenAI → Cohere | Non-voluntary context switches per request | 0.001 | Valid | — |
| OpenAI → Cohere | Cost window major page faults | 0 | Valid | — |
| OpenAI → Cohere | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI → Cohere | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI → Cohere | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI → Cohere | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI → Cohere | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI → Cohere | Streams sustained | — | Untestable | — |
| OpenAI → Cohere | Idle RSS (MiB) | 408.1 | Valid | — |
| OpenAI → Cohere | Steady-state RSS (MiB) | 620.3 | Valid | — |
| OpenAI → Cohere | Recovered RSS (MiB) | 620.7 | Valid | — |
| OpenAI → Cohere | Peak RSS (MiB) | 620.8 | Valid | — |
| OpenAI → Cohere | Peak RSS high-water mark (MiB) | 622.1 | Valid | — |
| OpenAI → Cohere | Time to plateau (s) | 66 | Valid | — |
| OpenAI → Cohere | Memory growth (MiB/min) | 0.409 | Valid | — |
| OpenAI → Cohere | Memory load duration (s) | 300 | Valid | — |
| OpenAI → Cohere | Memory shape under load | — | Not measured | — |
| OpenAI → Cohere | Idle memory shape | — | Not measured | — |
| OpenAI → Cohere | Idle memory static | 1 | Valid | — |
| OpenAI → Cohere | Idle memory growth (MiB/min) | 0.256 | Valid | — |
| OpenAI → Cohere | Throughput (req/s) | 12,570 | concurrency 4; n=75,430 | 1 ms |
| OpenAI → Cohere | Throughput (req/s) | 18,172 | concurrency 16; n=109,045 | 5 ms |
| OpenAI → Cohere | Throughput (req/s) | 18,962 | concurrency 32; n=113,777 | 10 ms |
| OpenAI → Cohere | Throughput (req/s) | 20,178 | concurrency 128; n=121,088 | 50 ms |
| OpenAI → Cohere | Throughput (req/s) | 20,178 | concurrency 128; n=121,088 | 100 ms |
| OpenAI → Cohere | Throughput (req/s) | 20,178 | concurrency 128; n=121,088 | Unbounded |
| Cohere → Cohere | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Cohere | — | — | Not declared: recorded note 1 | — |
| Gemini → Cohere | — | — | Not declared: recorded note 1 | — |
| Bedrock → Cohere | — | — | Not declared: recorded note 1 | — |
| Anthropic → Cohere | — | — | Not declared: recorded note 1 | — |
| Cohere → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| OpenAI → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Anthropic → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Gemini → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Bedrock → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| OpenAI → Gemini | Added latency p50 (µs) | 214 | Valid · n=23,733 gateway / 242,637 direct | — |
| OpenAI → Gemini | Added latency p99 (µs) | 438 | Valid · n=23,733 gateway / 242,637 direct | — |
| OpenAI → Gemini | Gateway p99 at concurrency 1 (µs) | 468 | Valid · n=23,733 | — |
| OpenAI → Gemini | Direct-to-mock p99 at concurrency 1 (µs) | 30 | Valid · n=242,637 | — |
| OpenAI → Gemini | CPU per request (µs) | 214.8 | Valid | — |
| OpenAI → Gemini | Requests per CPU-second | 4,656 | Valid | — |
| OpenAI → Gemini | Cost window concurrency | 8 | Valid | — |
| OpenAI → Gemini | Cost window core utilisation | 0.971 | Valid | — |
| OpenAI → Gemini | Cost window successful requests | 108,858 | Valid | — |
| OpenAI → Gemini | Cost window throughput (req/s) | 18,140 | Valid | — |
| OpenAI → Gemini | Cost window threads | 5 | Valid | — |
| OpenAI → Gemini | Non-voluntary context switches per request | 0.001 | Valid | — |
| OpenAI → Gemini | Cost window major page faults | 0 | Valid | — |
| OpenAI → Gemini | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI → Gemini | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI → Gemini | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI → Gemini | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI → Gemini | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI → Gemini | Streams sustained | — | Untestable | — |
| OpenAI → Gemini | Idle RSS (MiB) | 408.6 | Valid | — |
| OpenAI → Gemini | Steady-state RSS (MiB) | 622.8 | Valid | — |
| OpenAI → Gemini | Recovered RSS (MiB) | 623 | Valid | — |
| OpenAI → Gemini | Peak RSS (MiB) | 623.1 | Valid | — |
| OpenAI → Gemini | Peak RSS high-water mark (MiB) | 624.2 | Valid | — |
| OpenAI → Gemini | Time to plateau (s) | 66 | Valid | — |
| OpenAI → Gemini | Memory growth (MiB/min) | 0.261 | Valid | — |
| OpenAI → Gemini | Memory load duration (s) | 300 | Valid | — |
| OpenAI → Gemini | Memory shape under load | — | Not measured | — |
| OpenAI → Gemini | Idle memory shape | — | Not measured | — |
| OpenAI → Gemini | Idle memory static | 1 | Valid | — |
| OpenAI → Gemini | Idle memory growth (MiB/min) | 0.387 | Valid | — |
| OpenAI → Gemini | Throughput (req/s) | 18,455 | concurrency 8; n=110,748 | 1 ms |
| OpenAI → Gemini | Throughput (req/s) | 18,455 | concurrency 8; n=110,748 | 5 ms |
| OpenAI → Gemini | Throughput (req/s) | 19,059 | concurrency 32; n=114,368 | 10 ms |
| OpenAI → Gemini | Throughput (req/s) | 19,713 | concurrency 128; n=118,296 | 50 ms |
| OpenAI → Gemini | Throughput (req/s) | 19,713 | concurrency 128; n=118,296 | 100 ms |
| OpenAI → Gemini | Throughput (req/s) | 19,713 | concurrency 128; n=118,296 | Unbounded |
| OpenAI Responses → Gemini | — | — | Not declared: recorded note 1 | — |
| Cohere → Gemini | — | — | Not declared: recorded note 1 | — |
| Anthropic → Gemini | — | — | Not declared: recorded note 1 | — |
| Gemini → Gemini | Added latency p50 (µs) | 193 | Valid · n=26,272 gateway / 244,214 direct | — |
| Gemini → Gemini | Added latency p99 (µs) | 402 | Valid · n=26,272 gateway / 244,214 direct | — |
| Gemini → Gemini | Gateway p99 at concurrency 1 (µs) | 432 | Valid · n=26,272 | — |
| Gemini → Gemini | Direct-to-mock p99 at concurrency 1 (µs) | 30 | Valid · n=244,214 | — |
| Gemini → Gemini | CPU per request (µs) | 195.5 | Valid | — |
| Gemini → Gemini | Requests per CPU-second | 5,115.7 | Valid | — |
| Gemini → Gemini | Cost window concurrency | 8 | Valid | — |
| Gemini → Gemini | Cost window core utilisation | 0.984 | Valid | — |
| Gemini → Gemini | Cost window successful requests | 121,242 | Valid | — |
| Gemini → Gemini | Cost window throughput (req/s) | 20,204 | Valid | — |
| Gemini → Gemini | Cost window threads | 5 | Valid | — |
| Gemini → Gemini | Non-voluntary context switches per request | 0.000759 | Valid | — |
| Gemini → Gemini | Cost window major page faults | 0 | Valid | — |
| Gemini → Gemini | Added time to first event p50 (µs) | — | Untestable | — |
| Gemini → Gemini | Added time to first event p99 (µs) | — | Untestable | — |
| Gemini → Gemini | Added inter-frame gap p50 (µs) | — | Untestable | — |
| Gemini → Gemini | Added inter-frame gap p99 (µs) | — | Untestable | — |
| Gemini → Gemini | Streams sustained (frames/s) | — | Untestable | — |
| Gemini → Gemini | Streams sustained | — | Untestable | — |
| Gemini → Gemini | Idle RSS (MiB) | 405.8 | Valid | — |
| Gemini → Gemini | Steady-state RSS (MiB) | 619.2 | Valid | — |
| Gemini → Gemini | Recovered RSS (MiB) | 619.6 | Valid | — |
| Gemini → Gemini | Peak RSS (MiB) | 619.8 | Valid | — |
| Gemini → Gemini | Peak RSS high-water mark (MiB) | 621 | Valid | — |
| Gemini → Gemini | Time to plateau (s) | 66 | Valid | — |
| Gemini → Gemini | Memory growth (MiB/min) | 0.356 | Valid | — |
| Gemini → Gemini | Memory load duration (s) | 300 | Valid | — |
| Gemini → Gemini | Memory shape under load | — | Not measured | — |
| Gemini → Gemini | Idle memory shape | — | Not measured | — |
| Gemini → Gemini | Idle memory static | 1 | Valid | — |
| Gemini → Gemini | Idle memory growth (MiB/min) | 0.467 | Valid | — |
| Gemini → Gemini | Throughput (req/s) | 13,349 | concurrency 4; n=80,104 | 1 ms |
| Gemini → Gemini | Throughput (req/s) | 20,568 | concurrency 32; n=123,412 | 5 ms |
| Gemini → Gemini | Throughput (req/s) | 21,659 | concurrency 64; n=129,964 | 10 ms |
| Gemini → Gemini | Throughput (req/s) | 21,659 | concurrency 64; n=129,964 | 50 ms |
| Gemini → Gemini | Throughput (req/s) | 21,659 | concurrency 64; n=129,964 | 100 ms |
| Gemini → Gemini | Throughput (req/s) | 21,659 | concurrency 64; n=129,964 | Unbounded |
| Bedrock → Gemini | — | — | Not declared: recorded note 1 | — |
| Anthropic → Bedrock | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Bedrock | — | — | Not declared: recorded note 1 | — |
| Cohere → Bedrock | — | — | Not declared: recorded note 1 | — |
| OpenAI → Bedrock | Added latency p50 (µs) | 1,038 | Valid · n=5,167 gateway / 237,310 direct | — |
| OpenAI → Bedrock | Added latency p99 (µs) | 2,133 | Valid · n=5,167 gateway / 237,310 direct | — |
| OpenAI → Bedrock | Gateway p99 at concurrency 1 (µs) | 2,164 | Valid · n=5,167 | — |
| OpenAI → Bedrock | Direct-to-mock p99 at concurrency 1 (µs) | 31 | Valid · n=237,310 | — |
| OpenAI → Bedrock | CPU per request (µs) | 1,041.3 | Valid | — |
| OpenAI → Bedrock | Requests per CPU-second | 960.3 | Valid | — |
| OpenAI → Bedrock | Cost window concurrency | 8 | Valid | — |
| OpenAI → Bedrock | Cost window core utilisation | 0.985 | Valid | — |
| OpenAI → Bedrock | Cost window successful requests | 22,779 | Valid | — |
| OpenAI → Bedrock | Cost window throughput (req/s) | 3,796 | Valid | — |
| OpenAI → Bedrock | Cost window threads | 5 | Valid | — |
| OpenAI → Bedrock | Non-voluntary context switches per request | 0.004 | Valid | — |
| OpenAI → Bedrock | Cost window major page faults | 0 | Valid | — |
| OpenAI → Bedrock | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI → Bedrock | Streams sustained | — | Untestable | — |
| OpenAI → Bedrock | Idle RSS (MiB) | 407.8 | Valid | — |
| OpenAI → Bedrock | Steady-state RSS (MiB) | 3,411.8 | Valid | — |
| OpenAI → Bedrock | Recovered RSS (MiB) | 3,448.5 | Valid | — |
| OpenAI → Bedrock | Peak RSS (MiB) | 3,466.1 | Valid | — |
| OpenAI → Bedrock | Peak RSS high-water mark (MiB) | 3,477 | Valid | — |
| OpenAI → Bedrock | Time to plateau (s) | — | Not measured | — |
| OpenAI → Bedrock | Memory growth (MiB/min) | 10.9 | Valid | — |
| OpenAI → Bedrock | Memory load duration (s) | 300 | Valid | — |
| OpenAI → Bedrock | Memory shape under load | 0 | Valid | — |
| OpenAI → Bedrock | Idle memory shape | — | Not measured | — |
| OpenAI → Bedrock | Idle memory static | 1 | Valid | — |
| OpenAI → Bedrock | Idle memory growth (MiB/min) | 0.341 | Valid | — |
| OpenAI → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI → Bedrock | Throughput (req/s) | 3,961 | concurrency 8; n=23,772 | 5 ms |
| OpenAI → Bedrock | Throughput (req/s) | 3,961 | concurrency 8; n=23,772 | 10 ms |
| OpenAI → Bedrock | Throughput (req/s) | 3,961 | concurrency 8; n=23,772 | 50 ms |
| OpenAI → Bedrock | Throughput (req/s) | 3,961 | concurrency 8; n=23,772 | 100 ms |
| OpenAI → Bedrock | Throughput (req/s) | 3,961 | concurrency 8; n=23,772 | Unbounded |
| Gemini → Bedrock | — | — | Not declared: recorded note 1 | — |
| Bedrock → Bedrock | Added latency p50 (µs) | 957 | Valid · n=5,584 gateway / 241,921 direct | — |
| Bedrock → Bedrock | Added latency p99 (µs) | 1,988 | Valid · n=5,584 gateway / 241,921 direct | — |
| Bedrock → Bedrock | Gateway p99 at concurrency 1 (µs) | 2,018 | Valid · n=5,584 | — |
| Bedrock → Bedrock | Direct-to-mock p99 at concurrency 1 (µs) | 30 | Valid · n=241,921 | — |
| Bedrock → Bedrock | CPU per request (µs) | 1,094.7 | Valid | — |
| Bedrock → Bedrock | Requests per CPU-second | 913.5 | Valid | — |
| Bedrock → Bedrock | Cost window concurrency | 8 | Valid | — |
| Bedrock → Bedrock | Cost window core utilisation | 0.989 | Valid | — |
| Bedrock → Bedrock | Cost window successful requests | 21,741 | Valid | — |
| Bedrock → Bedrock | Cost window throughput (req/s) | 3,623 | Valid | — |
| Bedrock → Bedrock | Cost window threads | 5 | Valid | — |
| Bedrock → Bedrock | Non-voluntary context switches per request | 0.007 | Valid | — |
| Bedrock → Bedrock | Cost window major page faults | 0 | Valid | — |
| Bedrock → Bedrock | Added time to first event p50 (µs) | — | Untestable | — |
| Bedrock → Bedrock | Added time to first event p99 (µs) | — | Untestable | — |
| Bedrock → Bedrock | Added inter-frame gap p50 (µs) | — | Untestable | — |
| Bedrock → Bedrock | Added inter-frame gap p99 (µs) | — | Untestable | — |
| Bedrock → Bedrock | Streams sustained (frames/s) | — | Untestable | — |
| Bedrock → Bedrock | Streams sustained | — | Untestable | — |
| Bedrock → Bedrock | Idle RSS (MiB) | 407.7 | Valid | — |
| Bedrock → Bedrock | Steady-state RSS (MiB) | 3,782.2 | Valid | — |
| Bedrock → Bedrock | Recovered RSS (MiB) | 3,821.5 | Valid | — |
| Bedrock → Bedrock | Peak RSS (MiB) | 3,840.8 | Valid | — |
| Bedrock → Bedrock | Peak RSS high-water mark (MiB) | 3,862.1 | Valid | — |
| Bedrock → Bedrock | Time to plateau (s) | — | Not measured | — |
| Bedrock → Bedrock | Memory growth (MiB/min) | 46.31 | Valid | — |
| Bedrock → Bedrock | Memory load duration (s) | 300 | Valid | — |
| Bedrock → Bedrock | Memory shape under load | 0 | Valid | — |
| Bedrock → Bedrock | Idle memory shape | — | Not measured | — |
| Bedrock → Bedrock | Idle memory static | 1 | Valid | — |
| Bedrock → Bedrock | Idle memory growth (MiB/min) | 0.326 | Valid | — |
| Bedrock → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| Bedrock → Bedrock | Throughput (req/s) | 3,922 | concurrency 8; n=23,539 | 5 ms |
| Bedrock → Bedrock | Throughput (req/s) | 3,922 | concurrency 8; n=23,539 | 10 ms |
| Bedrock → Bedrock | Throughput (req/s) | 4,006 | concurrency 64; n=24,042 | 50 ms |
| Bedrock → Bedrock | Throughput (req/s) | 4,006 | concurrency 64; n=24,042 | 100 ms |
| Bedrock → Bedrock | Throughput (req/s) | 4,006 | concurrency 64; n=24,042 | Unbounded |
Probe evidence and instrument provenance
Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.
| Ingress → upstream | Cell state | Probe HTTP status | Recorded reason | Recorded detail | Evidence scope | Response excerpt | Tested build |
|---|---|---|---|---|---|---|---|
| Cohere → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Bedrock → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Anthropic → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI → OpenAI | Served | — | — | — | Recorded run evidence | — | kong:3.9.4 |
| Gemini → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI Responses → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Anthropic → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Cohere → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI → Anthropic | Served | — | — | — | Recorded run evidence | — | kong:3.9.4 |
| OpenAI Responses → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Gemini → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Bedrock → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI → Cohere | Served | — | — | — | Recorded run evidence | — | kong:3.9.4 |
| Cohere → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI Responses → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Gemini → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Bedrock → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Anthropic → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Cohere → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Anthropic → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Gemini → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Bedrock → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI Responses → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI → Gemini | Served | — | — | — | Recorded run evidence | — | kong:3.9.4 |
| OpenAI Responses → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Cohere → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Anthropic → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Gemini → Gemini | Served | — | — | — | Recorded run evidence | — | kong:3.9.4 |
| Bedrock → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Anthropic → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI Responses → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Cohere → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| OpenAI → Bedrock | Served | — | — | — | Recorded run evidence | — | kong:3.9.4 |
| Gemini → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | kong:3.9.4 |
| Bedrock → Bedrock | Served | — | — | — | Recorded run evidence | — | kong:3.9.4 |
- Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2- Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8- Load generator SHA-256
not recorded- net.core.somaxconn
- 65,535
- net.ipv4.tcp_max_syn_backlog
- 65,535
Stream sweep diagnostics
Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.
| Ingress → upstream | Concurrency | Lanes by HTTP status | Lanes with no response |
|---|---|---|---|
| OpenAI → OpenAI | 1 | HTTP 200: 1 | 0 |
| OpenAI → OpenAI | 2 | HTTP 200: 2 | 0 |
| OpenAI → OpenAI | 4 | HTTP 200: 4 | 0 |
| OpenAI → OpenAI | 8 | HTTP 200: 8 | 0 |
| OpenAI → OpenAI | 16 | HTTP 200: 16 | 0 |
| OpenAI → OpenAI | 32 | HTTP 200: 32 | 0 |
| OpenAI → OpenAI | 64 | HTTP 200: 64 | 0 |
| OpenAI → OpenAI | 128 | HTTP 200: 128 | 0 |
| OpenAI → OpenAI | 96 | HTTP 200: 96 | 0 |
| OpenAI → OpenAI | 80 | HTTP 200: 80 | 0 |
| OpenAI → OpenAI | 72 | HTTP 200: 72 | 0 |
| OpenAI → OpenAI | 76 | HTTP 200: 76 | 0 |
| OpenAI → OpenAI | 78 | HTTP 200: 78 | 0 |
| OpenAI → OpenAI | 79 | HTTP 200: 79 | 0 |
| OpenAI → OpenAI | 78 | HTTP 200: 78 | 0 |
| OpenAI → OpenAI | 78 | HTTP 200: 78 | 0 |
| OpenAI → Anthropic | 1 | HTTP 200: 1 | 0 |
| OpenAI → Anthropic | 2 | HTTP 200: 2 | 0 |
| OpenAI → Anthropic | 4 | HTTP 200: 4 | 0 |
| OpenAI → Anthropic | 8 | HTTP 200: 8 | 0 |
| OpenAI → Anthropic | 16 | HTTP 200: 16 | 0 |
| OpenAI → Anthropic | 32 | HTTP 200: 32 | 0 |
| OpenAI → Anthropic | 64 | HTTP 200: 64 | 0 |
| OpenAI → Anthropic | 128 | HTTP 200: 128 | 0 |
| OpenAI → Anthropic | 256 | HTTP 200: 256 | 0 |
| OpenAI → Anthropic | 512 | HTTP 200: 512 | 0 |
| OpenAI → Anthropic | 384 | HTTP 200: 384 | 0 |
| OpenAI → Anthropic | 320 | HTTP 200: 320 | 0 |
| OpenAI → Anthropic | 288 | HTTP 200: 288 | 0 |
| OpenAI → Anthropic | 272 | HTTP 200: 272 | 0 |
| OpenAI → Anthropic | 280 | HTTP 200: 280 | 0 |
| OpenAI → Anthropic | 276 | HTTP 200: 276 | 0 |
| OpenAI → Anthropic | 274 | HTTP 200: 274 | 0 |
| OpenAI → Anthropic | 273 | HTTP 200: 273 | 0 |
| OpenAI → Anthropic | 272 | HTTP 200: 272 | 0 |
| OpenAI → Anthropic | 272 | HTTP 200: 272 | 0 |
| OpenAI → Anthropic | 264 | HTTP 200: 264 | 0 |
| OpenAI → Anthropic | 264 | HTTP 200: 264 | 0 |
| OpenAI → Anthropic | 264 | HTTP 200: 264 | 0 |
Recorded notes
Text recorded with the cells above. Each note is shown once; rows refer to it by number.
- Pinned Kong 3.9.4 ai-proxy schema supports OpenAI ingress plus native Gemini and Bedrock llm_format, with native formats restricted to their matching provider. OpenAI ingress is configured for five providers including Cohere v1. Seven exact pairs passed local response and upstream-recorder preflight. OpenAI Responses and native Anthropic/Cohere ingress are not exposed by this ai-proxy schema. Evidence: results/diagnostics/refresh-configuration/kong-protocol-preflight.json
- Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.
Evidence for the current measurements
- run-kong-d4b6d413ea59fd281dacf70be5988600ace51a15e8665c6c7dedcc87dd2cd2fb · complete raw record including RSS series
Configuration recorded by the run
_format_version: "3.0"
# ONE static config, five upstream providers and two native ingress routes. The client picks the upstream with a request header on
# the SAME uniform OpenAI path: \`x-llm-provider: anthropic|gemini|bedrock\`; no header = openai.
# Route priority puts every header-matched route above the header-less openai fallback
# (kong/router/transform.lua get_priority: match_weight occupies the top 3 bits).
services:
- name: llm
# Placeholder: ai-proxy overrides host/port/scheme/path per plugin instance via
# kong.service.set_target() + set_path() from model.options.upstream_url.
url: http://127.0.0.1:1
routes:
- name: chat-anthropic
paths: ["/v1/chat/completions"]
headers:
x-llm-provider: ["anthropic"]
strip_path: false
plugins:
- name: ai-proxy
config:
route_type: llm/v1/chat
auth:
header_name: Authorization
header_value: "Bearer dummy"
model:
provider: anthropic
name: gpt-4o-mini
options:
anthropic_version: "2023-06-01"
upstream_url: "http://127.0.0.1:8000/v1/messages"
- name: chat-gemini
paths: ["/v1/chat/completions"]
headers:
x-llm-provider: ["gemini"]
strip_path: false
plugins:
- name: ai-proxy
config:
route_type: llm/v1/chat
auth:
header_name: Authorization
header_value: "Bearer dummy"
model:
provider: gemini
name: gpt-4o-mini
options:
upstream_url: "http://127.0.0.1:8000/v1beta/models/gpt-4o-mini:generateContent"
- name: chat-bedrock
paths: ["/v1/chat/completions"]
headers:
x-llm-provider: ["bedrock"]
strip_path: false
plugins:
- name: ai-proxy
config:
route_type: llm/v1/chat
auth:
aws_access_key_id: "AKIAMOCKACCESSKEY"
aws_secret_access_key: "mock-secret-access-key"
model:
provider: bedrock
name: gpt-4o-mini
options:
bedrock:
aws_region: "us-east-1"
upstream_url: "http://127.0.0.1:8000/model/gpt-4o-mini/converse"
- name: chat-cohere
paths: ["/v1/chat/completions"]
headers:
x-llm-provider: ["cohere"]
strip_path: false
plugins:
- name: ai-proxy
config:
route_type: llm/v1/chat
auth:
header_name: Authorization
header_value: "Bearer dummy"
model:
provider: cohere
name: gpt-4o-mini
options:
upstream_url: "http://127.0.0.1:8000/v1/chat"
- name: native-gemini
paths: ["/v1beta/models/gpt-4o-mini:generateContent"]
strip_path: false
plugins:
- name: ai-proxy
config:
route_type: llm/v1/chat
llm_format: gemini
auth:
header_name: Authorization
header_value: "Bearer dummy"
model:
provider: gemini
name: gpt-4o-mini
options:
upstream_url: "http://127.0.0.1:8000/v1beta/models/gpt-4o-mini:generateContent"
- name: native-bedrock
paths: ["/model/gpt-4o-mini/converse"]
strip_path: false
plugins:
- name: ai-proxy
config:
route_type: llm/v1/chat
llm_format: bedrock
auth:
aws_access_key_id: "AKIAMOCKACCESSKEY"
aws_secret_access_key: "mock-secret-access-key"
model:
provider: bedrock
name: gpt-4o-mini
options:
bedrock:
aws_region: "us-east-1"
upstream_url: "http://127.0.0.1:8000/model/gpt-4o-mini/converse"
# Header-less fallback: the OOTB OpenAI client experience (lowest route priority).
- name: chat
paths: ["/v1/chat/completions"]
strip_path: false
plugins:
- name: ai-proxy
config:
route_type: llm/v1/chat
auth:
header_name: Authorization
header_value: "Bearer dummy"
model:
provider: openai
name: gpt-4o-mini
options:
upstream_url: "http://127.0.0.1:8000/v1/chat/completions"
Compare a shared protocol cell
- Kong vs agentgateway
- Kong vs AISIX (api7)
- Kong vs APISIX
- Kong vs Bifrost
- Kong vs Busbar
- Kong vs GoModel
- Kong vs Helicone
- Kong vs Higress
- Kong vs LiteLLM · Python
- Kong vs LiteLLM · Rust
- Kong vs One-API
- Kong vs Plano
- Kong vs Portkey
- Kong vs TensorZero