Plano benchmark measurements
Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.
Rust · recorded build: katanemo/plano:0.4.39. Measured: .
JSON · Markdown · CSV · Project source · Interactive dashboard
Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.
Published measurements
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| OpenAI → Bedrock | Added latency p50 (µs) | 205,998 | Valid · n=29 gateway / 262,765 direct | — |
| OpenAI → Bedrock | Added latency p99 (µs) | 212,984 | Valid · n=29 gateway / 262,765 direct | — |
| OpenAI → Bedrock | Gateway p99 at concurrency 1 (µs) | 213,012 | Valid · n=29 | — |
| OpenAI → Bedrock | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=262,765 | — |
| OpenAI → Bedrock | CPU per request (µs) | 189,176.5 | Valid | — |
| OpenAI → Bedrock | Requests per CPU-second | 5.29 | Valid | — |
| OpenAI → Bedrock | Cost window concurrency | 8 | Valid | — |
| OpenAI → Bedrock | Cost window core utilisation | 0.617 | Valid | — |
| OpenAI → Bedrock | Cost window successful requests | 85 | Valid | — |
| OpenAI → Bedrock | Cost window throughput (req/s) | 14 | Valid | — |
| OpenAI → Bedrock | Cost window threads | 68 | Valid | — |
| OpenAI → Bedrock | Non-voluntary context switches per request | 5.14 | Valid | — |
| OpenAI → Bedrock | Cost window major page faults | 0 | Valid | — |
| OpenAI → Bedrock | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI → Bedrock | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI → Bedrock | Streams sustained | — | Untestable | — |
| OpenAI → Bedrock | Idle RSS (MiB) | 598.8 | Valid | — |
| OpenAI → Bedrock | Steady-state RSS (MiB) | 1,006.3 | Valid | — |
| OpenAI → Bedrock | Recovered RSS (MiB) | 1,006.2 | Valid | — |
| OpenAI → Bedrock | Peak RSS (MiB) | 1,006.7 | Valid | — |
| OpenAI → Bedrock | Peak RSS high-water mark (MiB) | 1,006.5 | Valid | — |
| OpenAI → Bedrock | Time to plateau (s) | 61 | Valid | — |
| OpenAI → Bedrock | Memory growth (MiB/min) | -0.329 | Valid | — |
| OpenAI → Bedrock | Memory load duration (s) | 306 | Valid | — |
| OpenAI → Bedrock | Memory shape under load | — | Not measured | — |
| OpenAI → Bedrock | Idle memory shape | — | Not measured | — |
| OpenAI → Bedrock | Idle memory static | 1 | Valid | — |
| OpenAI → Bedrock | Idle memory growth (MiB/min) | 0.047 | Valid | — |
| OpenAI → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| OpenAI → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| OpenAI → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound | 50 ms |
| OpenAI → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound | 100 ms |
| OpenAI → Bedrock | Throughput (req/s) | 21 | concurrency 8; n=127 | Unbounded |
| Cohere → Bedrock | — | — | Not declared: recorded note 1 | — |
| Gemini → Bedrock | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Bedrock | Added latency p50 (µs) | 200,013 | Valid · n=30 gateway / 265,071 direct | — |
| OpenAI Responses → Bedrock | Added latency p99 (µs) | 205,956 | Valid · n=30 gateway / 265,071 direct | — |
| OpenAI Responses → Bedrock | Gateway p99 at concurrency 1 (µs) | 205,984 | Valid · n=30 | — |
| OpenAI Responses → Bedrock | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=265,071 | — |
| OpenAI Responses → Bedrock | CPU per request (µs) | 182,500 | Valid | — |
| OpenAI Responses → Bedrock | Requests per CPU-second | 5.48 | Valid | — |
| OpenAI Responses → Bedrock | Cost window concurrency | 8 | Valid | — |
| OpenAI Responses → Bedrock | Cost window core utilisation | 0.859 | Valid | — |
| OpenAI Responses → Bedrock | Cost window successful requests | 116 | Valid | — |
| OpenAI Responses → Bedrock | Cost window throughput (req/s) | 19 | Valid | — |
| OpenAI Responses → Bedrock | Cost window threads | 68 | Valid | — |
| OpenAI Responses → Bedrock | Non-voluntary context switches per request | 4.62 | Valid | — |
| OpenAI Responses → Bedrock | Cost window major page faults | 0 | Valid | — |
| OpenAI Responses → Bedrock | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI Responses → Bedrock | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI Responses → Bedrock | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI Responses → Bedrock | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI Responses → Bedrock | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI Responses → Bedrock | Streams sustained | — | Untestable | — |
| OpenAI Responses → Bedrock | Idle RSS (MiB) | 621.6 | Valid | — |
| OpenAI Responses → Bedrock | Steady-state RSS (MiB) | 1,055.5 | Valid | — |
| OpenAI Responses → Bedrock | Recovered RSS (MiB) | 1,055.9 | Valid | — |
| OpenAI Responses → Bedrock | Peak RSS (MiB) | 1,055.9 | Valid | — |
| OpenAI Responses → Bedrock | Peak RSS high-water mark (MiB) | 1,055.9 | Valid | — |
| OpenAI Responses → Bedrock | Time to plateau (s) | 66 | Valid | — |
| OpenAI Responses → Bedrock | Memory growth (MiB/min) | 0.072 | Valid | — |
| OpenAI Responses → Bedrock | Memory load duration (s) | 305 | Valid | — |
| OpenAI Responses → Bedrock | Memory shape under load | — | Not measured | — |
| OpenAI Responses → Bedrock | Idle memory shape | — | Not measured | — |
| OpenAI Responses → Bedrock | Idle memory static | 1 | Valid | — |
| OpenAI Responses → Bedrock | Idle memory growth (MiB/min) | 0.048 | Valid | — |
| OpenAI Responses → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI Responses → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| OpenAI Responses → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| OpenAI Responses → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound | 50 ms |
| OpenAI Responses → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound | 100 ms |
| OpenAI Responses → Bedrock | Throughput (req/s) | 20 | concurrency 16; n=121 | Unbounded |
| Anthropic → Bedrock | Added latency p50 (µs) | 193,978 | Valid · n=31 gateway / 259,772 direct | — |
| Anthropic → Bedrock | Added latency p99 (µs) | 197,949 | Valid · n=31 gateway / 259,772 direct | — |
| Anthropic → Bedrock | Gateway p99 at concurrency 1 (µs) | 197,977 | Valid · n=31 | — |
| Anthropic → Bedrock | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=259,772 | — |
| Anthropic → Bedrock | CPU per request (µs) | 175,360 | Valid | — |
| Anthropic → Bedrock | Requests per CPU-second | 5.7 | Valid | — |
| Anthropic → Bedrock | Cost window concurrency | 8 | Valid | — |
| Anthropic → Bedrock | Cost window core utilisation | 0.86 | Valid | — |
| Anthropic → Bedrock | Cost window successful requests | 125 | Valid | — |
| Anthropic → Bedrock | Cost window throughput (req/s) | 20 | Valid | — |
| Anthropic → Bedrock | Cost window threads | 68 | Valid | — |
| Anthropic → Bedrock | Non-voluntary context switches per request | 5.32 | Valid | — |
| Anthropic → Bedrock | Cost window major page faults | 0 | Valid | — |
| Anthropic → Bedrock | Added time to first event p50 (µs) | — | Untestable | — |
| Anthropic → Bedrock | Added time to first event p99 (µs) | — | Untestable | — |
| Anthropic → Bedrock | Added inter-frame gap p50 (µs) | — | Untestable | — |
| Anthropic → Bedrock | Added inter-frame gap p99 (µs) | — | Untestable | — |
| Anthropic → Bedrock | Streams sustained (frames/s) | — | Untestable | — |
| Anthropic → Bedrock | Streams sustained | — | Untestable | — |
| Anthropic → Bedrock | Idle RSS (MiB) | 612.5 | Valid | — |
| Anthropic → Bedrock | Steady-state RSS (MiB) | 967.7 | Valid | — |
| Anthropic → Bedrock | Recovered RSS (MiB) | 967.4 | Valid | — |
| Anthropic → Bedrock | Peak RSS (MiB) | 968 | Valid | — |
| Anthropic → Bedrock | Peak RSS high-water mark (MiB) | 967.7 | Valid | — |
| Anthropic → Bedrock | Time to plateau (s) | 60 | Valid | — |
| Anthropic → Bedrock | Memory growth (MiB/min) | 0.252 | Valid | — |
| Anthropic → Bedrock | Memory load duration (s) | 302 | Valid | — |
| Anthropic → Bedrock | Memory shape under load | — | Not measured | — |
| Anthropic → Bedrock | Idle memory shape | — | Not measured | — |
| Anthropic → Bedrock | Idle memory static | 1 | Valid | — |
| Anthropic → Bedrock | Idle memory growth (MiB/min) | 0.052 | Valid | — |
| Anthropic → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| Anthropic → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| Anthropic → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| Anthropic → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound | 50 ms |
| Anthropic → Bedrock | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound | 100 ms |
| Anthropic → Bedrock | Throughput (req/s) | 21 | concurrency 16; n=127 | Unbounded |
| Bedrock → Bedrock | — | — | Not declared: recorded note 1 | — |
| OpenAI → Anthropic | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Anthropic | — | — | Not declared: recorded note 1 | — |
| Anthropic → Anthropic | Added latency p50 (µs) | 193,933 | Valid · n=31 gateway / 264,589 direct | — |
| Anthropic → Anthropic | Added latency p99 (µs) | 195,022 | Valid · n=31 gateway / 264,589 direct | — |
| Anthropic → Anthropic | Gateway p99 at concurrency 1 (µs) | 195,049 | Valid · n=31 | — |
| Anthropic → Anthropic | Direct-to-mock p99 at concurrency 1 (µs) | 27 | Valid · n=264,589 | — |
| Anthropic → Anthropic | CPU per request (µs) | 181,353.4 | Valid | — |
| Anthropic → Anthropic | Requests per CPU-second | 5.51 | Valid | — |
| Anthropic → Anthropic | Cost window concurrency | 8 | Valid | — |
| Anthropic → Anthropic | Cost window core utilisation | 0.961 | Valid | — |
| Anthropic → Anthropic | Cost window successful requests | 133 | Valid | — |
| Anthropic → Anthropic | Cost window throughput (req/s) | 22 | Valid | — |
| Anthropic → Anthropic | Cost window threads | 69 | Valid | — |
| Anthropic → Anthropic | Non-voluntary context switches per request | 5.88 | Valid | — |
| Anthropic → Anthropic | Cost window major page faults | 0 | Valid | — |
| Anthropic → Anthropic | Added time to first event p50 (µs) | 153,935 | Valid · n=100 gateway / 100 direct | — |
| Anthropic → Anthropic | Added time to first event p99 (µs) | 165,859 | Valid · n=100 gateway / 100 direct | — |
| Anthropic → Anthropic | Added inter-frame gap p50 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| Anthropic → Anthropic | Added inter-frame gap p99 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| Anthropic → Anthropic | Streams sustained (frames/s) | 920 | Valid | — |
| Anthropic → Anthropic | Streams sustained | 65 | Valid | — |
| Anthropic → Anthropic | Idle RSS (MiB) | 593.1 | Valid | — |
| Anthropic → Anthropic | Steady-state RSS (MiB) | 948.5 | Valid | — |
| Anthropic → Anthropic | Recovered RSS (MiB) | 948.4 | Valid | — |
| Anthropic → Anthropic | Peak RSS (MiB) | 949 | Valid | — |
| Anthropic → Anthropic | Peak RSS high-water mark (MiB) | 948.8 | Valid | — |
| Anthropic → Anthropic | Time to plateau (s) | 67 | Valid | — |
| Anthropic → Anthropic | Memory growth (MiB/min) | -0.301 | Valid | — |
| Anthropic → Anthropic | Memory load duration (s) | 303 | Valid | — |
| Anthropic → Anthropic | Memory shape under load | — | Not measured | — |
| Anthropic → Anthropic | Idle memory shape | — | Not measured | — |
| Anthropic → Anthropic | Idle memory static | 1 | Valid | — |
| Anthropic → Anthropic | Idle memory growth (MiB/min) | 0.052 | Valid | — |
| Anthropic → Anthropic | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| Anthropic → Anthropic | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| Anthropic → Anthropic | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| Anthropic → Anthropic | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound | 50 ms |
| Anthropic → Anthropic | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound | 100 ms |
| Anthropic → Anthropic | Throughput (req/s) | 21 | concurrency 8; n=128 | Unbounded |
| Cohere → Anthropic | — | — | Not declared: recorded note 1 | — |
| Gemini → Anthropic | — | — | Not declared: recorded note 1 | — |
| Bedrock → Anthropic | — | — | Not declared: recorded note 1 | — |
| Anthropic → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Gemini → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| OpenAI → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → OpenAI Responses | Added latency p50 (µs) | 194,984 | Valid · n=30 gateway / 268,348 direct | — |
| OpenAI Responses → OpenAI Responses | Added latency p99 (µs) | 198,964 | Valid · n=30 gateway / 268,348 direct | — |
| OpenAI Responses → OpenAI Responses | Gateway p99 at concurrency 1 (µs) | 198,991 | Valid · n=30 | — |
| OpenAI Responses → OpenAI Responses | Direct-to-mock p99 at concurrency 1 (µs) | 27 | Valid · n=268,348 | — |
| OpenAI Responses → OpenAI Responses | CPU per request (µs) | 177,760 | Valid | — |
| OpenAI Responses → OpenAI Responses | Requests per CPU-second | 5.63 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window concurrency | 8 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window core utilisation | 0.868 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window successful requests | 125 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window throughput (req/s) | 20 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window threads | 68 | Valid | — |
| OpenAI Responses → OpenAI Responses | Non-voluntary context switches per request | 4.71 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window major page faults | 0 | Valid | — |
| OpenAI Responses → OpenAI Responses | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Streams sustained | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Idle RSS (MiB) | 612 | Valid | — |
| OpenAI Responses → OpenAI Responses | Steady-state RSS (MiB) | 1,019.7 | Valid | — |
| OpenAI Responses → OpenAI Responses | Recovered RSS (MiB) | 1,019.9 | Valid | — |
| OpenAI Responses → OpenAI Responses | Peak RSS (MiB) | 1,020 | Valid | — |
| OpenAI Responses → OpenAI Responses | Peak RSS high-water mark (MiB) | 1,019.9 | Valid | — |
| OpenAI Responses → OpenAI Responses | Time to plateau (s) | 66 | Valid | — |
| OpenAI Responses → OpenAI Responses | Memory growth (MiB/min) | -0.199 | Valid | — |
| OpenAI Responses → OpenAI Responses | Memory load duration (s) | 301 | Valid | — |
| OpenAI Responses → OpenAI Responses | Memory shape under load | — | Not measured | — |
| OpenAI Responses → OpenAI Responses | Idle memory shape | — | Not measured | — |
| OpenAI Responses → OpenAI Responses | Idle memory static | 1 | Valid | — |
| OpenAI Responses → OpenAI Responses | Idle memory growth (MiB/min) | 0.046 | Valid | — |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound | 50 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound | 100 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | 21 | concurrency 16; n=127 | Unbounded |
| Cohere → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Bedrock → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Cohere → Gemini | — | — | Not declared: recorded note 1 | — |
| OpenAI → Gemini | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Gemini | — | — | Not declared: recorded note 1 | — |
| Anthropic → Gemini | — | — | Not declared: recorded note 1 | — |
| Gemini → Gemini | — | — | Not declared: recorded note 1 | — |
| Bedrock → Gemini | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Cohere | — | — | Not declared: recorded note 1 | — |
| Cohere → Cohere | — | — | Not declared: recorded note 1 | — |
| OpenAI → Cohere | — | — | Not declared: recorded note 1 | — |
| Gemini → Cohere | — | — | Not declared: recorded note 1 | — |
| Anthropic → Cohere | — | — | Not declared: recorded note 1 | — |
| Bedrock → Cohere | — | — | Not declared: recorded note 1 | — |
| OpenAI → OpenAI | Added latency p50 (µs) | 198,988 | Valid · n=30 gateway / 266,566 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 205,966 | Valid · n=30 gateway / 266,566 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 205,993 | Valid · n=30 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 27 | Valid · n=266,566 | — |
| OpenAI → OpenAI | CPU per request (µs) | 182,230.8 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 5.49 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.965 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 130 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 21 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 68 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 6.28 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | 158,773 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | 170,494 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | 1 | Valid · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | 38 | Valid · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 669.7 | Valid | — |
| OpenAI → OpenAI | Streams sustained | 29 | Valid | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 617.2 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 971.8 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 972.3 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 972.4 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 972.3 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | 61 | Valid | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | 0.129 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 305 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | — | Not measured | — |
| OpenAI → OpenAI | Idle memory shape | — | Not measured | — |
| OpenAI → OpenAI | Idle memory static | 1 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | 0.052 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 19 | concurrency 8; n=118 | Unbounded |
| OpenAI Responses → OpenAI | — | — | Not declared: recorded note 1 | — |
| Bedrock → OpenAI | — | — | Not declared: recorded note 1 | — |
| Anthropic → OpenAI | Added latency p50 (µs) | 195,998 | Valid · n=30 gateway / 261,879 direct | — |
| Anthropic → OpenAI | Added latency p99 (µs) | 198,984 | Valid · n=30 gateway / 261,879 direct | — |
| Anthropic → OpenAI | Gateway p99 at concurrency 1 (µs) | 199,012 | Valid · n=30 | — |
| Anthropic → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=261,879 | — |
| Anthropic → OpenAI | CPU per request (µs) | 178,977.3 | Valid | — |
| Anthropic → OpenAI | Requests per CPU-second | 5.59 | Valid | — |
| Anthropic → OpenAI | Cost window concurrency | 8 | Valid | — |
| Anthropic → OpenAI | Cost window core utilisation | 0.647 | Valid | — |
| Anthropic → OpenAI | Cost window successful requests | 88 | Valid | — |
| Anthropic → OpenAI | Cost window throughput (req/s) | 14 | Valid | — |
| Anthropic → OpenAI | Cost window threads | 84 | Valid | — |
| Anthropic → OpenAI | Non-voluntary context switches per request | 5.84 | Valid | — |
| Anthropic → OpenAI | Cost window major page faults | 0 | Valid | — |
| Anthropic → OpenAI | Added time to first event p50 (µs) | 156,909 | Valid · n=100 gateway / 100 direct | — |
| Anthropic → OpenAI | Added time to first event p99 (µs) | 168,479 | Valid · n=100 gateway / 100 direct | — |
| Anthropic → OpenAI | Added inter-frame gap p50 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| Anthropic → OpenAI | Added inter-frame gap p99 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| Anthropic → OpenAI | Streams sustained (frames/s) | 787.6 | Valid | — |
| Anthropic → OpenAI | Streams sustained | 47 | Valid | — |
| Anthropic → OpenAI | Idle RSS (MiB) | 605.9 | Valid | — |
| Anthropic → OpenAI | Steady-state RSS (MiB) | 961.5 | Valid | — |
| Anthropic → OpenAI | Recovered RSS (MiB) | 961.3 | Valid | — |
| Anthropic → OpenAI | Peak RSS (MiB) | 961.9 | Valid | — |
| Anthropic → OpenAI | Peak RSS high-water mark (MiB) | 961.7 | Valid | — |
| Anthropic → OpenAI | Time to plateau (s) | 61 | Valid | — |
| Anthropic → OpenAI | Memory growth (MiB/min) | 0.206 | Valid | — |
| Anthropic → OpenAI | Memory load duration (s) | 304 | Valid | — |
| Anthropic → OpenAI | Memory shape under load | — | Not measured | — |
| Anthropic → OpenAI | Idle memory shape | — | Not measured | — |
| Anthropic → OpenAI | Idle memory static | 1 | Valid | — |
| Anthropic → OpenAI | Idle memory growth (MiB/min) | 0.172 | Valid | — |
| Anthropic → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| Anthropic → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 ms |
| Anthropic → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 ms |
| Anthropic → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound | 50 ms |
| Anthropic → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound | 100 ms |
| Anthropic → OpenAI | Throughput (req/s) | 21 | concurrency 8; n=128 | Unbounded |
| Cohere → OpenAI | — | — | Not declared: recorded note 1 | — |
| Gemini → OpenAI | — | — | Not declared: recorded note 1 | — |
Probe evidence and instrument provenance
Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.
| Ingress → upstream | Cell state | Probe HTTP status | Recorded reason | Recorded detail | Evidence scope | Response excerpt | Tested build |
|---|---|---|---|---|---|---|---|
| OpenAI → Bedrock | Served | — | — | — | Recorded run evidence | — | katanemo/plano:0.4.39 |
| Cohere → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Gemini → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI Responses → Bedrock | Served | — | — | — | Recorded run evidence | — | katanemo/plano:0.4.39 |
| Anthropic → Bedrock | Served | — | — | — | Recorded run evidence | — | katanemo/plano:0.4.39 |
| Bedrock → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI Responses → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Anthropic → Anthropic | Served | — | — | — | Recorded run evidence | — | katanemo/plano:0.4.39 |
| Cohere → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Gemini → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Bedrock → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Anthropic → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Gemini → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI Responses → OpenAI Responses | Served | — | — | — | Recorded run evidence | — | katanemo/plano:0.4.39 |
| Cohere → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Bedrock → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Cohere → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI Responses → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Anthropic → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Gemini → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Bedrock → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI Responses → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Cohere → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Gemini → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Anthropic → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Bedrock → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| OpenAI → OpenAI | Served | — | — | — | Recorded run evidence | — | katanemo/plano:0.4.39 |
| OpenAI Responses → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Bedrock → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Anthropic → OpenAI | Served | — | — | — | Recorded run evidence | — | katanemo/plano:0.4.39 |
| Cohere → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
| Gemini → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | katanemo/plano:0.4.39 |
- Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2- Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8- Load generator SHA-256
not recorded- net.core.somaxconn
- 65,535
- net.ipv4.tcp_max_syn_backlog
- 65,535
Stream sweep diagnostics
Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.
| Ingress → upstream | Concurrency | Lanes by HTTP status | Lanes with no response |
|---|---|---|---|
| Anthropic → Anthropic | 1 | HTTP 200: 1 | 0 |
| Anthropic → Anthropic | 2 | HTTP 200: 2 | 0 |
| Anthropic → Anthropic | 4 | HTTP 200: 4 | 0 |
| Anthropic → Anthropic | 8 | HTTP 200: 8 | 0 |
| Anthropic → Anthropic | 16 | HTTP 200: 16 | 0 |
| Anthropic → Anthropic | 32 | HTTP 200: 32 | 0 |
| Anthropic → Anthropic | 64 | HTTP 200: 64 | 0 |
| Anthropic → Anthropic | 128 | HTTP 200: 128 | 0 |
| Anthropic → Anthropic | 96 | HTTP 200: 96 | 0 |
| Anthropic → Anthropic | 80 | HTTP 200: 80 | 0 |
| Anthropic → Anthropic | 72 | HTTP 200: 72 | 0 |
| Anthropic → Anthropic | 68 | HTTP 200: 68 | 0 |
| Anthropic → Anthropic | 66 | HTTP 200: 66 | 0 |
| Anthropic → Anthropic | 65 | HTTP 200: 65 | 0 |
| Anthropic → Anthropic | 65 | HTTP 200: 65 | 0 |
| Anthropic → Anthropic | 65 | HTTP 200: 65 | 0 |
| OpenAI → OpenAI | 1 | HTTP 200: 1 | 0 |
| OpenAI → OpenAI | 2 | HTTP 200: 2 | 0 |
| OpenAI → OpenAI | 4 | HTTP 200: 4 | 0 |
| OpenAI → OpenAI | 8 | HTTP 200: 8 | 0 |
| OpenAI → OpenAI | 16 | HTTP 200: 16 | 0 |
| OpenAI → OpenAI | 32 | HTTP 200: 32 | 0 |
| OpenAI → OpenAI | 24 | HTTP 200: 24 | 0 |
| OpenAI → OpenAI | 28 | HTTP 200: 28 | 0 |
| OpenAI → OpenAI | 30 | HTTP 200: 30 | 0 |
| OpenAI → OpenAI | 29 | HTTP 200: 29 | 0 |
| OpenAI → OpenAI | 29 | HTTP 200: 29 | 0 |
| OpenAI → OpenAI | 29 | HTTP 200: 29 | 0 |
| Anthropic → OpenAI | 1 | HTTP 200: 1 | 0 |
| Anthropic → OpenAI | 2 | HTTP 200: 2 | 0 |
| Anthropic → OpenAI | 4 | HTTP 200: 4 | 0 |
| Anthropic → OpenAI | 8 | HTTP 200: 8 | 0 |
| Anthropic → OpenAI | 16 | HTTP 200: 16 | 0 |
| Anthropic → OpenAI | 32 | HTTP 200: 32 | 0 |
| Anthropic → OpenAI | 64 | HTTP 200: 64 | 0 |
| Anthropic → OpenAI | 48 | HTTP 200: 48 | 0 |
| Anthropic → OpenAI | 40 | HTTP 200: 40 | 0 |
| Anthropic → OpenAI | 44 | HTTP 200: 44 | 0 |
| Anthropic → OpenAI | 46 | HTTP 200: 46 | 0 |
| Anthropic → OpenAI | 47 | HTTP 200: 47 | 0 |
| Anthropic → OpenAI | 47 | HTTP 200: 47 | 0 |
| Anthropic → OpenAI | 47 | HTTP 200: 47 | 0 |
Recorded notes
Text recorded with the cells above. Each note is shown once; rows refer to it by number.
- Pinned 0.4.39 source b13ddd632bde1fea7678386c345665ea5fa671af: hermesllm/src/clients/endpoints.rs accepts Chat, Responses and Messages ingress, with no Gemini, Cohere or Bedrock client API. providers/id.rs::compatible_api_for_client selects native Responses only for Responses ingress to OpenAI, Chat for Messages-to-OpenAI, native Messages for Messages-to-Anthropic, and Converse for all three ingress APIs to amazon_bedrock. Anthropic Chat/Responses ingress and Gemini provider use OpenAI-compatible upstream wire; they are not native Anthropic/Gemini translations. The normal provider mapping is retained rather than substituting a different provider for the OpenAI column. Seven supported pairs are enabled, including the documented Bedrock upstream.
- Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.
Evidence for the current measurements
- run-plano-0a4ce97da36f11e7c407eef9280f69688b7ea1f007616709f0b914993cb8ba8e · complete raw record including RSS series
Configuration recorded by the run
# Plano, configured as an LLM gateway pointed at the harness mock.
#
# `type: model` is the LLM listener; `type: agent` is the orchestration listener the project's README
# leads with, and it is NOT what this benchmark measures - an agent listener would put Plano's router
# and its agent hop inside a number the rest of the field takes without one.
#
# The listener port is 12000, not 10000. Plano always renders a prompt-gateway listener on 10000
# (utils.py convert_legacy_listeners), so an LLM listener on 10000 double-binds and envoy exits at
# boot with "'egress_traffic' has duplicate address '0.0.0.0:10000'". 12000 is Plano's own default
# for this listener.
#
# NO provider_interface key here, deliberately. When the model prefix is one Plano already knows -
# openai, anthropic and gemini all are - setting provider_interface as well is rejected by
# config_generator.py:445, which then exits and takes the container down with it. The prefix selects
# the interface on its own. (The error text reads "Please provide provider interface as part of model
# name", which is the reverse of what the check does; the condition is `provider in
# SUPPORTED_PROVIDERS and provider_interface is not None`.)
#
# base_url is what makes Plano testable at all: several entrants pin the vendor host and their
# upstream cells can only be recorded untestable. Plano's model_providers take base_url, so the mock
# stands in for every provider and the measured number is Plano's own cost.
#
# Provider prefixes select the released adapters. Capability declarations are source-backed in
# definition.json; correct ingress and upstream wire shapes are checked separately.
version: v0.3.0
listeners:
- name: llm
type: model
port: 12000
model_providers:
- model: openai/gpt-4o-mini
base_url: 'http://127.0.0.1:8000'
access_key: dummy
default: true
- model: anthropic/claude-3-5-sonnet
base_url: 'http://127.0.0.1:8000'
access_key: dummy
- model: gemini/gemini-1.5-pro
base_url: 'http://127.0.0.1:8000'
access_key: dummy
- model: amazon_bedrock/anthropic.claude-3-sonnet-20240229-v1:0
base_url: 'http://127.0.0.1:8000'
access_key: dummy
Compare a shared protocol cell
- Plano vs agentgateway
- Plano vs AISIX (api7)
- Plano vs APISIX
- Plano vs Bifrost
- Plano vs Busbar
- Plano vs GoModel
- Plano vs Helicone
- Plano vs Higress
- Plano vs Kong
- Plano vs LiteLLM · Python
- Plano vs LiteLLM · Rust
- Plano vs One-API
- Plano vs Portkey
- Plano vs TensorZero