Plano benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Rust · recorded build: katanemo/plano:0.4.39. Measured: .

JSON · Markdown · CSV · Project source · Interactive dashboard

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

Published measurements

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
OpenAI → BedrockAdded latency p50 (µs)205,998Valid · n=29 gateway / 262,765 direct—
OpenAI → BedrockAdded latency p99 (µs)212,984Valid · n=29 gateway / 262,765 direct—
OpenAI → BedrockGateway p99 at concurrency 1 (µs)213,012Valid · n=29—
OpenAI → BedrockDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=262,765—
OpenAI → BedrockCPU per request (µs)189,176.5Valid—
OpenAI → BedrockRequests per CPU-second5.29Valid—
OpenAI → BedrockCost window concurrency8Valid—
OpenAI → BedrockCost window core utilisation0.617Valid—
OpenAI → BedrockCost window successful requests85Valid—
OpenAI → BedrockCost window throughput (req/s)14Valid—
OpenAI → BedrockCost window threads68Valid—
OpenAI → BedrockNon-voluntary context switches per request5.14Valid—
OpenAI → BedrockCost window major page faults0Valid—
OpenAI → BedrockAdded time to first event p50 (µs)—Untestable—
OpenAI → BedrockAdded time to first event p99 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p50 (µs)—Untestable—
OpenAI → BedrockAdded inter-frame gap p99 (µs)—Untestable—
OpenAI → BedrockStreams sustained (frames/s)—Untestable—
OpenAI → BedrockStreams sustained—Untestable—
OpenAI → BedrockIdle RSS (MiB)598.8Valid—
OpenAI → BedrockSteady-state RSS (MiB)1,006.3Valid—
OpenAI → BedrockRecovered RSS (MiB)1,006.2Valid—
OpenAI → BedrockPeak RSS (MiB)1,006.7Valid—
OpenAI → BedrockPeak RSS high-water mark (MiB)1,006.5Valid—
OpenAI → BedrockTime to plateau (s)61Valid—
OpenAI → BedrockMemory growth (MiB/min)-0.329Valid—
OpenAI → BedrockMemory load duration (s)306Valid—
OpenAI → BedrockMemory shape under load—Not measured—
OpenAI → BedrockIdle memory shape—Not measured—
OpenAI → BedrockIdle memory static1Valid—
OpenAI → BedrockIdle memory growth (MiB/min)0.047Valid—
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound50 ms
OpenAI → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound100 ms
OpenAI → BedrockThroughput (req/s)21concurrency 8; n=127Unbounded
Cohere → Bedrock——Not declared: recorded note 1—
Gemini → Bedrock——Not declared: recorded note 1—
OpenAI Responses → BedrockAdded latency p50 (µs)200,013Valid · n=30 gateway / 265,071 direct—
OpenAI Responses → BedrockAdded latency p99 (µs)205,956Valid · n=30 gateway / 265,071 direct—
OpenAI Responses → BedrockGateway p99 at concurrency 1 (µs)205,984Valid · n=30—
OpenAI Responses → BedrockDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=265,071—
OpenAI Responses → BedrockCPU per request (µs)182,500Valid—
OpenAI Responses → BedrockRequests per CPU-second5.48Valid—
OpenAI Responses → BedrockCost window concurrency8Valid—
OpenAI Responses → BedrockCost window core utilisation0.859Valid—
OpenAI Responses → BedrockCost window successful requests116Valid—
OpenAI Responses → BedrockCost window throughput (req/s)19Valid—
OpenAI Responses → BedrockCost window threads68Valid—
OpenAI Responses → BedrockNon-voluntary context switches per request4.62Valid—
OpenAI Responses → BedrockCost window major page faults0Valid—
OpenAI Responses → BedrockAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → BedrockAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → BedrockAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → BedrockAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → BedrockStreams sustained (frames/s)—Untestable—
OpenAI Responses → BedrockStreams sustained—Untestable—
OpenAI Responses → BedrockIdle RSS (MiB)621.6Valid—
OpenAI Responses → BedrockSteady-state RSS (MiB)1,055.5Valid—
OpenAI Responses → BedrockRecovered RSS (MiB)1,055.9Valid—
OpenAI Responses → BedrockPeak RSS (MiB)1,055.9Valid—
OpenAI Responses → BedrockPeak RSS high-water mark (MiB)1,055.9Valid—
OpenAI Responses → BedrockTime to plateau (s)66Valid—
OpenAI Responses → BedrockMemory growth (MiB/min)0.072Valid—
OpenAI Responses → BedrockMemory load duration (s)305Valid—
OpenAI Responses → BedrockMemory shape under load—Not measured—
OpenAI Responses → BedrockIdle memory shape—Not measured—
OpenAI Responses → BedrockIdle memory static1Valid—
OpenAI Responses → BedrockIdle memory growth (MiB/min)0.048Valid—
OpenAI Responses → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI Responses → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI Responses → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI Responses → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound50 ms
OpenAI Responses → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound100 ms
OpenAI Responses → BedrockThroughput (req/s)20concurrency 16; n=121Unbounded
Anthropic → BedrockAdded latency p50 (µs)193,978Valid · n=31 gateway / 259,772 direct—
Anthropic → BedrockAdded latency p99 (µs)197,949Valid · n=31 gateway / 259,772 direct—
Anthropic → BedrockGateway p99 at concurrency 1 (µs)197,977Valid · n=31—
Anthropic → BedrockDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=259,772—
Anthropic → BedrockCPU per request (µs)175,360Valid—
Anthropic → BedrockRequests per CPU-second5.7Valid—
Anthropic → BedrockCost window concurrency8Valid—
Anthropic → BedrockCost window core utilisation0.86Valid—
Anthropic → BedrockCost window successful requests125Valid—
Anthropic → BedrockCost window throughput (req/s)20Valid—
Anthropic → BedrockCost window threads68Valid—
Anthropic → BedrockNon-voluntary context switches per request5.32Valid—
Anthropic → BedrockCost window major page faults0Valid—
Anthropic → BedrockAdded time to first event p50 (µs)—Untestable—
Anthropic → BedrockAdded time to first event p99 (µs)—Untestable—
Anthropic → BedrockAdded inter-frame gap p50 (µs)—Untestable—
Anthropic → BedrockAdded inter-frame gap p99 (µs)—Untestable—
Anthropic → BedrockStreams sustained (frames/s)—Untestable—
Anthropic → BedrockStreams sustained—Untestable—
Anthropic → BedrockIdle RSS (MiB)612.5Valid—
Anthropic → BedrockSteady-state RSS (MiB)967.7Valid—
Anthropic → BedrockRecovered RSS (MiB)967.4Valid—
Anthropic → BedrockPeak RSS (MiB)968Valid—
Anthropic → BedrockPeak RSS high-water mark (MiB)967.7Valid—
Anthropic → BedrockTime to plateau (s)60Valid—
Anthropic → BedrockMemory growth (MiB/min)0.252Valid—
Anthropic → BedrockMemory load duration (s)302Valid—
Anthropic → BedrockMemory shape under load—Not measured—
Anthropic → BedrockIdle memory shape—Not measured—
Anthropic → BedrockIdle memory static1Valid—
Anthropic → BedrockIdle memory growth (MiB/min)0.052Valid—
Anthropic → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
Anthropic → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
Anthropic → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound50 ms
Anthropic → BedrockThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound100 ms
Anthropic → BedrockThroughput (req/s)21concurrency 16; n=127Unbounded
Bedrock → Bedrock——Not declared: recorded note 1—
OpenAI → Anthropic——Not declared: recorded note 1—
OpenAI Responses → Anthropic——Not declared: recorded note 1—
Anthropic → AnthropicAdded latency p50 (µs)193,933Valid · n=31 gateway / 264,589 direct—
Anthropic → AnthropicAdded latency p99 (µs)195,022Valid · n=31 gateway / 264,589 direct—
Anthropic → AnthropicGateway p99 at concurrency 1 (µs)195,049Valid · n=31—
Anthropic → AnthropicDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=264,589—
Anthropic → AnthropicCPU per request (µs)181,353.4Valid—
Anthropic → AnthropicRequests per CPU-second5.51Valid—
Anthropic → AnthropicCost window concurrency8Valid—
Anthropic → AnthropicCost window core utilisation0.961Valid—
Anthropic → AnthropicCost window successful requests133Valid—
Anthropic → AnthropicCost window throughput (req/s)22Valid—
Anthropic → AnthropicCost window threads69Valid—
Anthropic → AnthropicNon-voluntary context switches per request5.88Valid—
Anthropic → AnthropicCost window major page faults0Valid—
Anthropic → AnthropicAdded time to first event p50 (µs)153,935Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded time to first event p99 (µs)165,859Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → AnthropicAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → AnthropicStreams sustained (frames/s)920Valid—
Anthropic → AnthropicStreams sustained65Valid—
Anthropic → AnthropicIdle RSS (MiB)593.1Valid—
Anthropic → AnthropicSteady-state RSS (MiB)948.5Valid—
Anthropic → AnthropicRecovered RSS (MiB)948.4Valid—
Anthropic → AnthropicPeak RSS (MiB)949Valid—
Anthropic → AnthropicPeak RSS high-water mark (MiB)948.8Valid—
Anthropic → AnthropicTime to plateau (s)67Valid—
Anthropic → AnthropicMemory growth (MiB/min)-0.301Valid—
Anthropic → AnthropicMemory load duration (s)303Valid—
Anthropic → AnthropicMemory shape under load—Not measured—
Anthropic → AnthropicIdle memory shape—Not measured—
Anthropic → AnthropicIdle memory static1Valid—
Anthropic → AnthropicIdle memory growth (MiB/min)0.052Valid—
Anthropic → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
Anthropic → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
Anthropic → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound50 ms
Anthropic → AnthropicThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound100 ms
Anthropic → AnthropicThroughput (req/s)21concurrency 8; n=128Unbounded
Cohere → Anthropic——Not declared: recorded note 1—
Gemini → Anthropic——Not declared: recorded note 1—
Bedrock → Anthropic——Not declared: recorded note 1—
Anthropic → OpenAI Responses——Not declared: recorded note 1—
Gemini → OpenAI Responses——Not declared: recorded note 1—
OpenAI → OpenAI Responses——Not declared: recorded note 1—
OpenAI Responses → OpenAI ResponsesAdded latency p50 (µs)194,984Valid · n=30 gateway / 268,348 direct—
OpenAI Responses → OpenAI ResponsesAdded latency p99 (µs)198,964Valid · n=30 gateway / 268,348 direct—
OpenAI Responses → OpenAI ResponsesGateway p99 at concurrency 1 (µs)198,991Valid · n=30—
OpenAI Responses → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=268,348—
OpenAI Responses → OpenAI ResponsesCPU per request (µs)177,760Valid—
OpenAI Responses → OpenAI ResponsesRequests per CPU-second5.63Valid—
OpenAI Responses → OpenAI ResponsesCost window concurrency8Valid—
OpenAI Responses → OpenAI ResponsesCost window core utilisation0.868Valid—
OpenAI Responses → OpenAI ResponsesCost window successful requests125Valid—
OpenAI Responses → OpenAI ResponsesCost window throughput (req/s)20Valid—
OpenAI Responses → OpenAI ResponsesCost window threads68Valid—
OpenAI Responses → OpenAI ResponsesNon-voluntary context switches per request4.71Valid—
OpenAI Responses → OpenAI ResponsesCost window major page faults0Valid—
OpenAI Responses → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained—Untestable—
OpenAI Responses → OpenAI ResponsesIdle RSS (MiB)612Valid—
OpenAI Responses → OpenAI ResponsesSteady-state RSS (MiB)1,019.7Valid—
OpenAI Responses → OpenAI ResponsesRecovered RSS (MiB)1,019.9Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS (MiB)1,020Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS high-water mark (MiB)1,019.9Valid—
OpenAI Responses → OpenAI ResponsesTime to plateau (s)66Valid—
OpenAI Responses → OpenAI ResponsesMemory growth (MiB/min)-0.199Valid—
OpenAI Responses → OpenAI ResponsesMemory load duration (s)301Valid—
OpenAI Responses → OpenAI ResponsesMemory shape under load—Not measured—
OpenAI Responses → OpenAI ResponsesIdle memory shape—Not measured—
OpenAI Responses → OpenAI ResponsesIdle memory static1Valid—
OpenAI Responses → OpenAI ResponsesIdle memory growth (MiB/min)0.046Valid—
OpenAI Responses → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound50 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound100 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)21concurrency 16; n=127Unbounded
Cohere → OpenAI Responses——Not declared: recorded note 1—
Bedrock → OpenAI Responses——Not declared: recorded note 1—
Cohere → Gemini——Not declared: recorded note 1—
OpenAI → Gemini——Not declared: recorded note 1—
OpenAI Responses → Gemini——Not declared: recorded note 1—
Anthropic → Gemini——Not declared: recorded note 1—
Gemini → Gemini——Not declared: recorded note 1—
Bedrock → Gemini——Not declared: recorded note 1—
OpenAI Responses → Cohere——Not declared: recorded note 1—
Cohere → Cohere——Not declared: recorded note 1—
OpenAI → Cohere——Not declared: recorded note 1—
Gemini → Cohere——Not declared: recorded note 1—
Anthropic → Cohere——Not declared: recorded note 1—
Bedrock → Cohere——Not declared: recorded note 1—
OpenAI → OpenAIAdded latency p50 (µs)198,988Valid · n=30 gateway / 266,566 direct—
OpenAI → OpenAIAdded latency p99 (µs)205,966Valid · n=30 gateway / 266,566 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)205,993Valid · n=30—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=266,566—
OpenAI → OpenAICPU per request (µs)182,230.8Valid—
OpenAI → OpenAIRequests per CPU-second5.49Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.965Valid—
OpenAI → OpenAICost window successful requests130Valid—
OpenAI → OpenAICost window throughput (req/s)21Valid—
OpenAI → OpenAICost window threads68Valid—
OpenAI → OpenAINon-voluntary context switches per request6.28Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)158,773Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)170,494Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)1Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)38Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)669.7Valid—
OpenAI → OpenAIStreams sustained29Valid—
OpenAI → OpenAIIdle RSS (MiB)617.2Valid—
OpenAI → OpenAISteady-state RSS (MiB)971.8Valid—
OpenAI → OpenAIRecovered RSS (MiB)972.3Valid—
OpenAI → OpenAIPeak RSS (MiB)972.4Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)972.3Valid—
OpenAI → OpenAITime to plateau (s)61Valid—
OpenAI → OpenAIMemory growth (MiB/min)0.129Valid—
OpenAI → OpenAIMemory load duration (s)305Valid—
OpenAI → OpenAIMemory shape under load—Not measured—
OpenAI → OpenAIIdle memory shape—Not measured—
OpenAI → OpenAIIdle memory static1Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)0.052Valid—
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound50 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound100 ms
OpenAI → OpenAIThroughput (req/s)19concurrency 8; n=118Unbounded
OpenAI Responses → OpenAI——Not declared: recorded note 1—
Bedrock → OpenAI——Not declared: recorded note 1—
Anthropic → OpenAIAdded latency p50 (µs)195,998Valid · n=30 gateway / 261,879 direct—
Anthropic → OpenAIAdded latency p99 (µs)198,984Valid · n=30 gateway / 261,879 direct—
Anthropic → OpenAIGateway p99 at concurrency 1 (µs)199,012Valid · n=30—
Anthropic → OpenAIDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=261,879—
Anthropic → OpenAICPU per request (µs)178,977.3Valid—
Anthropic → OpenAIRequests per CPU-second5.59Valid—
Anthropic → OpenAICost window concurrency8Valid—
Anthropic → OpenAICost window core utilisation0.647Valid—
Anthropic → OpenAICost window successful requests88Valid—
Anthropic → OpenAICost window throughput (req/s)14Valid—
Anthropic → OpenAICost window threads84Valid—
Anthropic → OpenAINon-voluntary context switches per request5.84Valid—
Anthropic → OpenAICost window major page faults0Valid—
Anthropic → OpenAIAdded time to first event p50 (µs)156,909Valid · n=100 gateway / 100 direct—
Anthropic → OpenAIAdded time to first event p99 (µs)168,479Valid · n=100 gateway / 100 direct—
Anthropic → OpenAIAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → OpenAIAdded inter-frame gap p99 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → OpenAIStreams sustained (frames/s)787.6Valid—
Anthropic → OpenAIStreams sustained47Valid—
Anthropic → OpenAIIdle RSS (MiB)605.9Valid—
Anthropic → OpenAISteady-state RSS (MiB)961.5Valid—
Anthropic → OpenAIRecovered RSS (MiB)961.3Valid—
Anthropic → OpenAIPeak RSS (MiB)961.9Valid—
Anthropic → OpenAIPeak RSS high-water mark (MiB)961.7Valid—
Anthropic → OpenAITime to plateau (s)61Valid—
Anthropic → OpenAIMemory growth (MiB/min)0.206Valid—
Anthropic → OpenAIMemory load duration (s)304Valid—
Anthropic → OpenAIMemory shape under load—Not measured—
Anthropic → OpenAIIdle memory shape—Not measured—
Anthropic → OpenAIIdle memory static1Valid—
Anthropic → OpenAIIdle memory growth (MiB/min)0.172Valid—
Anthropic → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
Anthropic → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
Anthropic → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
Anthropic → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound50 ms
Anthropic → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound100 ms
Anthropic → OpenAIThroughput (req/s)21concurrency 8; n=128Unbounded
Cohere → OpenAI——Not declared: recorded note 1—
Gemini → OpenAI——Not declared: recorded note 1—

Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

Ingress → upstreamCell stateProbe HTTP statusRecorded reasonRecorded detailEvidence scopeResponse excerptTested build
OpenAI → BedrockServed———Recorded run evidence—katanemo/plano:0.4.39
Cohere → BedrockNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Gemini → BedrockNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI Responses → BedrockServed———Recorded run evidence—katanemo/plano:0.4.39
Anthropic → BedrockServed———Recorded run evidence—katanemo/plano:0.4.39
Bedrock → BedrockNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI Responses → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Anthropic → AnthropicServed———Recorded run evidence—katanemo/plano:0.4.39
Cohere → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Gemini → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Bedrock → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Anthropic → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Gemini → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI Responses → OpenAI ResponsesServed———Recorded run evidence—katanemo/plano:0.4.39
Cohere → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Bedrock → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Cohere → GeminiNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI → GeminiNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI Responses → GeminiNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Anthropic → GeminiNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Gemini → GeminiNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Bedrock → GeminiNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI Responses → CohereNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Cohere → CohereNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI → CohereNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Gemini → CohereNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Anthropic → CohereNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Bedrock → CohereNot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
OpenAI → OpenAIServed———Recorded run evidence—katanemo/plano:0.4.39
OpenAI Responses → OpenAINot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Bedrock → OpenAINot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Anthropic → OpenAIServed———Recorded run evidence—katanemo/plano:0.4.39
Cohere → OpenAINot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Gemini → OpenAINot declared—recorded note 1recorded note 1recorded note 2—katanemo/plano:0.4.39
Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2
Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8
Load generator SHA-256
not recorded
net.core.somaxconn
65,535
net.ipv4.tcp_max_syn_backlog
65,535

Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

Ingress → upstreamConcurrencyLanes by HTTP statusLanes with no response
Anthropic → Anthropic1HTTP 200: 10
Anthropic → Anthropic2HTTP 200: 20
Anthropic → Anthropic4HTTP 200: 40
Anthropic → Anthropic8HTTP 200: 80
Anthropic → Anthropic16HTTP 200: 160
Anthropic → Anthropic32HTTP 200: 320
Anthropic → Anthropic64HTTP 200: 640
Anthropic → Anthropic128HTTP 200: 1280
Anthropic → Anthropic96HTTP 200: 960
Anthropic → Anthropic80HTTP 200: 800
Anthropic → Anthropic72HTTP 200: 720
Anthropic → Anthropic68HTTP 200: 680
Anthropic → Anthropic66HTTP 200: 660
Anthropic → Anthropic65HTTP 200: 650
Anthropic → Anthropic65HTTP 200: 650
Anthropic → Anthropic65HTTP 200: 650
OpenAI → OpenAI1HTTP 200: 10
OpenAI → OpenAI2HTTP 200: 20
OpenAI → OpenAI4HTTP 200: 40
OpenAI → OpenAI8HTTP 200: 80
OpenAI → OpenAI16HTTP 200: 160
OpenAI → OpenAI32HTTP 200: 320
OpenAI → OpenAI24HTTP 200: 240
OpenAI → OpenAI28HTTP 200: 280
OpenAI → OpenAI30HTTP 200: 300
OpenAI → OpenAI29HTTP 200: 290
OpenAI → OpenAI29HTTP 200: 290
OpenAI → OpenAI29HTTP 200: 290
Anthropic → OpenAI1HTTP 200: 10
Anthropic → OpenAI2HTTP 200: 20
Anthropic → OpenAI4HTTP 200: 40
Anthropic → OpenAI8HTTP 200: 80
Anthropic → OpenAI16HTTP 200: 160
Anthropic → OpenAI32HTTP 200: 320
Anthropic → OpenAI64HTTP 200: 640
Anthropic → OpenAI48HTTP 200: 480
Anthropic → OpenAI40HTTP 200: 400
Anthropic → OpenAI44HTTP 200: 440
Anthropic → OpenAI46HTTP 200: 460
Anthropic → OpenAI47HTTP 200: 470
Anthropic → OpenAI47HTTP 200: 470
Anthropic → OpenAI47HTTP 200: 470

Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

  1. Pinned 0.4.39 source b13ddd632bde1fea7678386c345665ea5fa671af: hermesllm/src/clients/endpoints.rs accepts Chat, Responses and Messages ingress, with no Gemini, Cohere or Bedrock client API. providers/id.rs::compatible_api_for_client selects native Responses only for Responses ingress to OpenAI, Chat for Messages-to-OpenAI, native Messages for Messages-to-Anthropic, and Converse for all three ingress APIs to amazon_bedrock. Anthropic Chat/Responses ingress and Gemini provider use OpenAI-compatible upstream wire; they are not native Anthropic/Gemini translations. The normal provider mapping is retained rather than substituting a different provider for the OpenAI column. Seven supported pairs are enabled, including the documented Bedrock upstream.
  2. Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

Evidence for the current measurements

Configuration recorded by the run

# Plano, configured as an LLM gateway pointed at the harness mock.
#
# `type: model` is the LLM listener; `type: agent` is the orchestration listener the project's README
# leads with, and it is NOT what this benchmark measures - an agent listener would put Plano's router
# and its agent hop inside a number the rest of the field takes without one.
#
# The listener port is 12000, not 10000. Plano always renders a prompt-gateway listener on 10000
# (utils.py convert_legacy_listeners), so an LLM listener on 10000 double-binds and envoy exits at
# boot with "'egress_traffic' has duplicate address '0.0.0.0:10000'". 12000 is Plano's own default
# for this listener.
#
# NO provider_interface key here, deliberately. When the model prefix is one Plano already knows -
# openai, anthropic and gemini all are - setting provider_interface as well is rejected by
# config_generator.py:445, which then exits and takes the container down with it. The prefix selects
# the interface on its own. (The error text reads "Please provide provider interface as part of model
# name", which is the reverse of what the check does; the condition is `provider in
# SUPPORTED_PROVIDERS and provider_interface is not None`.)
#
# base_url is what makes Plano testable at all: several entrants pin the vendor host and their
# upstream cells can only be recorded untestable. Plano's model_providers take base_url, so the mock
# stands in for every provider and the measured number is Plano's own cost.
#
# Provider prefixes select the released adapters. Capability declarations are source-backed in
# definition.json; correct ingress and upstream wire shapes are checked separately.
version: v0.3.0

listeners:
  - name: llm
    type: model
    port: 12000

model_providers:
  - model: openai/gpt-4o-mini
    base_url: 'http://127.0.0.1:8000'
    access_key: dummy
    default: true

  - model: anthropic/claude-3-5-sonnet
    base_url: 'http://127.0.0.1:8000'
    access_key: dummy

  - model: gemini/gemini-1.5-pro
    base_url: 'http://127.0.0.1:8000'
    access_key: dummy

  - model: amazon_bedrock/anthropic.claude-3-sonnet-20240229-v1:0
    base_url: 'http://127.0.0.1:8000'
    access_key: dummy

Compare a shared protocol cell

Historical measurements