Portkey vs Plano

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Requested cell: OpenAI → OpenAI. Metric: Added latency p99 (µs).

Comparability: Comparable. Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.

Ratio (Plano ÷ Portkey): 52.78. No statistical significance is asserted.

Comparison JSON · All Portkey evidence · All Plano evidence

Portkey

Build portkeyai/gateway:1.15.2. Measured . Run evidence

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
OpenAI → OpenAIAdded latency p50 (µs)1,196Valid · n=4,304 gateway / 246,357 direct—
OpenAI → OpenAIAdded latency p99 (µs)3,902Valid · n=4,304 gateway / 246,357 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)3,938Valid · n=4,304—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)36Valid · n=246,357—
OpenAI → OpenAICPU per request (µs)1,528.4Valid—
OpenAI → OpenAIRequests per CPU-second654.3Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.295Valid—
OpenAI → OpenAICost window successful requests4,652Valid—
OpenAI → OpenAICost window throughput (req/s)775Valid—
OpenAI → OpenAICost window threads22Valid—
OpenAI → OpenAINon-voluntary context switches per request0.007Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)28,114Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)29,752Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)0Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)440Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)4,996.2Valid—
OpenAI → OpenAIStreams sustained1,519Valid—
OpenAI → OpenAIIdle RSS (MiB)125.1Valid—
OpenAI → OpenAISteady-state RSS (MiB)245.2Valid—
OpenAI → OpenAIRecovered RSS (MiB)142.1Valid—
OpenAI → OpenAIPeak RSS (MiB)250Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)264Valid—
OpenAI → OpenAITime to plateau (s)—Not measured—
OpenAI → OpenAIMemory growth (MiB/min)0.202Valid—
OpenAI → OpenAIMemory load duration (s)301Valid—
OpenAI → OpenAIMemory shape under load0Valid—
OpenAI → OpenAIIdle memory shape-1Valid—
OpenAI → OpenAIIdle memory static0Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)-13.96Valid—
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → OpenAIThroughput (req/s)831concurrency 1; n=4,9925 ms
OpenAI → OpenAIThroughput (req/s)835concurrency 2; n=5,01210 ms
OpenAI → OpenAIThroughput (req/s)836concurrency 8; n=5,01850 ms
OpenAI → OpenAIThroughput (req/s)836concurrency 8; n=5,018100 ms
OpenAI → OpenAIThroughput (req/s)836concurrency 8; n=5,018Unbounded

Plano

Build katanemo/plano:0.4.39. Measured . Run evidence

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
OpenAI → OpenAIAdded latency p50 (µs)198,988Valid · n=30 gateway / 266,566 direct—
OpenAI → OpenAIAdded latency p99 (µs)205,966Valid · n=30 gateway / 266,566 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)205,993Valid · n=30—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=266,566—
OpenAI → OpenAICPU per request (µs)182,230.8Valid—
OpenAI → OpenAIRequests per CPU-second5.49Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.965Valid—
OpenAI → OpenAICost window successful requests130Valid—
OpenAI → OpenAICost window throughput (req/s)21Valid—
OpenAI → OpenAICost window threads68Valid—
OpenAI → OpenAINon-voluntary context switches per request6.28Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)158,773Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)170,494Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)1Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)38Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)669.7Valid—
OpenAI → OpenAIStreams sustained29Valid—
OpenAI → OpenAIIdle RSS (MiB)617.2Valid—
OpenAI → OpenAISteady-state RSS (MiB)971.8Valid—
OpenAI → OpenAIRecovered RSS (MiB)972.3Valid—
OpenAI → OpenAIPeak RSS (MiB)972.4Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)972.3Valid—
OpenAI → OpenAITime to plateau (s)61Valid—
OpenAI → OpenAIMemory growth (MiB/min)0.129Valid—
OpenAI → OpenAIMemory load duration (s)305Valid—
OpenAI → OpenAIMemory shape under load—Not measured—
OpenAI → OpenAIIdle memory shape—Not measured—
OpenAI → OpenAIIdle memory static1Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)0.052Valid—
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound1 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound5 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound10 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 50ms, so this gateway carried no measurable throughput under that bound50 ms
OpenAI → OpenAIThroughput (req/s)—Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 100ms, so this gateway carried no measurable throughput under that bound100 ms
OpenAI → OpenAIThroughput (req/s)19concurrency 8; n=118Unbounded