Portkey vs AISIX (api7)
Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.
Requested cell: OpenAI → OpenAI. Metric: Added latency p99 (µs).
Comparability: Comparable. Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.
- Load-generator binary digests were not recorded; instrument equivalence is based on the identical clean harness source commit and mock digest.
- Measurements were made at different times on independently provisioned machines.
Ratio (AISIX (api7) ÷ Portkey): 0.045. No statistical significance is asserted.
Comparison JSON · All Portkey evidence · All AISIX (api7) evidence
Portkey
Build portkeyai/gateway:1.15.2. Measured . Run evidence
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| OpenAI → OpenAI | Added latency p50 (µs) | 1,196 | Valid · n=4,304 gateway / 246,357 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 3,902 | Valid · n=4,304 gateway / 246,357 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 3,938 | Valid · n=4,304 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 36 | Valid · n=246,357 | — |
| OpenAI → OpenAI | CPU per request (µs) | 1,528.4 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 654.3 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.295 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 4,652 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 775 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 22 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 0.007 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | 28,114 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | 29,752 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | 0 | Valid · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | 440 | Valid · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 4,996.2 | Valid | — |
| OpenAI → OpenAI | Streams sustained | 1,519 | Valid | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 125.1 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 245.2 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 142.1 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 250 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 264 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | — | Not measured | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | 0.202 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 301 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | 0 | Valid | — |
| OpenAI → OpenAI | Idle memory shape | -1 | Valid | — |
| OpenAI → OpenAI | Idle memory static | 0 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | -13.96 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | — | Bound not met; concurrency not recorded: every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | 831 | concurrency 1; n=4,992 | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | 835 | concurrency 2; n=5,012 | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | 836 | concurrency 8; n=5,018 | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | 836 | concurrency 8; n=5,018 | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 836 | concurrency 8; n=5,018 | Unbounded |
AISIX (api7)
Build ghcr.io/api7/aisix:1.9.0. Measured . Run evidence
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| OpenAI → OpenAI | Added latency p50 (µs) | 146 | Valid · n=35,048 gateway / 262,627 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 174 | Valid · n=35,048 gateway / 262,627 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 202 | Valid · n=35,048 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=262,627 | — |
| OpenAI → OpenAI | CPU per request (µs) | 152.6 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 6,551.3 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.945 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 148,977 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 24,826 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 16 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | 277 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | 366 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | 2 | Valid · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 30,259.2 | Valid | — |
| OpenAI → OpenAI | Streams sustained | 645 | Valid | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 34.29 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 68.63 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 52.3 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 70.43 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 70.29 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | — | Not measured | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | 0.376 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 300 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | 0 | Valid | — |
| OpenAI → OpenAI | Idle memory shape | -1 | Valid | — |
| OpenAI → OpenAI | Idle memory static | 0 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | -0.853 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | 28,888 | concurrency 16; n=173,346 | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | 31,374 | concurrency 64; n=188,254 | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | 32,092 | concurrency 128; n=192,583 | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | 32,092 | concurrency 128; n=192,583 | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | 32,092 | concurrency 128; n=192,583 | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 32,092 | concurrency 128; n=192,583 | Unbounded |