Helicone vs Higress
Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.
Requested cell: OpenAI → OpenAI. Metric: Added latency p99 (µs).
Comparability: Comparable. Ratios require an identical protocol pair, hardware, architecture, metric definition, clean harness commit, recorded mock digest, and metric-specific workload. Missing evidence never matches missing evidence. These are observed gateway measurements against a local mock, not a claim about model latency or statistical significance.
- Load-generator binary digests were not recorded; instrument equivalence is based on the identical clean harness source commit and mock digest.
- Measurements were made at different times on independently provisioned machines.
Ratio (Higress ÷ Helicone): 3.44. No statistical significance is asserted.
Comparison JSON · All Helicone evidence · All Higress evidence
Helicone
Build target/release/ai-gateway. Measured . Run evidence
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| OpenAI → OpenAI | Added latency p50 (µs) | 256 | Valid · n=21,418 gateway / 260,967 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 274 | Valid · n=21,418 gateway / 260,967 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 302 | Valid · n=21,418 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=260,967 | — |
| OpenAI → OpenAI | CPU per request (µs) | 259.4 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 3,855.3 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.984 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 91,294 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 15,213 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 24 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | 399 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | 493 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | 0 | Valid · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 21,883.1 | Valid | — |
| OpenAI → OpenAI | Streams sustained | 463 | Valid | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 42.97 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 56.18 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 53.69 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 56.95 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 56.61 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | 126 | Valid | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | 0.5 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 300 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | — | Not measured | — |
| OpenAI → OpenAI | Idle memory shape | — | Not measured | — |
| OpenAI → OpenAI | Idle memory static | 1 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | 0.124 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | 15,405 | concurrency 8; n=92,444 | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | 16,288 | concurrency 32; n=97,741 | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | 16,376 | concurrency 64; n=98,266 | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | 16,376 | concurrency 64; n=98,266 | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | 16,376 | concurrency 64; n=98,266 | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 16,376 | concurrency 64; n=98,266 | Unbounded |
Higress
Build higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:2.2.5. Measured . Run evidence
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| OpenAI → OpenAI | Added latency p50 (µs) | 718 | Valid · n=7,996 gateway / 262,593 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 943 | Valid · n=7,996 gateway / 262,593 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 971 | Valid · n=7,996 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=262,593 | — |
| OpenAI → OpenAI | CPU per request (µs) | 709.3 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 1,409.8 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.733 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 24,883 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 4,146 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 142 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 0.008 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | 881 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | 1,103 | Valid · n=100 gateway / 100 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | 2 | Valid · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | — | Below resolution · n=63 gateway / 63 direct | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 52,911.8 | Valid | — |
| OpenAI → OpenAI | Streams sustained | 1,688 | Valid | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 1,163.5 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 1,170.9 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 1,172 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 1,181.8 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 1,287.1 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | 66 | Valid | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | 0.896 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 300 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | — | Not measured | — |
| OpenAI → OpenAI | Idle memory shape | — | Not measured | — |
| OpenAI → OpenAI | Idle memory static | 1 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | 0.606 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | 2,688 | concurrency 2; n=16,131 | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | 5,685 | concurrency 8; n=34,120 | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | 6,095 | concurrency 32; n=36,576 | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | 6,138 | concurrency 64; n=36,832 | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | 6,138 | concurrency 64; n=36,832 | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 6,138 | concurrency 64; n=36,832 | Unbounded |