Charts · Gateways · On the Bench · AI tool benchmarks
Every chart here is drawn from the same numbers the tables rank, on the cell and tail-latency bound last chosen on the table tabs; the line under each chart names them. Bars are coloured by implementation language (key above the chart), a neutral property, never by rank or brand.
- Rust
- Other
- Go
- Node
- Python
| Rank | Gateway | CPU per request |
|---|---|---|
| 1 | Busbar | 69 µs |
| 2 | AISIX (api7) | 153 µs |
| 3 | APISIX | 186 µs |
| 4 | agentgateway | 189 µs |
| 5 | Kong | 221 µs |
| 6 | Helicone | 259 µs |
| 7 | LiteLLM · Rust | 273 µs |
| 8 | GoModel | 439 µs |
| 9 | TensorZero | 459 µs |
| 10 | Higress | 709 µs |
| 11 | Bifrost | 865 µs |
| 12 | Portkey | 1.5 ms |
| 13 | LiteLLM · Python | 8.6 ms |
| 14 | Plano | 182.2 ms |
Microseconds of gateway CPU per completed request, measured at one concurrency held identical for every gateway. Lower is better. Read on each gateway's own representative same-dialect cell, as last chosen on the table tabs. Drawn on a log scale: this metric spans orders of magnitude, and a linear axis would collapse most of the field into the width of a line.
Not shown (no value on the chosen cell): One-API.