# On the Bench · AI tool benchmarks

Published results are available below.  [Browse all data and evidence](https://onthebench.ai/gateways/data) .

Comparison warning: Default rows use each gateway’s representative protocol cell, which can differ between gateways. This is not a like-for-like ranking. Select the same ingress and upstream protocols and check the  [comparison evidence](https://onthebench.ai/gateways/data)  before naming a winner.

  

## Reproducible, honest benchmarks for the AI tooling you actually run.

 

Every entrant is measured the same way, on neutral hardware: same box, same mock upstream, same load, same CPU pin.

 

Every number regenerates from committed JSON. Fully open source, re-runnable by anyone.

 

Built and operated by the Busbar team, who are also one of the entrants. Same harness for everyone. A failure is our bug until proven the gateway's.

 

[### Gateways

15 self-hostable AI gateways, measured for overhead, throughput, streaming, and protocol translation.

See the results →](https://onthebench.ai/gateways) 

### Models

Coming soon.

 

[Benchmark repository on GitHub](https://github.com/GetBusbar/benchmarking)  ·  [How we measure](https://onthebench.ai/gateways/method)

  

## Published OpenAI → OpenAI measurements

Gateway overhead against a local mock. Independent runs; compare configurations before drawing ratios.

| Gateway | Added p99 (µs) | State | Measured at |
| --- | --- | --- | --- |
| [agentgateway](https://onthebench.ai/gateways/agentgateway) | 251 | valid | 2026-11-10T17:39:19Z |
| [aisix](https://onthebench.ai/gateways/aisix) | 174 | valid | 2026-11-10T17:39:23Z |
| [apisix](https://onthebench.ai/gateways/apisix) | 340 | valid | 2026-11-10T17:39:33Z |
| [bifrost](https://onthebench.ai/gateways/bifrost) | 1057 | valid | 2026-11-11T03:17:59Z |
| [busbar](https://onthebench.ai/gateways/busbar) | 89 | valid | 2026-11-10T17:39:31Z |
| [gomodel](https://onthebench.ai/gateways/gomodel) | 565 | valid | 2026-11-10T17:39:52Z |
| [helicone](https://onthebench.ai/gateways/helicone) | 274 | valid | 2026-11-10T17:39:37Z |
| [higress](https://onthebench.ai/gateways/higress) | 943 | valid | 2026-11-10T17:39:42Z |
| [kong](https://onthebench.ai/gateways/kong) | 410 | valid | 2026-11-10T17:39:46Z |
| [litellm-python](https://onthebench.ai/gateways/litellm-python) | 9522 | valid | 2026-11-10T17:39:41Z |
| [litellm-rust](https://onthebench.ai/gateways/litellm-rust) | 279 | valid | 2026-11-10T17:39:51Z |
| [one-api](https://onthebench.ai/gateways/one-api) | 589609 | valid | 2026-11-10T17:39:58Z |
| [plano](https://onthebench.ai/gateways/plano) | 205966 | valid | 2026-11-10T17:40:07Z |
| [portkey](https://onthebench.ai/gateways/portkey) | 3902 | valid | 2026-11-10T17:39:55Z |
| [tensorzero](https://onthebench.ai/gateways/tensorzero) | 41966 | valid | 2026-11-10T17:40:04Z |
