# charts — OpenAI → OpenAI

Independent historical runs against a local mock. Compare only compatible configurations. Use the API for other protocol pairs.

## agentgateway

Build: ghcr.io/agentgateway/agentgateway:v1.7.0@sha256:4562330cd6e580bfac2de5b8905fee608dcf3fff6f1f4900744d27bdddb3b41b. Measured: 2026-11-10T17:39:19Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 216 | valid; gateway_c1_samples=24618; direct_c1_samples=233879 | — |
| openai → openai | added_latency_p99_us | 251 | valid; gateway_c1_samples=24618; direct_c1_samples=233879 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-agentgateway-fa351efdcee949e977dd1d4331440512b3e913a74b878f4bd1b19a7c7ca2900c)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-agentgateway-fa351efdcee949e977dd1d4331440512b3e913a74b878f4bd1b19a7c7ca2900c/raw.json)

## aisix

Build: ghcr.io/api7/aisix:1.9.0@sha256:dbe230bdaa4f353f7bf99d5437ab11f8b67435afabb217cf371c441ede43270a. Measured: 2026-11-10T17:39:23Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 146 | valid; gateway_c1_samples=35048; direct_c1_samples=262627 | — |
| openai → openai | added_latency_p99_us | 174 | valid; gateway_c1_samples=35048; direct_c1_samples=262627 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-aisix-ef0cff28391e0017f17c4b7c01e0171c62359468b787de48402bc3408df76915)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-aisix-ef0cff28391e0017f17c4b7c01e0171c62359468b787de48402bc3408df76915/raw.json)

## apisix

Build: apache/apisix:3.19.0-debian@sha256:7401ce7cfd56de1b9dcecfe7d7c7e5a681bc29222767fd28391995a0936eaa38. Measured: 2026-11-10T17:39:33Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 179 | valid; gateway_c1_samples=28247; direct_c1_samples=260315 | — |
| openai → openai | added_latency_p99_us | 340 | valid; gateway_c1_samples=28247; direct_c1_samples=260315 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-apisix-cc145fc7e19bd52b5c47dc4c3db32fb0511f656418b54e198d2b3cf922523d3f)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-apisix-cc145fc7e19bd52b5c47dc4c3db32fb0511f656418b54e198d2b3cf922523d3f/raw.json)

## bifrost

Build: maximhq/bifrost:v2.5.0@sha256:445bc49544bc64971b63581a93c6ca5716a03ccf27a58ced4c285cbafc133e78. Measured: 2026-11-11T03:17:59Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 658 | valid; gateway_c1_samples=8543; direct_c1_samples=269310 | — |
| openai → openai | added_latency_p99_us | 1057 | valid; gateway_c1_samples=8543; direct_c1_samples=269310 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-bifrost-c253d4668350f1793f587661b425702d5448ca78d04ec9144589514b6064ebd7)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-bifrost-c253d4668350f1793f587661b425702d5448ca78d04ec9144589514b6064ebd7/raw.json)

## busbar

Build: getbusbar/busbar:1.5.5@sha256:b4cfbc7431946204aef5b1f627616e33c654a4b1e12a13758b62a9e6f71501d3. Measured: 2026-11-10T17:39:31Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 78 | valid; gateway_c1_samples=59266; direct_c1_samples=266330 | — |
| openai → openai | added_latency_p99_us | 89 | valid; gateway_c1_samples=59266; direct_c1_samples=266330 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-busbar-16d2da7bc28375e55402b153f0f49b1550d5556f6ebb71eda60b8c3c47ad8d5f)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-busbar-16d2da7bc28375e55402b153f0f49b1550d5556f6ebb71eda60b8c3c47ad8d5f/raw.json)

## gomodel

Build: enterpilot/gomodel:0.2.3@sha256:c50627e093ff88bff311e582a43d7de14e0be29d52cb333ecb3191eac7874f05. Measured: 2026-11-10T17:39:52Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 343 | valid; gateway_c1_samples=15837; direct_c1_samples=239533 | — |
| openai → openai | added_latency_p99_us | 565 | valid; gateway_c1_samples=15837; direct_c1_samples=239533 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-gomodel-f6e038780b29f86549f840f902ac96740154f0f0ef2a66a3fd9dcd4bc3b83bad)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-gomodel-f6e038780b29f86549f840f902ac96740154f0f0ef2a66a3fd9dcd4bc3b83bad/raw.json)

## helicone

Build: target/release/ai-gateway. Measured: 2026-11-10T17:39:37Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 256 | valid; gateway_c1_samples=21418; direct_c1_samples=260967 | — |
| openai → openai | added_latency_p99_us | 274 | valid; gateway_c1_samples=21418; direct_c1_samples=260967 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-helicone-4a55ff2dcc509c5e03f85bd92e36b949ac410653128ccc77d60019bb3aaf2558)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-helicone-4a55ff2dcc509c5e03f85bd92e36b949ac410653128ccc77d60019bb3aaf2558/raw.json)

## higress

Build: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:2.2.5@sha256:9846006cb3cf2dafa942deab0e1d004ed30e219fbc9580438f9e3c58e7096422. Measured: 2026-11-10T17:39:42Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 718 | valid; gateway_c1_samples=7996; direct_c1_samples=262593 | — |
| openai → openai | added_latency_p99_us | 943 | valid; gateway_c1_samples=7996; direct_c1_samples=262593 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-higress-558a1502d4a7ea1f2b74c745739723084c953a3ae5dcf16760a2694a92c21270)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-higress-558a1502d4a7ea1f2b74c745739723084c953a3ae5dcf16760a2694a92c21270/raw.json)

## kong

Build: kong:3.9.4@sha256:2d572848be6fc58b73ba75edc84f028837bb7be998b3a8b8117fdb44a6892220. Measured: 2026-11-10T17:39:46Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 213 | valid; gateway_c1_samples=23861; direct_c1_samples=246038 | — |
| openai → openai | added_latency_p99_us | 410 | valid; gateway_c1_samples=23861; direct_c1_samples=246038 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-kong-d4b6d413ea59fd281dacf70be5988600ace51a15e8665c6c7dedcc87dd2cd2fb)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-kong-d4b6d413ea59fd281dacf70be5988600ace51a15e8665c6c7dedcc87dd2cd2fb/raw.json)

## litellm-python

Build: ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5. Measured: 2026-11-10T17:39:41Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 8512 | valid; gateway_c1_samples=701; direct_c1_samples=267297 | — |
| openai → openai | added_latency_p99_us | 9522 | valid; gateway_c1_samples=701; direct_c1_samples=267297 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-litellm-python-10bb57507b26d310b59f12956034785b615cd5338cdaf5cee0e587c350c53acf)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-litellm-python-10bb57507b26d310b59f12956034785b615cd5338cdaf5cee0e587c350c53acf/raw.json)

## litellm-rust

Build: src/litellm-rust/target/release/litellm-gateway. Measured: 2026-11-10T17:39:51Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 249 | valid; gateway_c1_samples=22021; direct_c1_samples=267021 | — |
| openai → openai | added_latency_p99_us | 279 | valid; gateway_c1_samples=22021; direct_c1_samples=267021 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-litellm-rust-69d8e6ed325740d9da7f0aaf31b815418c4801eb6549b2e2c68f768a92af3ad4)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-litellm-rust-69d8e6ed325740d9da7f0aaf31b815418c4801eb6549b2e2c68f768a92af3ad4/raw.json)

## one-api

Build: justsong/one-api:v0.6.10@sha256:af6b361630bdf408b86fce7aa8fc131b37ea5177ae866d121ba7c002e773e477. Measured: 2026-11-10T17:39:58Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 34017 | valid; gateway_c1_samples=102; direct_c1_samples=263027 | — |
| openai → openai | added_latency_p99_us | 589609 | valid; gateway_c1_samples=102; direct_c1_samples=263027 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-one-api-c9427cc70e392f48beb8e68d908f8924a7330748ac291dab1cd902b3f14c542f)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-one-api-c9427cc70e392f48beb8e68d908f8924a7330748ac291dab1cd902b3f14c542f/raw.json)

## plano

Build: katanemo/plano:0.4.39@sha256:35a84cb873ee5f16cded33eeda72736f90798962b7489d58650c0db7b546e9ac. Measured: 2026-11-10T17:40:07Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 198988 | valid; gateway_c1_samples=30; direct_c1_samples=266566 | — |
| openai → openai | added_latency_p99_us | 205966 | valid; gateway_c1_samples=30; direct_c1_samples=266566 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-plano-0a4ce97da36f11e7c407eef9280f69688b7ea1f007616709f0b914993cb8ba8e)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-plano-0a4ce97da36f11e7c407eef9280f69688b7ea1f007616709f0b914993cb8ba8e/raw.json)

## portkey

Build: portkeyai/gateway:1.15.2@sha256:9065e15701d5f90ae1728c57369f8cec137d8c7dffcd027a402a67e64f27b9d3. Measured: 2026-11-10T17:39:55Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 1196 | valid; gateway_c1_samples=4304; direct_c1_samples=246357 | — |
| openai → openai | added_latency_p99_us | 3902 | valid; gateway_c1_samples=4304; direct_c1_samples=246357 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-portkey-670dce412ba19689b30f84f64cb8b9c31fc07125970bd210617247c56c23bfcc)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-portkey-670dce412ba19689b30f84f64cb8b9c31fc07125970bd210617247c56c23bfcc/raw.json)

## tensorzero

Build: tensorzero/gateway:2026.6.0@sha256:9133135ccdaa23c0fffc2965f20b2a89b077ea3ad9a86a6f15b2a641bff67c89. Measured: 2026-11-10T17:40:04Z. Published: 2026-11-14T11:09:23.855Z. 3 days old at collection generation.

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| openai → openai | added_latency_p50_us | 40976 | valid; gateway_c1_samples=150; direct_c1_samples=253662 | — |
| openai → openai | added_latency_p99_us | 41966 | valid; gateway_c1_samples=150; direct_c1_samples=253662 | — |

[Run evidence and recorded definitions](https://onthebench.ai/gateways/runs/run-tensorzero-e1970318de8934076256f5f37dbf5d7fb6019f1dd5bb339bd959ac71a0a1a8e1)  ·  [Raw record](https://onthebench.ai/api/v1/runs/run-tensorzero-e1970318de8934076256f5f37dbf5d7fb6019f1dd5bb339bd959ac71a0a1a8e1/raw.json)

[Methodology](https://onthebench.ai/gateways/method)  ·  [Metric definitions](https://onthebench.ai/api/v1/definitions)
