APISIX benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Other · recorded build: apache/apisix:3.19.0-debian. Measured: .

JSON · Markdown · CSV · Project source · Interactive dashboard

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

Published measurements

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
Bedrock → OpenAI——Not declared: recorded note 1—
OpenAI → OpenAIAdded latency p50 (µs)179Valid · n=28,247 gateway / 260,315 direct—
OpenAI → OpenAIAdded latency p99 (µs)340Valid · n=28,247 gateway / 260,315 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)368Valid · n=28,247—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=260,315—
OpenAI → OpenAICPU per request (µs)185.7Valid—
OpenAI → OpenAIRequests per CPU-second5,386.2Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.711Valid—
OpenAI → OpenAICost window successful requests92,266Valid—
OpenAI → OpenAICost window throughput (req/s)15,375Valid—
OpenAI → OpenAICost window threads12Valid—
OpenAI → OpenAINon-voluntary context switches per request0.000455Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)10,584Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded time to first event p99 (µs)11,228Valid · n=100 gateway / 100 direct—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)8,877Valid · n=63 gateway / 63 direct—
OpenAI → OpenAIStreams sustained (frames/s)63,882.4Valid—
OpenAI → OpenAIStreams sustained1,470Valid—
OpenAI → OpenAIIdle RSS (MiB)191.2Valid—
OpenAI → OpenAISteady-state RSS (MiB)228.2Valid—
OpenAI → OpenAIRecovered RSS (MiB)228.4Valid—
OpenAI → OpenAIPeak RSS (MiB)228.4Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)233.2Valid—
OpenAI → OpenAITime to plateau (s)66Valid—
OpenAI → OpenAIMemory growth (MiB/min)0.05Valid—
OpenAI → OpenAIMemory load duration (s)300Valid—
OpenAI → OpenAIMemory shape under load—Not measured—
OpenAI → OpenAIIdle memory shape0Valid—
OpenAI → OpenAIIdle memory static0Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)-0.464Valid—
OpenAI → OpenAIThroughput (req/s)12,187concurrency 4; n=73,1361 ms
OpenAI → OpenAIThroughput (req/s)23,089concurrency 32; n=138,5505 ms
OpenAI → OpenAIThroughput (req/s)25,474concurrency 64; n=152,85110 ms
OpenAI → OpenAIThroughput (req/s)26,590concurrency 256; n=159,56050 ms
OpenAI → OpenAIThroughput (req/s)26,590concurrency 256; n=159,560100 ms
OpenAI → OpenAIThroughput (req/s)26,590concurrency 256; n=159,560Unbounded
Anthropic → OpenAIAdded latency p50 (µs)193Valid · n=26,470 gateway / 257,076 direct—
Anthropic → OpenAIAdded latency p99 (µs)373Valid · n=26,470 gateway / 257,076 direct—
Anthropic → OpenAIGateway p99 at concurrency 1 (µs)402Valid · n=26,470—
Anthropic → OpenAIDirect-to-mock p99 at concurrency 1 (µs)29Valid · n=257,076—
Anthropic → OpenAICPU per request (µs)198.4Valid—
Anthropic → OpenAIRequests per CPU-second5,039.2Valid—
Anthropic → OpenAICost window concurrency8Valid—
Anthropic → OpenAICost window core utilisation0.716Valid—
Anthropic → OpenAICost window successful requests86,876Valid—
Anthropic → OpenAICost window throughput (req/s)14,477Valid—
Anthropic → OpenAICost window threads12Valid—
Anthropic → OpenAINon-voluntary context switches per request0.000576Valid—
Anthropic → OpenAICost window major page faults0Valid—
Anthropic → OpenAIAdded time to first event p50 (µs)10,646Valid · n=100 gateway / 100 direct—
Anthropic → OpenAIAdded time to first event p99 (µs)11,159Valid · n=100 gateway / 100 direct—
Anthropic → OpenAIAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → OpenAIAdded inter-frame gap p99 (µs)9,055Valid · n=63 gateway / 63 direct—
Anthropic → OpenAIStreams sustained (frames/s)77,198.5Valid—
Anthropic → OpenAIStreams sustained1,919Valid—
Anthropic → OpenAIIdle RSS (MiB)190.6Valid—
Anthropic → OpenAISteady-state RSS (MiB)224.6Valid—
Anthropic → OpenAIRecovered RSS (MiB)224.7Valid—
Anthropic → OpenAIPeak RSS (MiB)224.8Valid—
Anthropic → OpenAIPeak RSS high-water mark (MiB)229.6Valid—
Anthropic → OpenAITime to plateau (s)66Valid—
Anthropic → OpenAIMemory growth (MiB/min)0.026Valid—
Anthropic → OpenAIMemory load duration (s)300Valid—
Anthropic → OpenAIMemory shape under load—Not measured—
Anthropic → OpenAIIdle memory shape0Valid—
Anthropic → OpenAIIdle memory static0Valid—
Anthropic → OpenAIIdle memory growth (MiB/min)-0.452Valid—
Anthropic → OpenAIThroughput (req/s)11,298concurrency 4; n=67,8011 ms
Anthropic → OpenAIThroughput (req/s)21,698concurrency 32; n=130,2055 ms
Anthropic → OpenAIThroughput (req/s)24,156concurrency 64; n=144,94310 ms
Anthropic → OpenAIThroughput (req/s)25,454concurrency 256; n=152,73750 ms
Anthropic → OpenAIThroughput (req/s)25,454concurrency 256; n=152,737100 ms
Anthropic → OpenAIThroughput (req/s)25,454concurrency 256; n=152,737Unbounded
Gemini → OpenAI——Not declared: recorded note 1—
OpenAI Responses → OpenAI——Not declared: recorded note 1—
Cohere → OpenAI——Not declared: recorded note 1—
Gemini → Gemini——Not declared: recorded note 1—
OpenAI Responses → Gemini——Not declared: recorded note 1—
Bedrock → Gemini——Not declared: recorded note 1—
OpenAI → Gemini——Not declared: recorded note 1—
Cohere → Gemini——Not declared: recorded note 1—
Anthropic → Gemini——Not declared: recorded note 1—
Anthropic → Bedrock——Not declared: recorded note 1—
OpenAI → Bedrock——Not declared: recorded note 1—
Gemini → Bedrock——Not declared: recorded note 1—
Bedrock → BedrockAdded latency p50 (µs)234Valid · n=21,813 gateway / 263,299 direct—
Bedrock → BedrockAdded latency p99 (µs)463Valid · n=21,813 gateway / 263,299 direct—
Bedrock → BedrockGateway p99 at concurrency 1 (µs)491Valid · n=21,813—
Bedrock → BedrockDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=263,299—
Bedrock → BedrockCPU per request (µs)244.2Valid—
Bedrock → BedrockRequests per CPU-second4,095.4Valid—
Bedrock → BedrockCost window concurrency8Valid—
Bedrock → BedrockCost window core utilisation0.749Valid—
Bedrock → BedrockCost window successful requests73,840Valid—
Bedrock → BedrockCost window throughput (req/s)12,305Valid—
Bedrock → BedrockCost window threads12Valid—
Bedrock → BedrockNon-voluntary context switches per request0.000637Valid—
Bedrock → BedrockCost window major page faults0Valid—
Bedrock → BedrockAdded time to first event p50 (µs)—Untestable—
Bedrock → BedrockAdded time to first event p99 (µs)—Untestable—
Bedrock → BedrockAdded inter-frame gap p50 (µs)—Untestable—
Bedrock → BedrockAdded inter-frame gap p99 (µs)—Untestable—
Bedrock → BedrockStreams sustained (frames/s)—Untestable—
Bedrock → BedrockStreams sustained—Untestable—
Bedrock → BedrockIdle RSS (MiB)190.5Valid—
Bedrock → BedrockSteady-state RSS (MiB)238.9Valid—
Bedrock → BedrockRecovered RSS (MiB)239.4Valid—
Bedrock → BedrockPeak RSS (MiB)240.4Valid—
Bedrock → BedrockPeak RSS high-water mark (MiB)244.8Valid—
Bedrock → BedrockTime to plateau (s)—Not measured—
Bedrock → BedrockMemory growth (MiB/min)0.375Valid—
Bedrock → BedrockMemory load duration (s)300Valid—
Bedrock → BedrockMemory shape under load0Valid—
Bedrock → BedrockIdle memory shape0Valid—
Bedrock → BedrockIdle memory static0Valid—
Bedrock → BedrockIdle memory growth (MiB/min)-0.458Valid—
Bedrock → BedrockThroughput (req/s)6,434concurrency 2; n=38,6111 ms
Bedrock → BedrockThroughput (req/s)18,244concurrency 32; n=109,4755 ms
Bedrock → BedrockThroughput (req/s)19,877concurrency 64; n=119,27110 ms
Bedrock → BedrockThroughput (req/s)20,783concurrency 256; n=124,71350 ms
Bedrock → BedrockThroughput (req/s)20,783concurrency 256; n=124,713100 ms
Bedrock → BedrockThroughput (req/s)20,783concurrency 256; n=124,713Unbounded
Cohere → Bedrock——Not declared: recorded note 1—
OpenAI Responses → Bedrock——Not declared: recorded note 1—
OpenAI → Cohere——Not declared: recorded note 1—
OpenAI Responses → Cohere——Not declared: recorded note 1—
Bedrock → Cohere——Not declared: recorded note 1—
Gemini → Cohere——Not declared: recorded note 1—
Cohere → Cohere——Not declared: recorded note 1—
Anthropic → Cohere——Not declared: recorded note 1—
Cohere → Anthropic——Not declared: recorded note 1—
Anthropic → AnthropicAdded latency p50 (µs)181Valid · n=28,060 gateway / 258,354 direct—
Anthropic → AnthropicAdded latency p99 (µs)340Valid · n=28,060 gateway / 258,354 direct—
Anthropic → AnthropicGateway p99 at concurrency 1 (µs)368Valid · n=28,060—
Anthropic → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=258,354—
Anthropic → AnthropicCPU per request (µs)185Valid—
Anthropic → AnthropicRequests per CPU-second5,405.8Valid—
Anthropic → AnthropicCost window concurrency8Valid—
Anthropic → AnthropicCost window core utilisation0.681Valid—
Anthropic → AnthropicCost window successful requests88,601Valid—
Anthropic → AnthropicCost window throughput (req/s)14,764Valid—
Anthropic → AnthropicCost window threads12Valid—
Anthropic → AnthropicNon-voluntary context switches per request0.000372Valid—
Anthropic → AnthropicCost window major page faults0Valid—
Anthropic → AnthropicAdded time to first event p50 (µs)10,516Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded time to first event p99 (µs)11,171Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded inter-frame gap p50 (µs)—Below resolution · n=63 gateway / 63 direct—
Anthropic → AnthropicAdded inter-frame gap p99 (µs)8,998Valid · n=63 gateway / 63 direct—
Anthropic → AnthropicStreams sustained (frames/s)80,658.8Valid—
Anthropic → AnthropicStreams sustained1,844Valid—
Anthropic → AnthropicIdle RSS (MiB)193.1Valid—
Anthropic → AnthropicSteady-state RSS (MiB)230Valid—
Anthropic → AnthropicRecovered RSS (MiB)230.2Valid—
Anthropic → AnthropicPeak RSS (MiB)230.4Valid—
Anthropic → AnthropicPeak RSS high-water mark (MiB)235.2Valid—
Anthropic → AnthropicTime to plateau (s)66Valid—
Anthropic → AnthropicMemory growth (MiB/min)0.164Valid—
Anthropic → AnthropicMemory load duration (s)300Valid—
Anthropic → AnthropicMemory shape under load—Not measured—
Anthropic → AnthropicIdle memory shape0Valid—
Anthropic → AnthropicIdle memory static0Valid—
Anthropic → AnthropicIdle memory growth (MiB/min)-0.455Valid—
Anthropic → AnthropicThroughput (req/s)12,499concurrency 4; n=75,0051 ms
Anthropic → AnthropicThroughput (req/s)23,800concurrency 32; n=142,8175 ms
Anthropic → AnthropicThroughput (req/s)26,599concurrency 64; n=159,60910 ms
Anthropic → AnthropicThroughput (req/s)27,850concurrency 256; n=167,11650 ms
Anthropic → AnthropicThroughput (req/s)27,850concurrency 256; n=167,116100 ms
Anthropic → AnthropicThroughput (req/s)27,850concurrency 256; n=167,116Unbounded
OpenAI Responses → Anthropic——Not declared: recorded note 1—
OpenAI → Anthropic——Not declared: recorded note 1—
Bedrock → Anthropic——Not declared: recorded note 1—
Gemini → Anthropic——Not declared: recorded note 1—
Gemini → OpenAI Responses——Not declared: recorded note 1—
Anthropic → OpenAI Responses——Not declared: recorded note 1—
OpenAI → OpenAI Responses——Not declared: recorded note 1—
Cohere → OpenAI Responses——Not declared: recorded note 1—
Bedrock → OpenAI Responses——Not declared: recorded note 1—
OpenAI Responses → OpenAI ResponsesAdded latency p50 (µs)189Valid · n=26,997 gateway / 261,509 direct—
OpenAI Responses → OpenAI ResponsesAdded latency p99 (µs)358Valid · n=26,997 gateway / 261,509 direct—
OpenAI Responses → OpenAI ResponsesGateway p99 at concurrency 1 (µs)386Valid · n=26,997—
OpenAI Responses → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=261,509—
OpenAI Responses → OpenAI ResponsesCPU per request (µs)189.5Valid—
OpenAI Responses → OpenAI ResponsesRequests per CPU-second5,278.1Valid—
OpenAI Responses → OpenAI ResponsesCost window concurrency8Valid—
OpenAI Responses → OpenAI ResponsesCost window core utilisation0.757Valid—
OpenAI Responses → OpenAI ResponsesCost window successful requests96,219Valid—
OpenAI Responses → OpenAI ResponsesCost window throughput (req/s)16,034Valid—
OpenAI Responses → OpenAI ResponsesCost window threads12Valid—
OpenAI Responses → OpenAI ResponsesNon-voluntary context switches per request0.000437Valid—
OpenAI Responses → OpenAI ResponsesCost window major page faults0Valid—
OpenAI Responses → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained—Untestable—
OpenAI Responses → OpenAI ResponsesIdle RSS (MiB)192.4Valid—
OpenAI Responses → OpenAI ResponsesSteady-state RSS (MiB)228.3Valid—
OpenAI Responses → OpenAI ResponsesRecovered RSS (MiB)228.3Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS (MiB)228.5Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS high-water mark (MiB)233Valid—
OpenAI Responses → OpenAI ResponsesTime to plateau (s)66Valid—
OpenAI Responses → OpenAI ResponsesMemory growth (MiB/min)0.018Valid—
OpenAI Responses → OpenAI ResponsesMemory load duration (s)300Valid—
OpenAI Responses → OpenAI ResponsesMemory shape under load—Not measured—
OpenAI Responses → OpenAI ResponsesIdle memory shape0Valid—
OpenAI Responses → OpenAI ResponsesIdle memory static0Valid—
OpenAI Responses → OpenAI ResponsesIdle memory growth (MiB/min)-0.464Valid—
OpenAI Responses → OpenAI ResponsesThroughput (req/s)11,636concurrency 4; n=69,8301 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)24,247concurrency 32; n=145,4905 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)25,825concurrency 64; n=154,95110 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)28,066concurrency 128; n=168,42250 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)28,066concurrency 128; n=168,422100 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)28,066concurrency 128; n=168,422Unbounded

Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

Ingress → upstreamCell stateProbe HTTP statusRecorded reasonRecorded detailEvidence scopeResponse excerptTested build
Bedrock → OpenAINot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI → OpenAIServed———Recorded run evidence—apache/apisix:3.19.0-debian
Anthropic → OpenAIServed———Recorded run evidence—apache/apisix:3.19.0-debian
Gemini → OpenAINot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI Responses → OpenAINot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Cohere → OpenAINot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Gemini → GeminiNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI Responses → GeminiNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Bedrock → GeminiNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI → GeminiNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Cohere → GeminiNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Anthropic → GeminiNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Anthropic → BedrockNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI → BedrockNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Gemini → BedrockNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Bedrock → BedrockServed———Recorded run evidence—apache/apisix:3.19.0-debian
Cohere → BedrockNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI Responses → BedrockNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI → CohereNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI Responses → CohereNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Bedrock → CohereNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Gemini → CohereNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Cohere → CohereNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Anthropic → CohereNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Cohere → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Anthropic → AnthropicServed———Recorded run evidence—apache/apisix:3.19.0-debian
OpenAI Responses → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Bedrock → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Gemini → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Gemini → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Anthropic → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Cohere → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
Bedrock → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—apache/apisix:3.19.0-debian
OpenAI Responses → OpenAI ResponsesServed———Recorded run evidence—apache/apisix:3.19.0-debian
Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2
Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8
Load generator SHA-256
not recorded
net.core.somaxconn
65,535
net.ipv4.tcp_max_syn_backlog
65,535

Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

Ingress → upstreamConcurrencyLanes by HTTP statusLanes with no response
OpenAI → OpenAI1HTTP 200: 10
OpenAI → OpenAI2HTTP 200: 20
OpenAI → OpenAI4HTTP 200: 40
OpenAI → OpenAI8HTTP 200: 80
OpenAI → OpenAI16HTTP 200: 160
OpenAI → OpenAI32HTTP 200: 320
OpenAI → OpenAI64HTTP 200: 640
OpenAI → OpenAI128HTTP 200: 1280
OpenAI → OpenAI256HTTP 200: 2560
OpenAI → OpenAI512HTTP 200: 5120
OpenAI → OpenAI1,024HTTP 200: 1,0240
OpenAI → OpenAI2,048HTTP 200: 2,0480
OpenAI → OpenAI1,536HTTP 200: 1,5360
OpenAI → OpenAI1,280HTTP 200: 1,2800
OpenAI → OpenAI1,408HTTP 200: 1,4080
OpenAI → OpenAI1,472HTTP 200: 1,4720
OpenAI → OpenAI1,440HTTP 200: 1,4400
OpenAI → OpenAI1,456HTTP 200: 1,4560
OpenAI → OpenAI1,464HTTP 200: 1,4640
OpenAI → OpenAI1,468HTTP 200: 1,4680
OpenAI → OpenAI1,470HTTP 200: 1,4700
OpenAI → OpenAI1,471HTTP 200: 1,4710
OpenAI → OpenAI1,470HTTP 200: 1,4700
OpenAI → OpenAI1,470HTTP 200: 1,4700
Anthropic → OpenAI1HTTP 200: 10
Anthropic → OpenAI2HTTP 200: 20
Anthropic → OpenAI4HTTP 200: 40
Anthropic → OpenAI8HTTP 200: 80
Anthropic → OpenAI16HTTP 200: 160
Anthropic → OpenAI32HTTP 200: 320
Anthropic → OpenAI64HTTP 200: 640
Anthropic → OpenAI128HTTP 200: 1280
Anthropic → OpenAI256HTTP 200: 2560
Anthropic → OpenAI512HTTP 200: 5120
Anthropic → OpenAI1,024HTTP 200: 1,0240
Anthropic → OpenAI2,048HTTP 200: 2,0480
Anthropic → OpenAI1,536HTTP 200: 1,5360
Anthropic → OpenAI1,792HTTP 200: 1,7920
Anthropic → OpenAI1,920HTTP 200: 1,9200
Anthropic → OpenAI1,856HTTP 200: 1,8560
Anthropic → OpenAI1,888HTTP 200: 1,8880
Anthropic → OpenAI1,904HTTP 200: 1,9040
Anthropic → OpenAI1,912HTTP 200: 1,9120
Anthropic → OpenAI1,916HTTP 200: 1,9160
Anthropic → OpenAI1,918HTTP 200: 1,9180
Anthropic → OpenAI1,919HTTP 200: 1,9190
Anthropic → OpenAI1,919HTTP 200: 1,9190
Anthropic → OpenAI1,919HTTP 200: 1,9190
Anthropic → Anthropic1HTTP 200: 10
Anthropic → Anthropic2HTTP 200: 20
Anthropic → Anthropic4HTTP 200: 40
Anthropic → Anthropic8HTTP 200: 80
Anthropic → Anthropic16HTTP 200: 160
Anthropic → Anthropic32HTTP 200: 320
Anthropic → Anthropic64HTTP 200: 640
Anthropic → Anthropic128HTTP 200: 1280
Anthropic → Anthropic256HTTP 200: 2560
Anthropic → Anthropic512HTTP 200: 5120
Anthropic → Anthropic1,024HTTP 200: 1,0240
Anthropic → Anthropic2,048HTTP 200: 2,0480
Anthropic → Anthropic1,536HTTP 200: 1,5360
Anthropic → Anthropic1,792HTTP 200: 1,7920
Anthropic → Anthropic1,920HTTP 200: 1,9200
Anthropic → Anthropic1,856HTTP 200: 1,8560
Anthropic → Anthropic1,824HTTP 200: 1,8240
Anthropic → Anthropic1,840HTTP 200: 1,8400
Anthropic → Anthropic1,848HTTP 200: 1,8480
Anthropic → Anthropic1,844HTTP 200: 1,8440
Anthropic → Anthropic1,846HTTP 200: 1,8460
Anthropic → Anthropic1,845HTTP 200: 1,8450
Anthropic → Anthropic1,844HTTP 200: 1,8440
Anthropic → Anthropic1,844HTTP 200: 1,8440

Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

  1. Pinned APISIX 3.19.0 source 473043623c161966f3f6ef4e8f12d2d8667a12d2: ai-providers capability tables and ai-protocols/converters registry support four native chat protocol paths plus Anthropic Messages to OpenAI Chat Completions conversion. The Gemini provider uses an OpenAI-compatible endpoint, not native Gemini generateContent; provider brands are not protocol support. Five exact pairs passed local response and recorder checks. Evidence: results/diagnostics/refresh-configuration/apisix-protocol-preflight.json
  2. Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

Evidence for the current measurements

Configuration recorded by the run

# apisix.gen.yaml
routes:
  - id: ai-proxy-chat
    uri: /v1/chat/completions
    methods: [POST]
    plugins:
      ai-proxy:
        provider: openai-compatible
        auth: { header: { Authorization: "Bearer sk-fake-benchmark-key" } }
        options: { model: gpt-4o-mini }
        override: { endpoint: "http://127.0.0.1:8000" }
  - id: ai-proxy-responses
    uri: /v1/responses
    methods: [POST]
    plugins:
      ai-proxy:
        provider: openai-compatible
        auth: { header: { Authorization: "Bearer sk-fake-benchmark-key" } }
        options: { model: gpt-4o-mini }
        override: { endpoint: "http://127.0.0.1:8000" }
  - id: ai-proxy-messages
    uri: /v1/messages
    methods: [POST]
    plugins:
      ai-proxy:
        provider: anthropic
        auth: { header: { Authorization: "Bearer sk-fake-benchmark-key" } }
        options: { model: claude-3-5-sonnet-20241022 }
        override: { endpoint: "http://127.0.0.1:8000" }
  - id: ai-proxy-converse
    uri: /model/anthropic.claude-3-sonnet-20240229-v1:0/converse
    methods: [POST]
    plugins:
      ai-proxy:
        provider: bedrock
        provider_conf: { region: "us-east-1" }
        auth:
          aws:
            access_key_id: "AKIAMOCKACCESSKEY"
            secret_access_key: "mock-secret-access-key"
        options: { model: anthropic.claude-3-sonnet-20240229-v1:0 }
        override: { endpoint: "http://127.0.0.1:8000" }
  - id: anthropic-to-openai
    uri: /v1/messages
    priority: 10
    vars: [["http_x_bench_egress", "==", "openai"]]
    methods: [POST]
    plugins:
      ai-proxy:
        provider: openai-compatible
        auth: { header: { Authorization: "Bearer sk-fake-benchmark-key" } }
        options: { model: gpt-4o-mini }
        override: { endpoint: "http://127.0.0.1:8000" }
#END


# config.gen.yaml
deployment:
  role: data_plane
  role_data_plane:
    config_provider: yaml
apisix:
  node_listen:
    - 9080
nginx_config:
  worker_processes: 4

Compare a shared protocol cell

Historical measurements