LiteLLM · Rust benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Rust · recorded build: built from source; recorded binary src/litellm-rust/target/release/litellm-gateway. Measured: .

JSON · Markdown · CSV · Project source · Interactive dashboard

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

Published measurements

Ingress → upstreamMetricValueState / reasonp99 bound (ms)
Anthropic → Bedrock——Not declared: recorded note 1—
Cohere → Bedrock——Not declared: recorded note 1—
Bedrock → Bedrock——Not declared: recorded note 1—
Gemini → Bedrock——Not declared: recorded note 1—
OpenAI → Bedrock——Not served: probed and answered failed (HTTP 501)—
OpenAI Responses → Bedrock——Not declared: recorded note 1—
Bedrock → OpenAI——Not declared: recorded note 1—
OpenAI → OpenAIAdded latency p50 (µs)249Valid · n=22,021 gateway / 267,021 direct—
OpenAI → OpenAIAdded latency p99 (µs)279Valid · n=22,021 gateway / 267,021 direct—
OpenAI → OpenAIGateway p99 at concurrency 1 (µs)306Valid · n=22,021—
OpenAI → OpenAIDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=267,021—
OpenAI → OpenAICPU per request (µs)272.6Valid—
OpenAI → OpenAIRequests per CPU-second3,668.4Valid—
OpenAI → OpenAICost window concurrency8Valid—
OpenAI → OpenAICost window core utilisation0.679Valid—
OpenAI → OpenAICost window successful requests59,941Valid—
OpenAI → OpenAICost window throughput (req/s)9,989Valid—
OpenAI → OpenAICost window threads5Valid—
OpenAI → OpenAINon-voluntary context switches per request0Valid—
OpenAI → OpenAICost window major page faults0Valid—
OpenAI → OpenAIAdded time to first event p50 (µs)—Not measured—
OpenAI → OpenAIAdded time to first event p99 (µs)—Not measured—
OpenAI → OpenAIAdded inter-frame gap p50 (µs)—Not measured—
OpenAI → OpenAIAdded inter-frame gap p99 (µs)—Not measured—
OpenAI → OpenAIStreams sustained (frames/s)0Valid — measured failure: the gateway was offered stream load and sustained none of it—
OpenAI → OpenAIStreams sustained0Valid — measured failure: the gateway was offered stream load and sustained none of it—
OpenAI → OpenAIIdle RSS (MiB)12.74Valid—
OpenAI → OpenAISteady-state RSS (MiB)18.99Valid—
OpenAI → OpenAIRecovered RSS (MiB)19Valid—
OpenAI → OpenAIPeak RSS (MiB)19.88Valid—
OpenAI → OpenAIPeak RSS high-water mark (MiB)19.48Valid—
OpenAI → OpenAITime to plateau (s)—Not measured—
OpenAI → OpenAIMemory growth (MiB/min)-0.179Valid—
OpenAI → OpenAIMemory load duration (s)300Valid—
OpenAI → OpenAIMemory shape under load0Valid—
OpenAI → OpenAIIdle memory shape—Not measured—
OpenAI → OpenAIIdle memory static1Valid—
OpenAI → OpenAIIdle memory growth (MiB/min)0Valid—
OpenAI → OpenAIThroughput (req/s)8,404concurrency 4; n=50,4351 ms
OpenAI → OpenAIThroughput (req/s)10,049concurrency 16; n=60,3065 ms
OpenAI → OpenAIThroughput (req/s)10,049concurrency 16; n=60,30610 ms
OpenAI → OpenAIThroughput (req/s)10,049concurrency 16; n=60,30650 ms
OpenAI → OpenAIThroughput (req/s)10,049concurrency 16; n=60,306100 ms
OpenAI → OpenAIThroughput (req/s)10,049concurrency 16; n=60,306Unbounded
Anthropic → OpenAI——Not declared: recorded note 1—
OpenAI Responses → OpenAI——Not declared: recorded note 1—
Gemini → OpenAI——Not declared: recorded note 1—
Cohere → OpenAI——Not declared: recorded note 1—
Gemini → OpenAI Responses——Not declared: recorded note 1—
Cohere → OpenAI Responses——Not declared: recorded note 1—
Bedrock → OpenAI Responses——Not declared: recorded note 1—
OpenAI Responses → OpenAI ResponsesAdded latency p50 (µs)267Valid · n=20,661 gateway / 267,569 direct—
OpenAI Responses → OpenAI ResponsesAdded latency p99 (µs)298Valid · n=20,661 gateway / 267,569 direct—
OpenAI Responses → OpenAI ResponsesGateway p99 at concurrency 1 (µs)325Valid · n=20,661—
OpenAI Responses → OpenAI ResponsesDirect-to-mock p99 at concurrency 1 (µs)27Valid · n=267,569—
OpenAI Responses → OpenAI ResponsesCPU per request (µs)287.7Valid—
OpenAI Responses → OpenAI ResponsesRequests per CPU-second3,475.3Valid—
OpenAI Responses → OpenAI ResponsesCost window concurrency8Valid—
OpenAI Responses → OpenAI ResponsesCost window core utilisation0.734Valid—
OpenAI Responses → OpenAI ResponsesCost window successful requests61,374Valid—
OpenAI Responses → OpenAI ResponsesCost window throughput (req/s)10,227Valid—
OpenAI Responses → OpenAI ResponsesCost window threads5Valid—
OpenAI Responses → OpenAI ResponsesNon-voluntary context switches per request0Valid—
OpenAI Responses → OpenAI ResponsesCost window major page faults0Valid—
OpenAI Responses → OpenAI ResponsesAdded time to first event p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded time to first event p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p50 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesAdded inter-frame gap p99 (µs)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained (frames/s)—Untestable—
OpenAI Responses → OpenAI ResponsesStreams sustained—Untestable—
OpenAI Responses → OpenAI ResponsesIdle RSS (MiB)12.74Valid—
OpenAI Responses → OpenAI ResponsesSteady-state RSS (MiB)19.47Valid—
OpenAI Responses → OpenAI ResponsesRecovered RSS (MiB)19.29Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS (MiB)19.5Valid—
OpenAI Responses → OpenAI ResponsesPeak RSS high-water mark (MiB)19.5Valid—
OpenAI Responses → OpenAI ResponsesTime to plateau (s)66Valid—
OpenAI Responses → OpenAI ResponsesMemory growth (MiB/min)0.028Valid—
OpenAI Responses → OpenAI ResponsesMemory load duration (s)300Valid—
OpenAI Responses → OpenAI ResponsesMemory shape under load—Not measured—
OpenAI Responses → OpenAI ResponsesIdle memory shape—Not measured—
OpenAI Responses → OpenAI ResponsesIdle memory static1Valid—
OpenAI Responses → OpenAI ResponsesIdle memory growth (MiB/min)0Valid—
OpenAI Responses → OpenAI ResponsesThroughput (req/s)8,050concurrency 4; n=48,3091 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)10,191concurrency 16; n=61,1575 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)10,191concurrency 16; n=61,15710 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)10,191concurrency 16; n=61,15750 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)10,191concurrency 16; n=61,157100 ms
OpenAI Responses → OpenAI ResponsesThroughput (req/s)10,191concurrency 16; n=61,157Unbounded
OpenAI → OpenAI Responses——Not declared: recorded note 1—
Anthropic → OpenAI Responses——Not declared: recorded note 1—
OpenAI → AnthropicAdded latency p50 (µs)253Valid · n=21,651 gateway / 262,942 direct—
OpenAI → AnthropicAdded latency p99 (µs)283Valid · n=21,651 gateway / 262,942 direct—
OpenAI → AnthropicGateway p99 at concurrency 1 (µs)311Valid · n=21,651—
OpenAI → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=262,942—
OpenAI → AnthropicCPU per request (µs)277.1Valid—
OpenAI → AnthropicRequests per CPU-second3,609.2Valid—
OpenAI → AnthropicCost window concurrency8Valid—
OpenAI → AnthropicCost window core utilisation0.697Valid—
OpenAI → AnthropicCost window successful requests60,490Valid—
OpenAI → AnthropicCost window throughput (req/s)10,080Valid—
OpenAI → AnthropicCost window threads5Valid—
OpenAI → AnthropicNon-voluntary context switches per request0Valid—
OpenAI → AnthropicCost window major page faults0Valid—
OpenAI → AnthropicAdded time to first event p50 (µs)—Not measured—
OpenAI → AnthropicAdded time to first event p99 (µs)—Not measured—
OpenAI → AnthropicAdded inter-frame gap p50 (µs)—Not measured—
OpenAI → AnthropicAdded inter-frame gap p99 (µs)—Not measured—
OpenAI → AnthropicStreams sustained (frames/s)0Valid — measured failure: the gateway was offered stream load and sustained none of it—
OpenAI → AnthropicStreams sustained0Valid — measured failure: the gateway was offered stream load and sustained none of it—
OpenAI → AnthropicIdle RSS (MiB)12.76Valid—
OpenAI → AnthropicSteady-state RSS (MiB)19.05Valid—
OpenAI → AnthropicRecovered RSS (MiB)18.98Valid—
OpenAI → AnthropicPeak RSS (MiB)19.66Valid—
OpenAI → AnthropicPeak RSS high-water mark (MiB)19.33Valid—
OpenAI → AnthropicTime to plateau (s)—Not measured—
OpenAI → AnthropicMemory growth (MiB/min)0.328Valid—
OpenAI → AnthropicMemory load duration (s)300Valid—
OpenAI → AnthropicMemory shape under load0Valid—
OpenAI → AnthropicIdle memory shape—Not measured—
OpenAI → AnthropicIdle memory static1Valid—
OpenAI → AnthropicIdle memory growth (MiB/min)0Valid—
OpenAI → AnthropicThroughput (req/s)8,572concurrency 4; n=51,4411 ms
OpenAI → AnthropicThroughput (req/s)10,082concurrency 16; n=60,5005 ms
OpenAI → AnthropicThroughput (req/s)10,082concurrency 16; n=60,50010 ms
OpenAI → AnthropicThroughput (req/s)10,082concurrency 16; n=60,50050 ms
OpenAI → AnthropicThroughput (req/s)10,082concurrency 16; n=60,500100 ms
OpenAI → AnthropicThroughput (req/s)10,082concurrency 16; n=60,500Unbounded
Cohere → Anthropic——Not declared: recorded note 1—
Bedrock → Anthropic——Not declared: recorded note 1—
Gemini → Anthropic——Not declared: recorded note 1—
Anthropic → AnthropicAdded latency p50 (µs)275Valid · n=20,084 gateway / 262,977 direct—
Anthropic → AnthropicAdded latency p99 (µs)304Valid · n=20,084 gateway / 262,977 direct—
Anthropic → AnthropicGateway p99 at concurrency 1 (µs)332Valid · n=20,084—
Anthropic → AnthropicDirect-to-mock p99 at concurrency 1 (µs)28Valid · n=262,977—
Anthropic → AnthropicCPU per request (µs)297.1Valid—
Anthropic → AnthropicRequests per CPU-second3,366Valid—
Anthropic → AnthropicCost window concurrency8Valid—
Anthropic → AnthropicCost window core utilisation0.706Valid—
Anthropic → AnthropicCost window successful requests57,188Valid—
Anthropic → AnthropicCost window throughput (req/s)9,530Valid—
Anthropic → AnthropicCost window threads5Valid—
Anthropic → AnthropicNon-voluntary context switches per request0Valid—
Anthropic → AnthropicCost window major page faults0Valid—
Anthropic → AnthropicAdded time to first event p50 (µs)406Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded time to first event p99 (µs)477Valid · n=100 gateway / 100 direct—
Anthropic → AnthropicAdded inter-frame gap p50 (µs)2Valid · n=63 gateway / 63 direct—
Anthropic → AnthropicAdded inter-frame gap p99 (µs)3Valid · n=63 gateway / 63 direct—
Anthropic → AnthropicStreams sustained (frames/s)25,279.9Valid—
Anthropic → AnthropicStreams sustained919Valid—
Anthropic → AnthropicIdle RSS (MiB)12.74Valid—
Anthropic → AnthropicSteady-state RSS (MiB)20.35Valid—
Anthropic → AnthropicRecovered RSS (MiB)20.21Valid—
Anthropic → AnthropicPeak RSS (MiB)21.01Valid—
Anthropic → AnthropicPeak RSS high-water mark (MiB)20.56Valid—
Anthropic → AnthropicTime to plateau (s)—Not measured—
Anthropic → AnthropicMemory growth (MiB/min)-0.011Valid—
Anthropic → AnthropicMemory load duration (s)300Valid—
Anthropic → AnthropicMemory shape under load0Valid—
Anthropic → AnthropicIdle memory shape—Not measured—
Anthropic → AnthropicIdle memory static1Valid—
Anthropic → AnthropicIdle memory growth (MiB/min)0Valid—
Anthropic → AnthropicThroughput (req/s)8,878concurrency 4; n=53,2761 ms
Anthropic → AnthropicThroughput (req/s)9,865concurrency 16; n=59,1965 ms
Anthropic → AnthropicThroughput (req/s)9,865concurrency 16; n=59,19610 ms
Anthropic → AnthropicThroughput (req/s)9,865concurrency 16; n=59,19650 ms
Anthropic → AnthropicThroughput (req/s)9,865concurrency 16; n=59,196100 ms
Anthropic → AnthropicThroughput (req/s)9,865concurrency 16; n=59,196Unbounded
OpenAI Responses → Anthropic——Not declared: recorded note 1—
Anthropic → Gemini——Not declared: recorded note 1—
Cohere → Gemini——Not declared: recorded note 1—
Gemini → Gemini——Not declared: recorded note 1—
Bedrock → Gemini——Not declared: recorded note 1—
OpenAI Responses → Gemini——Not declared: recorded note 1—
OpenAI → Gemini——Not declared: recorded note 1—
OpenAI Responses → Cohere——Not declared: recorded note 1—
OpenAI → Cohere——Not declared: recorded note 1—
Cohere → Cohere——Not declared: recorded note 1—
Gemini → Cohere——Not declared: recorded note 1—
Anthropic → Cohere——Not declared: recorded note 1—
Bedrock → Cohere——Not declared: recorded note 1—

Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

Ingress → upstreamCell stateProbe HTTP statusRecorded reasonRecorded detailEvidence scopeResponse excerptTested build
Anthropic → BedrockNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Cohere → BedrockNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Bedrock → BedrockNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Gemini → BedrockNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI → BedrockNot served501failedprobed and answered failed (HTTP 501)Recorded run evidence{"error":{"message":"unsupported by the rust path: unrecognized request parameter","type":"api_error","param":null,"code":501}}built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI Responses → BedrockNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Bedrock → OpenAINot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI → OpenAIServed———Recorded run evidence—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Anthropic → OpenAINot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI Responses → OpenAINot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Gemini → OpenAINot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Cohere → OpenAINot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Gemini → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Cohere → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Bedrock → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI Responses → OpenAI ResponsesServed———Recorded run evidence—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Anthropic → OpenAI ResponsesNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI → AnthropicServed———Recorded run evidence—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Cohere → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Bedrock → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Gemini → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Anthropic → AnthropicServed———Recorded run evidence—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI Responses → AnthropicNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Anthropic → GeminiNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Cohere → GeminiNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Gemini → GeminiNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Bedrock → GeminiNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI Responses → GeminiNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI → GeminiNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI Responses → CohereNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
OpenAI → CohereNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Cohere → CohereNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Gemini → CohereNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Anthropic → CohereNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Bedrock → CohereNot declared—recorded note 1recorded note 1recorded note 2—built from source; recorded binary src/litellm-rust/target/release/litellm-gateway
Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2
Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8
Load generator SHA-256
not recorded
net.core.somaxconn
65,535
net.ipv4.tcp_max_syn_backlog
65,535
Gateway build receipt
Binary name
litellm-gateway
Binary sha256
ae1fec119755a8a2ae5b6a9d56225c34205ce7a54548cb11696b03b2557ce01b
Cargo lock sha256
ae9550da9f00c3ffb10ac2974d7d33c3784902c702c74c6aa85f1b83f1efcb2d
Compiler
rustc 1.99.1 (58cf8f9c3 2026-10-13) binary: rustc commit-hash: 58cf8f9c35b355199727627d07ae34a4e2846d67 commit-date: 2026-10-13 host: aarch64-unknown-linux-gnu release: 1.99.1 LLVM version: 23.1.1
Source commit
b7d579fab2818968322ef96e5b8e98dc8178fafc
Source dirty
No

Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

Ingress → upstreamConcurrencyLanes by HTTP statusLanes with no response
OpenAI → OpenAI1HTTP 501: 10
OpenAI → Anthropic1HTTP 501: 10
Anthropic → Anthropic1HTTP 200: 10
Anthropic → Anthropic2HTTP 200: 20
Anthropic → Anthropic4HTTP 200: 40
Anthropic → Anthropic8HTTP 200: 80
Anthropic → Anthropic16HTTP 200: 160
Anthropic → Anthropic32HTTP 200: 320
Anthropic → Anthropic64HTTP 200: 640
Anthropic → Anthropic128HTTP 200: 1280
Anthropic → Anthropic256HTTP 200: 2560
Anthropic → Anthropic512HTTP 200: 5120
Anthropic → Anthropic1,024HTTP 200: 1,0240
Anthropic → Anthropic768HTTP 200: 7680
Anthropic → Anthropic896HTTP 200: 8960
Anthropic → Anthropic960HTTP 200: 9600
Anthropic → Anthropic928HTTP 200: 9280
Anthropic → Anthropic912HTTP 200: 9120
Anthropic → Anthropic920HTTP 200: 9200
Anthropic → Anthropic916HTTP 200: 9160
Anthropic → Anthropic918HTTP 200: 9180
Anthropic → Anthropic919HTTP 200: 9190
Anthropic → Anthropic919HTTP 200: 9190
Anthropic → Anthropic919HTTP 200: 9190

Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

  1. Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json
  2. Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

Evidence for the current measurements

Configuration recorded by the run

general_settings:
  master_key: "gwbench"
model_list:
  - model_name: bench-openai
    litellm_params:
      model: openai_like/gpt-4o-mini
      api_base: http://127.0.0.1:8000/v1
      api_key: dummy
  - model_name: bench-responses
    litellm_params:
      model: openai/gpt-4o-mini
      api_base: http://127.0.0.1:8000/v1
      api_key: dummy
  - model_name: bench-anthropic
    litellm_params:
      model: anthropic/claude-3-5-sonnet-20241022
      api_base: http://127.0.0.1:8000
      api_key: dummy
  - model_name: bench-bedrock
    litellm_params:
      model: bedrock/amazon.nova-micro-v1:0
      api_base: http://127.0.0.1:8000
      api_key: dummy
      aws_region_name: us-east-1

Compare a shared protocol cell

Historical measurements