LiteLLM · Rust benchmark measurements
Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.
Rust · recorded build: built from source; recorded binary src/litellm-rust/target/release/litellm-gateway. Measured: .
JSON · Markdown · CSV · Project source · Interactive dashboard
Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.
Published measurements
| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
|---|---|---|---|---|
| Anthropic → Bedrock | — | — | Not declared: recorded note 1 | — |
| Cohere → Bedrock | — | — | Not declared: recorded note 1 | — |
| Bedrock → Bedrock | — | — | Not declared: recorded note 1 | — |
| Gemini → Bedrock | — | — | Not declared: recorded note 1 | — |
| OpenAI → Bedrock | — | — | Not served: probed and answered failed (HTTP 501) | — |
| OpenAI Responses → Bedrock | — | — | Not declared: recorded note 1 | — |
| Bedrock → OpenAI | — | — | Not declared: recorded note 1 | — |
| OpenAI → OpenAI | Added latency p50 (µs) | 249 | Valid · n=22,021 gateway / 267,021 direct | — |
| OpenAI → OpenAI | Added latency p99 (µs) | 279 | Valid · n=22,021 gateway / 267,021 direct | — |
| OpenAI → OpenAI | Gateway p99 at concurrency 1 (µs) | 306 | Valid · n=22,021 | — |
| OpenAI → OpenAI | Direct-to-mock p99 at concurrency 1 (µs) | 27 | Valid · n=267,021 | — |
| OpenAI → OpenAI | CPU per request (µs) | 272.6 | Valid | — |
| OpenAI → OpenAI | Requests per CPU-second | 3,668.4 | Valid | — |
| OpenAI → OpenAI | Cost window concurrency | 8 | Valid | — |
| OpenAI → OpenAI | Cost window core utilisation | 0.679 | Valid | — |
| OpenAI → OpenAI | Cost window successful requests | 59,941 | Valid | — |
| OpenAI → OpenAI | Cost window throughput (req/s) | 9,989 | Valid | — |
| OpenAI → OpenAI | Cost window threads | 5 | Valid | — |
| OpenAI → OpenAI | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → OpenAI | Cost window major page faults | 0 | Valid | — |
| OpenAI → OpenAI | Added time to first event p50 (µs) | — | Not measured | — |
| OpenAI → OpenAI | Added time to first event p99 (µs) | — | Not measured | — |
| OpenAI → OpenAI | Added inter-frame gap p50 (µs) | — | Not measured | — |
| OpenAI → OpenAI | Added inter-frame gap p99 (µs) | — | Not measured | — |
| OpenAI → OpenAI | Streams sustained (frames/s) | 0 | Valid — measured failure: the gateway was offered stream load and sustained none of it | — |
| OpenAI → OpenAI | Streams sustained | 0 | Valid — measured failure: the gateway was offered stream load and sustained none of it | — |
| OpenAI → OpenAI | Idle RSS (MiB) | 12.74 | Valid | — |
| OpenAI → OpenAI | Steady-state RSS (MiB) | 18.99 | Valid | — |
| OpenAI → OpenAI | Recovered RSS (MiB) | 19 | Valid | — |
| OpenAI → OpenAI | Peak RSS (MiB) | 19.88 | Valid | — |
| OpenAI → OpenAI | Peak RSS high-water mark (MiB) | 19.48 | Valid | — |
| OpenAI → OpenAI | Time to plateau (s) | — | Not measured | — |
| OpenAI → OpenAI | Memory growth (MiB/min) | -0.179 | Valid | — |
| OpenAI → OpenAI | Memory load duration (s) | 300 | Valid | — |
| OpenAI → OpenAI | Memory shape under load | 0 | Valid | — |
| OpenAI → OpenAI | Idle memory shape | — | Not measured | — |
| OpenAI → OpenAI | Idle memory static | 1 | Valid | — |
| OpenAI → OpenAI | Idle memory growth (MiB/min) | 0 | Valid | — |
| OpenAI → OpenAI | Throughput (req/s) | 8,404 | concurrency 4; n=50,435 | 1 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | 5 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | 10 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | 50 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | 100 ms |
| OpenAI → OpenAI | Throughput (req/s) | 10,049 | concurrency 16; n=60,306 | Unbounded |
| Anthropic → OpenAI | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → OpenAI | — | — | Not declared: recorded note 1 | — |
| Gemini → OpenAI | — | — | Not declared: recorded note 1 | — |
| Cohere → OpenAI | — | — | Not declared: recorded note 1 | — |
| Gemini → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Cohere → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Bedrock → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → OpenAI Responses | Added latency p50 (µs) | 267 | Valid · n=20,661 gateway / 267,569 direct | — |
| OpenAI Responses → OpenAI Responses | Added latency p99 (µs) | 298 | Valid · n=20,661 gateway / 267,569 direct | — |
| OpenAI Responses → OpenAI Responses | Gateway p99 at concurrency 1 (µs) | 325 | Valid · n=20,661 | — |
| OpenAI Responses → OpenAI Responses | Direct-to-mock p99 at concurrency 1 (µs) | 27 | Valid · n=267,569 | — |
| OpenAI Responses → OpenAI Responses | CPU per request (µs) | 287.7 | Valid | — |
| OpenAI Responses → OpenAI Responses | Requests per CPU-second | 3,475.3 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window concurrency | 8 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window core utilisation | 0.734 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window successful requests | 61,374 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window throughput (req/s) | 10,227 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window threads | 5 | Valid | — |
| OpenAI Responses → OpenAI Responses | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI Responses → OpenAI Responses | Cost window major page faults | 0 | Valid | — |
| OpenAI Responses → OpenAI Responses | Added time to first event p50 (µs) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Added time to first event p99 (µs) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Added inter-frame gap p50 (µs) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Added inter-frame gap p99 (µs) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Streams sustained (frames/s) | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Streams sustained | — | Untestable | — |
| OpenAI Responses → OpenAI Responses | Idle RSS (MiB) | 12.74 | Valid | — |
| OpenAI Responses → OpenAI Responses | Steady-state RSS (MiB) | 19.47 | Valid | — |
| OpenAI Responses → OpenAI Responses | Recovered RSS (MiB) | 19.29 | Valid | — |
| OpenAI Responses → OpenAI Responses | Peak RSS (MiB) | 19.5 | Valid | — |
| OpenAI Responses → OpenAI Responses | Peak RSS high-water mark (MiB) | 19.5 | Valid | — |
| OpenAI Responses → OpenAI Responses | Time to plateau (s) | 66 | Valid | — |
| OpenAI Responses → OpenAI Responses | Memory growth (MiB/min) | 0.028 | Valid | — |
| OpenAI Responses → OpenAI Responses | Memory load duration (s) | 300 | Valid | — |
| OpenAI Responses → OpenAI Responses | Memory shape under load | — | Not measured | — |
| OpenAI Responses → OpenAI Responses | Idle memory shape | — | Not measured | — |
| OpenAI Responses → OpenAI Responses | Idle memory static | 1 | Valid | — |
| OpenAI Responses → OpenAI Responses | Idle memory growth (MiB/min) | 0 | Valid | — |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | 8,050 | concurrency 4; n=48,309 | 1 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | 10,191 | concurrency 16; n=61,157 | 5 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | 10,191 | concurrency 16; n=61,157 | 10 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | 10,191 | concurrency 16; n=61,157 | 50 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | 10,191 | concurrency 16; n=61,157 | 100 ms |
| OpenAI Responses → OpenAI Responses | Throughput (req/s) | 10,191 | concurrency 16; n=61,157 | Unbounded |
| OpenAI → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| Anthropic → OpenAI Responses | — | — | Not declared: recorded note 1 | — |
| OpenAI → Anthropic | Added latency p50 (µs) | 253 | Valid · n=21,651 gateway / 262,942 direct | — |
| OpenAI → Anthropic | Added latency p99 (µs) | 283 | Valid · n=21,651 gateway / 262,942 direct | — |
| OpenAI → Anthropic | Gateway p99 at concurrency 1 (µs) | 311 | Valid · n=21,651 | — |
| OpenAI → Anthropic | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=262,942 | — |
| OpenAI → Anthropic | CPU per request (µs) | 277.1 | Valid | — |
| OpenAI → Anthropic | Requests per CPU-second | 3,609.2 | Valid | — |
| OpenAI → Anthropic | Cost window concurrency | 8 | Valid | — |
| OpenAI → Anthropic | Cost window core utilisation | 0.697 | Valid | — |
| OpenAI → Anthropic | Cost window successful requests | 60,490 | Valid | — |
| OpenAI → Anthropic | Cost window throughput (req/s) | 10,080 | Valid | — |
| OpenAI → Anthropic | Cost window threads | 5 | Valid | — |
| OpenAI → Anthropic | Non-voluntary context switches per request | 0 | Valid | — |
| OpenAI → Anthropic | Cost window major page faults | 0 | Valid | — |
| OpenAI → Anthropic | Added time to first event p50 (µs) | — | Not measured | — |
| OpenAI → Anthropic | Added time to first event p99 (µs) | — | Not measured | — |
| OpenAI → Anthropic | Added inter-frame gap p50 (µs) | — | Not measured | — |
| OpenAI → Anthropic | Added inter-frame gap p99 (µs) | — | Not measured | — |
| OpenAI → Anthropic | Streams sustained (frames/s) | 0 | Valid — measured failure: the gateway was offered stream load and sustained none of it | — |
| OpenAI → Anthropic | Streams sustained | 0 | Valid — measured failure: the gateway was offered stream load and sustained none of it | — |
| OpenAI → Anthropic | Idle RSS (MiB) | 12.76 | Valid | — |
| OpenAI → Anthropic | Steady-state RSS (MiB) | 19.05 | Valid | — |
| OpenAI → Anthropic | Recovered RSS (MiB) | 18.98 | Valid | — |
| OpenAI → Anthropic | Peak RSS (MiB) | 19.66 | Valid | — |
| OpenAI → Anthropic | Peak RSS high-water mark (MiB) | 19.33 | Valid | — |
| OpenAI → Anthropic | Time to plateau (s) | — | Not measured | — |
| OpenAI → Anthropic | Memory growth (MiB/min) | 0.328 | Valid | — |
| OpenAI → Anthropic | Memory load duration (s) | 300 | Valid | — |
| OpenAI → Anthropic | Memory shape under load | 0 | Valid | — |
| OpenAI → Anthropic | Idle memory shape | — | Not measured | — |
| OpenAI → Anthropic | Idle memory static | 1 | Valid | — |
| OpenAI → Anthropic | Idle memory growth (MiB/min) | 0 | Valid | — |
| OpenAI → Anthropic | Throughput (req/s) | 8,572 | concurrency 4; n=51,441 | 1 ms |
| OpenAI → Anthropic | Throughput (req/s) | 10,082 | concurrency 16; n=60,500 | 5 ms |
| OpenAI → Anthropic | Throughput (req/s) | 10,082 | concurrency 16; n=60,500 | 10 ms |
| OpenAI → Anthropic | Throughput (req/s) | 10,082 | concurrency 16; n=60,500 | 50 ms |
| OpenAI → Anthropic | Throughput (req/s) | 10,082 | concurrency 16; n=60,500 | 100 ms |
| OpenAI → Anthropic | Throughput (req/s) | 10,082 | concurrency 16; n=60,500 | Unbounded |
| Cohere → Anthropic | — | — | Not declared: recorded note 1 | — |
| Bedrock → Anthropic | — | — | Not declared: recorded note 1 | — |
| Gemini → Anthropic | — | — | Not declared: recorded note 1 | — |
| Anthropic → Anthropic | Added latency p50 (µs) | 275 | Valid · n=20,084 gateway / 262,977 direct | — |
| Anthropic → Anthropic | Added latency p99 (µs) | 304 | Valid · n=20,084 gateway / 262,977 direct | — |
| Anthropic → Anthropic | Gateway p99 at concurrency 1 (µs) | 332 | Valid · n=20,084 | — |
| Anthropic → Anthropic | Direct-to-mock p99 at concurrency 1 (µs) | 28 | Valid · n=262,977 | — |
| Anthropic → Anthropic | CPU per request (µs) | 297.1 | Valid | — |
| Anthropic → Anthropic | Requests per CPU-second | 3,366 | Valid | — |
| Anthropic → Anthropic | Cost window concurrency | 8 | Valid | — |
| Anthropic → Anthropic | Cost window core utilisation | 0.706 | Valid | — |
| Anthropic → Anthropic | Cost window successful requests | 57,188 | Valid | — |
| Anthropic → Anthropic | Cost window throughput (req/s) | 9,530 | Valid | — |
| Anthropic → Anthropic | Cost window threads | 5 | Valid | — |
| Anthropic → Anthropic | Non-voluntary context switches per request | 0 | Valid | — |
| Anthropic → Anthropic | Cost window major page faults | 0 | Valid | — |
| Anthropic → Anthropic | Added time to first event p50 (µs) | 406 | Valid · n=100 gateway / 100 direct | — |
| Anthropic → Anthropic | Added time to first event p99 (µs) | 477 | Valid · n=100 gateway / 100 direct | — |
| Anthropic → Anthropic | Added inter-frame gap p50 (µs) | 2 | Valid · n=63 gateway / 63 direct | — |
| Anthropic → Anthropic | Added inter-frame gap p99 (µs) | 3 | Valid · n=63 gateway / 63 direct | — |
| Anthropic → Anthropic | Streams sustained (frames/s) | 25,279.9 | Valid | — |
| Anthropic → Anthropic | Streams sustained | 919 | Valid | — |
| Anthropic → Anthropic | Idle RSS (MiB) | 12.74 | Valid | — |
| Anthropic → Anthropic | Steady-state RSS (MiB) | 20.35 | Valid | — |
| Anthropic → Anthropic | Recovered RSS (MiB) | 20.21 | Valid | — |
| Anthropic → Anthropic | Peak RSS (MiB) | 21.01 | Valid | — |
| Anthropic → Anthropic | Peak RSS high-water mark (MiB) | 20.56 | Valid | — |
| Anthropic → Anthropic | Time to plateau (s) | — | Not measured | — |
| Anthropic → Anthropic | Memory growth (MiB/min) | -0.011 | Valid | — |
| Anthropic → Anthropic | Memory load duration (s) | 300 | Valid | — |
| Anthropic → Anthropic | Memory shape under load | 0 | Valid | — |
| Anthropic → Anthropic | Idle memory shape | — | Not measured | — |
| Anthropic → Anthropic | Idle memory static | 1 | Valid | — |
| Anthropic → Anthropic | Idle memory growth (MiB/min) | 0 | Valid | — |
| Anthropic → Anthropic | Throughput (req/s) | 8,878 | concurrency 4; n=53,276 | 1 ms |
| Anthropic → Anthropic | Throughput (req/s) | 9,865 | concurrency 16; n=59,196 | 5 ms |
| Anthropic → Anthropic | Throughput (req/s) | 9,865 | concurrency 16; n=59,196 | 10 ms |
| Anthropic → Anthropic | Throughput (req/s) | 9,865 | concurrency 16; n=59,196 | 50 ms |
| Anthropic → Anthropic | Throughput (req/s) | 9,865 | concurrency 16; n=59,196 | 100 ms |
| Anthropic → Anthropic | Throughput (req/s) | 9,865 | concurrency 16; n=59,196 | Unbounded |
| OpenAI Responses → Anthropic | — | — | Not declared: recorded note 1 | — |
| Anthropic → Gemini | — | — | Not declared: recorded note 1 | — |
| Cohere → Gemini | — | — | Not declared: recorded note 1 | — |
| Gemini → Gemini | — | — | Not declared: recorded note 1 | — |
| Bedrock → Gemini | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Gemini | — | — | Not declared: recorded note 1 | — |
| OpenAI → Gemini | — | — | Not declared: recorded note 1 | — |
| OpenAI Responses → Cohere | — | — | Not declared: recorded note 1 | — |
| OpenAI → Cohere | — | — | Not declared: recorded note 1 | — |
| Cohere → Cohere | — | — | Not declared: recorded note 1 | — |
| Gemini → Cohere | — | — | Not declared: recorded note 1 | — |
| Anthropic → Cohere | — | — | Not declared: recorded note 1 | — |
| Bedrock → Cohere | — | — | Not declared: recorded note 1 | — |
Probe evidence and instrument provenance
Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.
| Ingress → upstream | Cell state | Probe HTTP status | Recorded reason | Recorded detail | Evidence scope | Response excerpt | Tested build |
|---|---|---|---|---|---|---|---|
| Anthropic → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Cohere → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Bedrock → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Gemini → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI → Bedrock | Not served | 501 | failed | probed and answered failed (HTTP 501) | Recorded run evidence | {"error":{"message":"unsupported by the rust path: unrecognized request parameter","type":"api_error","param":null,"code":501}} | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI Responses → Bedrock | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Bedrock → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI → OpenAI | Served | — | — | — | Recorded run evidence | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Anthropic → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI Responses → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Gemini → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Cohere → OpenAI | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Gemini → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Cohere → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Bedrock → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI Responses → OpenAI Responses | Served | — | — | — | Recorded run evidence | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Anthropic → OpenAI Responses | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI → Anthropic | Served | — | — | — | Recorded run evidence | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Cohere → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Bedrock → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Gemini → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Anthropic → Anthropic | Served | — | — | — | Recorded run evidence | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI Responses → Anthropic | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Anthropic → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Cohere → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Gemini → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Bedrock → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI Responses → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI → Gemini | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI Responses → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| OpenAI → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Cohere → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Gemini → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Anthropic → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
| Bedrock → Cohere | Not declared | — | recorded note 1 | recorded note 1 | recorded note 2 | — | built from source; recorded binary src/litellm-rust/target/release/litellm-gateway |
- Harness commit
d240badd966933fa19eec0606b2d1f3945f18ad2- Mock SHA-256
99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8- Load generator SHA-256
not recorded- net.core.somaxconn
- 65,535
- net.ipv4.tcp_max_syn_backlog
- 65,535
- Gateway build receipt
- Binary name
- litellm-gateway
- Binary sha256
ae1fec119755a8a2ae5b6a9d56225c34205ce7a54548cb11696b03b2557ce01b- Cargo lock sha256
ae9550da9f00c3ffb10ac2974d7d33c3784902c702c74c6aa85f1b83f1efcb2d- Compiler
- rustc 1.99.1 (58cf8f9c3 2026-10-13) binary: rustc commit-hash: 58cf8f9c35b355199727627d07ae34a4e2846d67 commit-date: 2026-10-13 host: aarch64-unknown-linux-gnu release: 1.99.1 LLVM version: 23.1.1
- Source commit
b7d579fab2818968322ef96e5b8e98dc8178fafc- Source dirty
- No
Stream sweep diagnostics
Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.
| Ingress → upstream | Concurrency | Lanes by HTTP status | Lanes with no response |
|---|---|---|---|
| OpenAI → OpenAI | 1 | HTTP 501: 1 | 0 |
| OpenAI → Anthropic | 1 | HTTP 501: 1 | 0 |
| Anthropic → Anthropic | 1 | HTTP 200: 1 | 0 |
| Anthropic → Anthropic | 2 | HTTP 200: 2 | 0 |
| Anthropic → Anthropic | 4 | HTTP 200: 4 | 0 |
| Anthropic → Anthropic | 8 | HTTP 200: 8 | 0 |
| Anthropic → Anthropic | 16 | HTTP 200: 16 | 0 |
| Anthropic → Anthropic | 32 | HTTP 200: 32 | 0 |
| Anthropic → Anthropic | 64 | HTTP 200: 64 | 0 |
| Anthropic → Anthropic | 128 | HTTP 200: 128 | 0 |
| Anthropic → Anthropic | 256 | HTTP 200: 256 | 0 |
| Anthropic → Anthropic | 512 | HTTP 200: 512 | 0 |
| Anthropic → Anthropic | 1,024 | HTTP 200: 1,024 | 0 |
| Anthropic → Anthropic | 768 | HTTP 200: 768 | 0 |
| Anthropic → Anthropic | 896 | HTTP 200: 896 | 0 |
| Anthropic → Anthropic | 960 | HTTP 200: 960 | 0 |
| Anthropic → Anthropic | 928 | HTTP 200: 928 | 0 |
| Anthropic → Anthropic | 912 | HTTP 200: 912 | 0 |
| Anthropic → Anthropic | 920 | HTTP 200: 920 | 0 |
| Anthropic → Anthropic | 916 | HTTP 200: 916 | 0 |
| Anthropic → Anthropic | 918 | HTTP 200: 918 | 0 |
| Anthropic → Anthropic | 919 | HTTP 200: 919 | 0 |
| Anthropic → Anthropic | 919 | HTTP 200: 919 | 0 |
| Anthropic → Anthropic | 919 | HTTP 200: 919 | 0 |
Recorded notes
Text recorded with the cells above. Each note is shown once; rows refer to it by number.
- Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json
- Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.
Evidence for the current measurements
- run-litellm-rust-69d8e6ed325740d9da7f0aaf31b815418c4801eb6549b2e2c68f768a92af3ad4 · complete raw record including RSS series
Configuration recorded by the run
general_settings:
master_key: "gwbench"
model_list:
- model_name: bench-openai
litellm_params:
model: openai_like/gpt-4o-mini
api_base: http://127.0.0.1:8000/v1
api_key: dummy
- model_name: bench-responses
litellm_params:
model: openai/gpt-4o-mini
api_base: http://127.0.0.1:8000/v1
api_key: dummy
- model_name: bench-anthropic
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_base: http://127.0.0.1:8000
api_key: dummy
- model_name: bench-bedrock
litellm_params:
model: bedrock/amazon.nova-micro-v1:0
api_base: http://127.0.0.1:8000
api_key: dummy
aws_region_name: us-east-1
Compare a shared protocol cell
- LiteLLM · Rust vs agentgateway
- LiteLLM · Rust vs AISIX (api7)
- LiteLLM · Rust vs APISIX
- LiteLLM · Rust vs Bifrost
- LiteLLM · Rust vs Busbar
- LiteLLM · Rust vs GoModel
- LiteLLM · Rust vs Helicone
- LiteLLM · Rust vs Higress
- LiteLLM · Rust vs Kong
- LiteLLM · Rust vs LiteLLM · Python
- LiteLLM · Rust vs One-API
- LiteLLM · Rust vs Plano
- LiteLLM · Rust vs Portkey
- LiteLLM · Rust vs TensorZero