# LiteLLM · Rust benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency.  [Methodology and limitations](https://onthebench.ai/gateways/method) .

Rust · recorded build: src/litellm-rust/target/release/litellm-gateway. Measured: 2026-11-10T17:39:51Z.

[JSON](https://onthebench.ai/api/v1/gateways/litellm-rust)  ·  [Markdown](https://onthebench.ai/gateways/litellm-rust.md)  ·  [CSV](https://onthebench.ai/gateways/data.csv?gateway=litellm-rust&compact=1)  ·  [Project source](https://github.com/BerriAI/litellm)  ·  [Interactive dashboard](https://onthebench.ai/gateways)

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

## Published measurements

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| anthropic → bedrock | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| cohere → bedrock | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| bedrock → bedrock | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| gemini → bedrock | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| openai → bedrock | — | — | not_served: probed and answered failed (HTTP 501) | — |
| openai-responses → bedrock | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| bedrock → openai | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| openai → openai | added_latency_p50_us | 249 | valid; gateway_c1_samples=22021; direct_c1_samples=267021 | — |
| openai → openai | added_latency_p99_us | 279 | valid; gateway_c1_samples=22021; direct_c1_samples=267021 | — |
| openai → openai | gateway_c1_p99_us | 306 | valid; gateway_c1_samples=22021 | — |
| openai → openai | direct_c1_p99_us | 27 | valid; direct_c1_samples=267021 | — |
| openai → openai | cpu_us_per_request | 272.6013913681787 | valid | — |
| openai → openai | rps_per_cpu_second | 3668.359853121175 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.6790579018170796 | valid | — |
| openai → openai | cost_window_ok | 59941 | valid | — |
| openai → openai | cost_window_rps | 9989 | valid | — |
| openai → openai | cost_threads | 5 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | not_measured | not_measured | — |
| openai → openai | added_ttft_p99_us | not_measured | not_measured | — |
| openai → openai | added_gap_p50_us | not_measured | not_measured | — |
| openai → openai | added_gap_p99_us | not_measured | not_measured | — |
| openai → openai | streams_sustained_fps | 0 (measured_failure) | valid | — |
| openai → openai | streams_sustained | 0 (measured_failure) | valid | — |
| openai → openai | idle_rss_mib | 12.73828125 | valid | — |
| openai → openai | steady_state_rss_mib | 18.9921875 | valid | — |
| openai → openai | recovered_rss_mib | 18.99609375 | valid | — |
| openai → openai | peak_rss_mib | 19.875 | valid | — |
| openai → openai | peak_rss_hwm_mib | 19.484375 | valid | — |
| openai → openai | time_to_plateau_s | not_measured | not_measured | — |
| openai → openai | growth_rate_mib_per_min | -0.1791634760562065 | valid | — |
| openai → openai | load_s | 300 | valid | — |
| openai → openai | shape | 0 | valid | — |
| openai → openai | idle_shape | not_measured | not_measured | — |
| openai → openai | idle_static | 1 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → openai | throughput_rps | 8404 req/s | concurrency 4; samples 50435 | 1 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | 5 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | 10 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | 50 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | 100 |
| openai → openai | throughput_rps | 10049 req/s | concurrency 16; samples 60306 | unbounded |
| anthropic → openai | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| openai-responses → openai | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| gemini → openai | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| cohere → openai | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| gemini → openai-responses | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| cohere → openai-responses | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| bedrock → openai-responses | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| openai-responses → openai-responses | added_latency_p50_us | 267 | valid; gateway_c1_samples=20661; direct_c1_samples=267569 | — |
| openai-responses → openai-responses | added_latency_p99_us | 298 | valid; gateway_c1_samples=20661; direct_c1_samples=267569 | — |
| openai-responses → openai-responses | gateway_c1_p99_us | 325 | valid; gateway_c1_samples=20661 | — |
| openai-responses → openai-responses | direct_c1_p99_us | 27 | valid; direct_c1_samples=267569 | — |
| openai-responses → openai-responses | cpu_us_per_request | 287.74399582885263 | valid | — |
| openai-responses → openai-responses | rps_per_cpu_second | 3475.3114382785957 | valid | — |
| openai-responses → openai-responses | cost_window_conc | 8 | valid | — |
| openai-responses → openai-responses | cost_core_utilisation | 0.7339219155366118 | valid | — |
| openai-responses → openai-responses | cost_window_ok | 61374 | valid | — |
| openai-responses → openai-responses | cost_window_rps | 10227 | valid | — |
| openai-responses → openai-responses | cost_threads | 5 | valid | — |
| openai-responses → openai-responses | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai-responses → openai-responses | cost_majflt | 0 | valid | — |
| openai-responses → openai-responses | added_ttft_p50_us | untestable | untestable | — |
| openai-responses → openai-responses | added_ttft_p99_us | untestable | untestable | — |
| openai-responses → openai-responses | added_gap_p50_us | untestable | untestable | — |
| openai-responses → openai-responses | added_gap_p99_us | untestable | untestable | — |
| openai-responses → openai-responses | streams_sustained_fps | untestable | untestable | — |
| openai-responses → openai-responses | streams_sustained | untestable | untestable | — |
| openai-responses → openai-responses | idle_rss_mib | 12.7421875 | valid | — |
| openai-responses → openai-responses | steady_state_rss_mib | 19.47265625 | valid | — |
| openai-responses → openai-responses | recovered_rss_mib | 19.28515625 | valid | — |
| openai-responses → openai-responses | peak_rss_mib | 19.5 | valid | — |
| openai-responses → openai-responses | peak_rss_hwm_mib | 19.5 | valid | — |
| openai-responses → openai-responses | time_to_plateau_s | 66 | valid | — |
| openai-responses → openai-responses | growth_rate_mib_per_min | 0.028056430633918075 | valid | — |
| openai-responses → openai-responses | load_s | 300 | valid | — |
| openai-responses → openai-responses | shape | not_measured | not_measured | — |
| openai-responses → openai-responses | idle_shape | not_measured | not_measured | — |
| openai-responses → openai-responses | idle_static | 1 | valid | — |
| openai-responses → openai-responses | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → openai-responses | throughput_rps | 8050 req/s | concurrency 4; samples 48309 | 1 |
| openai-responses → openai-responses | throughput_rps | 10191 req/s | concurrency 16; samples 61157 | 5 |
| openai-responses → openai-responses | throughput_rps | 10191 req/s | concurrency 16; samples 61157 | 10 |
| openai-responses → openai-responses | throughput_rps | 10191 req/s | concurrency 16; samples 61157 | 50 |
| openai-responses → openai-responses | throughput_rps | 10191 req/s | concurrency 16; samples 61157 | 100 |
| openai-responses → openai-responses | throughput_rps | 10191 req/s | concurrency 16; samples 61157 | unbounded |
| openai → openai-responses | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| anthropic → openai-responses | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| openai → anthropic | added_latency_p50_us | 253 | valid; gateway_c1_samples=21651; direct_c1_samples=262942 | — |
| openai → anthropic | added_latency_p99_us | 283 | valid; gateway_c1_samples=21651; direct_c1_samples=262942 | — |
| openai → anthropic | gateway_c1_p99_us | 311 | valid; gateway_c1_samples=21651 | — |
| openai → anthropic | direct_c1_p99_us | 28 | valid; direct_c1_samples=262942 | — |
| openai → anthropic | cpu_us_per_request | 277.0705901801951 | valid | — |
| openai → anthropic | rps_per_cpu_second | 3609.1885441527443 | valid | — |
| openai → anthropic | cost_window_conc | 8 | valid | — |
| openai → anthropic | cost_core_utilisation | 0.6965431268306815 | valid | — |
| openai → anthropic | cost_window_ok | 60490 | valid | — |
| openai → anthropic | cost_window_rps | 10080 | valid | — |
| openai → anthropic | cost_threads | 5 | valid | — |
| openai → anthropic | cost_nonvol_ctxt_per_request | 0 | valid | — |
| openai → anthropic | cost_majflt | 0 | valid | — |
| openai → anthropic | added_ttft_p50_us | not_measured | not_measured | — |
| openai → anthropic | added_ttft_p99_us | not_measured | not_measured | — |
| openai → anthropic | added_gap_p50_us | not_measured | not_measured | — |
| openai → anthropic | added_gap_p99_us | not_measured | not_measured | — |
| openai → anthropic | streams_sustained_fps | 0 (measured_failure) | valid | — |
| openai → anthropic | streams_sustained | 0 (measured_failure) | valid | — |
| openai → anthropic | idle_rss_mib | 12.7578125 | valid | — |
| openai → anthropic | steady_state_rss_mib | 19.046875 | valid | — |
| openai → anthropic | recovered_rss_mib | 18.984375 | valid | — |
| openai → anthropic | peak_rss_mib | 19.66015625 | valid | — |
| openai → anthropic | peak_rss_hwm_mib | 19.33203125 | valid | — |
| openai → anthropic | time_to_plateau_s | not_measured | not_measured | — |
| openai → anthropic | growth_rate_mib_per_min | 0.3275657622729096 | valid | — |
| openai → anthropic | load_s | 300 | valid | — |
| openai → anthropic | shape | 0 | valid | — |
| openai → anthropic | idle_shape | not_measured | not_measured | — |
| openai → anthropic | idle_static | 1 | valid | — |
| openai → anthropic | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → anthropic | throughput_rps | 8572 req/s | concurrency 4; samples 51441 | 1 |
| openai → anthropic | throughput_rps | 10082 req/s | concurrency 16; samples 60500 | 5 |
| openai → anthropic | throughput_rps | 10082 req/s | concurrency 16; samples 60500 | 10 |
| openai → anthropic | throughput_rps | 10082 req/s | concurrency 16; samples 60500 | 50 |
| openai → anthropic | throughput_rps | 10082 req/s | concurrency 16; samples 60500 | 100 |
| openai → anthropic | throughput_rps | 10082 req/s | concurrency 16; samples 60500 | unbounded |
| cohere → anthropic | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| bedrock → anthropic | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| gemini → anthropic | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| anthropic → anthropic | added_latency_p50_us | 275 | valid; gateway_c1_samples=20084; direct_c1_samples=262977 | — |
| anthropic → anthropic | added_latency_p99_us | 304 | valid; gateway_c1_samples=20084; direct_c1_samples=262977 | — |
| anthropic → anthropic | gateway_c1_p99_us | 332 | valid; gateway_c1_samples=20084 | — |
| anthropic → anthropic | direct_c1_p99_us | 28 | valid; direct_c1_samples=262977 | — |
| anthropic → anthropic | cpu_us_per_request | 297.0902986640554 | valid | — |
| anthropic → anthropic | rps_per_cpu_second | 3365.97998822837 | valid | — |
| anthropic → anthropic | cost_window_conc | 8 | valid | — |
| anthropic → anthropic | cost_core_utilisation | 0.7061370243602647 | valid | — |
| anthropic → anthropic | cost_window_ok | 57188 | valid | — |
| anthropic → anthropic | cost_window_rps | 9530 | valid | — |
| anthropic → anthropic | cost_threads | 5 | valid | — |
| anthropic → anthropic | cost_nonvol_ctxt_per_request | 0 | valid | — |
| anthropic → anthropic | cost_majflt | 0 | valid | — |
| anthropic → anthropic | added_ttft_p50_us | 406 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| anthropic → anthropic | added_ttft_p99_us | 477 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| anthropic → anthropic | added_gap_p50_us | 2 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| anthropic → anthropic | added_gap_p99_us | 3 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| anthropic → anthropic | streams_sustained_fps | 25279.8938149557 | valid | — |
| anthropic → anthropic | streams_sustained | 919 | valid | — |
| anthropic → anthropic | idle_rss_mib | 12.7421875 | valid | — |
| anthropic → anthropic | steady_state_rss_mib | 20.3515625 | valid | — |
| anthropic → anthropic | recovered_rss_mib | 20.21484375 | valid | — |
| anthropic → anthropic | peak_rss_mib | 21.0078125 | valid | — |
| anthropic → anthropic | peak_rss_hwm_mib | 20.5625 | valid | — |
| anthropic → anthropic | time_to_plateau_s | not_measured | not_measured | — |
| anthropic → anthropic | growth_rate_mib_per_min | -0.01099763747584857 | valid | — |
| anthropic → anthropic | load_s | 300 | valid | — |
| anthropic → anthropic | shape | 0 | valid | — |
| anthropic → anthropic | idle_shape | not_measured | not_measured | — |
| anthropic → anthropic | idle_static | 1 | valid | — |
| anthropic → anthropic | idle_growth_rate_mib_per_min | 0 | valid | — |
| anthropic → anthropic | throughput_rps | 8878 req/s | concurrency 4; samples 53276 | 1 |
| anthropic → anthropic | throughput_rps | 9865 req/s | concurrency 16; samples 59196 | 5 |
| anthropic → anthropic | throughput_rps | 9865 req/s | concurrency 16; samples 59196 | 10 |
| anthropic → anthropic | throughput_rps | 9865 req/s | concurrency 16; samples 59196 | 50 |
| anthropic → anthropic | throughput_rps | 9865 req/s | concurrency 16; samples 59196 | 100 |
| anthropic → anthropic | throughput_rps | 9865 req/s | concurrency 16; samples 59196 | unbounded |
| openai-responses → anthropic | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| anthropic → gemini | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| cohere → gemini | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| gemini → gemini | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| bedrock → gemini | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| openai-responses → gemini | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| openai → gemini | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| openai-responses → cohere | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| openai → cohere | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| cohere → cohere | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| gemini → cohere | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| anthropic → cohere | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |
| bedrock → cohere | — | — | not_declared: Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json | — |

## Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

| Ingress → upstream | Cell state | Probe HTTP status | Recorded reason | Recorded detail | Evidence scope | Response excerpt | Tested build |
| --- | --- | --- | --- | --- | --- | --- | --- |
| anthropic → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| cohere → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| bedrock → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| gemini → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| openai → bedrock | not_served | 501 | failed | probed and answered failed (HTTP 501) | Recorded run evidence | `{"error":{"message":"unsupported by the rust path: unrecognized request parameter","type":"api_error","param":null,"code":501}}` | src/litellm-rust/target/release/litellm-gateway |
| openai-responses → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| bedrock → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| openai → openai | served | — | — | — | Recorded run evidence | — | src/litellm-rust/target/release/litellm-gateway |
| anthropic → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| openai-responses → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| gemini → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| cohere → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| gemini → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| cohere → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| bedrock → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| openai-responses → openai-responses | served | — | — | — | Recorded run evidence | — | src/litellm-rust/target/release/litellm-gateway |
| openai → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| anthropic → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| openai → anthropic | served | — | — | — | Recorded run evidence | — | src/litellm-rust/target/release/litellm-gateway |
| cohere → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| bedrock → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| gemini → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| anthropic → anthropic | served | — | — | — | Recorded run evidence | — | src/litellm-rust/target/release/litellm-gateway |
| openai-responses → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| anthropic → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| cohere → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| gemini → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| bedrock → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| openai-responses → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| openai → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| openai-responses → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| openai → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| cohere → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| gemini → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| anthropic → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |
| bedrock → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | src/litellm-rust/target/release/litellm-gateway |

**Harness commit**: `d240badd966933fa19eec0606b2d1f3945f18ad2`

**Mock SHA-256**: `99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8`

**Load generator SHA-256**: `not recorded`

**somaxconn**: 65535

**tcp_max_syn_backlog**: 65535

**gateway_provenance**: **binary_name**: litellm-gateway

**binary_sha256**: `ae1fec119755a8a2ae5b6a9d56225c34205ce7a54548cb11696b03b2557ce01b`

**cargo_lock_sha256**: `ae9550da9f00c3ffb10ac2974d7d33c3784902c702c74c6aa85f1b83f1efcb2d`

**compiler**: rustc 1.99.1 (58cf8f9c3 2026-10-13) binary: rustc commit-hash: 58cf8f9c35b355199727627d07ae34a4e2846d67 commit-date: 2026-10-13 host: aarch64-unknown-linux-gnu release: 1.99.1 LLVM version: 23.1.1

**source_commit**: `b7d579fab2818968322ef96e5b8e98dc8178fafc`

**source_dirty**: false

## Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

| Ingress → upstream | Concurrency | Lanes by HTTP status | Lanes with no response |
| --- | --- | --- | --- |
| openai → openai | 1 | {"501":1} | 0 |
| openai → anthropic | 1 | {"501":1} | 0 |
| anthropic → anthropic | 1 | {"200":1} | 0 |
| anthropic → anthropic | 2 | {"200":2} | 0 |
| anthropic → anthropic | 4 | {"200":4} | 0 |
| anthropic → anthropic | 8 | {"200":8} | 0 |
| anthropic → anthropic | 16 | {"200":16} | 0 |
| anthropic → anthropic | 32 | {"200":32} | 0 |
| anthropic → anthropic | 64 | {"200":64} | 0 |
| anthropic → anthropic | 128 | {"200":128} | 0 |
| anthropic → anthropic | 256 | {"200":256} | 0 |
| anthropic → anthropic | 512 | {"200":512} | 0 |
| anthropic → anthropic | 1024 | {"200":1024} | 0 |
| anthropic → anthropic | 768 | {"200":768} | 0 |
| anthropic → anthropic | 896 | {"200":896} | 0 |
| anthropic → anthropic | 960 | {"200":960} | 0 |
| anthropic → anthropic | 928 | {"200":928} | 0 |
| anthropic → anthropic | 912 | {"200":912} | 0 |
| anthropic → anthropic | 920 | {"200":920} | 0 |
| anthropic → anthropic | 916 | {"200":916} | 0 |
| anthropic → anthropic | 918 | {"200":918} | 0 |
| anthropic → anthropic | 919 | {"200":919} | 0 |
| anthropic → anthropic | 919 | {"200":919} | 0 |
| anthropic → anthropic | 919 | {"200":919} | 0 |

## Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

- Pinned v1.108.3 standalone Rust source b7d579fab2818968322ef96e5b8e98dc8178fafc: gateway-inference/src/lib.rs registers Chat, Responses and Messages ingress. inference-chat/src/common_utils.rs registers OpenAI-like, Anthropic and Bedrock Converse adapters; inference-responses/src/prepare.rs supports native OpenAI Responses; inference-messages/src/common_utils.rs has native Anthropic, but no OpenAI cross-conversion. Its Bedrock Messages path uses Invoke, not this matrix's Converse wire. Native Gemini, Cohere and Bedrock ingress are not registered. OpenAI Chat to Bedrock Converse is configured and probed with the standard workload request, which includes max_tokens; the released gateway answers HTTP 501 ("unsupported by the rust path: unrecognized request parameter"), which the engine records as not configured. Pairs outside the routes above are not configured. Evidence: results/diagnostics/refresh-configuration/litellm-rust-exploratory-preflight.json
- Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

## Evidence for the current measurements

- [run-litellm-rust-69d8e6ed325740d9da7f0aaf31b815418c4801eb6549b2e2c68f768a92af3ad4](https://onthebench.ai/gateways/runs/run-litellm-rust-69d8e6ed325740d9da7f0aaf31b815418c4801eb6549b2e2c68f768a92af3ad4)  ·  [complete raw record including RSS series](https://onthebench.ai/api/v1/runs/run-litellm-rust-69d8e6ed325740d9da7f0aaf31b815418c4801eb6549b2e2c68f768a92af3ad4/raw.json)

## Configuration recorded by the run

```
general_settings:
  master_key: "gwbench"
model_list:
  - model_name: bench-openai
    litellm_params:
      model: openai_like/gpt-4o-mini
      api_base: http://127.0.0.1:8000/v1
      api_key: dummy
  - model_name: bench-responses
    litellm_params:
      model: openai/gpt-4o-mini
      api_base: http://127.0.0.1:8000/v1
      api_key: dummy
  - model_name: bench-anthropic
    litellm_params:
      model: anthropic/claude-3-5-sonnet-20241022
      api_base: http://127.0.0.1:8000
      api_key: dummy
  - model_name: bench-bedrock
    litellm_params:
      model: bedrock/amazon.nova-micro-v1:0
      api_base: http://127.0.0.1:8000
      api_key: dummy
      aws_region_name: us-east-1

```

## Compare a shared protocol cell

- [LiteLLM · Rust vs agentgateway](https://onthebench.ai/gateways/compare/litellm-rust-vs-agentgateway)
- [LiteLLM · Rust vs AISIX (api7)](https://onthebench.ai/gateways/compare/litellm-rust-vs-aisix)
- [LiteLLM · Rust vs APISIX](https://onthebench.ai/gateways/compare/litellm-rust-vs-apisix)
- [LiteLLM · Rust vs Bifrost](https://onthebench.ai/gateways/compare/litellm-rust-vs-bifrost)
- [LiteLLM · Rust vs Busbar](https://onthebench.ai/gateways/compare/litellm-rust-vs-busbar)
- [LiteLLM · Rust vs GoModel](https://onthebench.ai/gateways/compare/litellm-rust-vs-gomodel)
- [LiteLLM · Rust vs Helicone](https://onthebench.ai/gateways/compare/litellm-rust-vs-helicone)
- [LiteLLM · Rust vs Higress](https://onthebench.ai/gateways/compare/litellm-rust-vs-higress)
- [LiteLLM · Rust vs Kong](https://onthebench.ai/gateways/compare/litellm-rust-vs-kong)
- [LiteLLM · Rust vs LiteLLM · Python](https://onthebench.ai/gateways/compare/litellm-rust-vs-litellm-python)
- [LiteLLM · Rust vs One-API](https://onthebench.ai/gateways/compare/litellm-rust-vs-one-api)
- [LiteLLM · Rust vs Plano](https://onthebench.ai/gateways/compare/litellm-rust-vs-plano)
- [LiteLLM · Rust vs Portkey](https://onthebench.ai/gateways/compare/litellm-rust-vs-portkey)
- [LiteLLM · Rust vs TensorZero](https://onthebench.ai/gateways/compare/litellm-rust-vs-tensorzero)

## Historical measurements

- [2026-11-10T17:39:51Z — src/litellm-rust/target/release/litellm-gateway](https://onthebench.ai/gateways/runs/run-litellm-rust-69d8e6ed325740d9da7f0aaf31b815418c4801eb6549b2e2c68f768a92af3ad4)
- [2026-08-03T21:30:01Z — litellm-ai-gateway](https://onthebench.ai/gateways/runs/run-litellm-rust-50a1b780253a9655d548618bbcdd0969dd22b4f1a867714256048d1d68820eb9)
