# LiteLLM · Python benchmark measurements

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency.  [Methodology and limitations](https://onthebench.ai/gateways/method) .

Python · recorded build: ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5. Measured: 2026-11-10T17:39:41Z.

[JSON](https://onthebench.ai/api/v1/gateways/litellm-python)  ·  [Markdown](https://onthebench.ai/gateways/litellm-python.md)  ·  [CSV](https://onthebench.ai/gateways/data.csv?gateway=litellm-python&compact=1)  ·  [Project source](https://github.com/BerriAI/litellm)  ·  [Interactive dashboard](https://onthebench.ai/gateways)

Each row below names its protocol pair. A representative dashboard cell may differ between gateways; compare an identical pair before interpreting a ranking. Unavailable metrics retain their reason; below-resolution readings are not exact zeroes.

## Published measurements

| Ingress → upstream | Metric | Value | State / reason | p99 bound (ms) |
| --- | --- | --- | --- | --- |
| cohere → cohere | added_latency_p50_us | 11850 | valid; gateway_c1_samples=503; direct_c1_samples=264219 | — |
| cohere → cohere | added_latency_p99_us | 12623 | valid; gateway_c1_samples=503; direct_c1_samples=264219 | — |
| cohere → cohere | gateway_c1_p99_us | 12651 | valid; gateway_c1_samples=503 | — |
| cohere → cohere | direct_c1_p99_us | 28 | valid; direct_c1_samples=264219 | — |
| cohere → cohere | cpu_us_per_request | 12063.492063492064 | valid | — |
| cohere → cohere | rps_per_cpu_second | 82.89473684210526 | valid | — |
| cohere → cohere | cost_window_conc | 8 | valid | — |
| cohere → cohere | cost_core_utilisation | 0.24961199687176264 | valid | — |
| cohere → cohere | cost_window_ok | 504 | valid | — |
| cohere → cohere | cost_window_rps | 83 | valid | — |
| cohere → cohere | cost_threads | 6 | valid | — |
| cohere → cohere | cost_nonvol_ctxt_per_request | 0.051587301587301584 | valid | — |
| cohere → cohere | cost_majflt | 0 | valid | — |
| cohere → cohere | added_ttft_p50_us | untestable | untestable | — |
| cohere → cohere | added_ttft_p99_us | untestable | untestable | — |
| cohere → cohere | added_gap_p50_us | untestable | untestable | — |
| cohere → cohere | added_gap_p99_us | untestable | untestable | — |
| cohere → cohere | streams_sustained_fps | untestable | untestable | — |
| cohere → cohere | streams_sustained | untestable | untestable | — |
| cohere → cohere | idle_rss_mib | 592.81640625 | valid | — |
| cohere → cohere | steady_state_rss_mib | 663.33203125 | valid | — |
| cohere → cohere | recovered_rss_mib | 663.33203125 | valid | — |
| cohere → cohere | peak_rss_mib | 663.33203125 | valid | — |
| cohere → cohere | peak_rss_hwm_mib | 663.33203125 | valid | — |
| cohere → cohere | time_to_plateau_s | 81 | valid | — |
| cohere → cohere | growth_rate_mib_per_min | 0 | valid | — |
| cohere → cohere | load_s | 302 | valid | — |
| cohere → cohere | shape | not_measured | not_measured | — |
| cohere → cohere | idle_shape | not_measured | not_measured | — |
| cohere → cohere | idle_static | 1 | valid | — |
| cohere → cohere | idle_growth_rate_mib_per_min | 0 | valid | — |
| cohere → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| cohere → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| cohere → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| cohere → cohere | throughput_rps | 83 req/s | concurrency 1; samples 499 | 50 |
| cohere → cohere | throughput_rps | 83 req/s | concurrency 1; samples 499 | 100 |
| cohere → cohere | throughput_rps | 112 req/s | concurrency 1024; samples 674 | unbounded |
| openai → cohere | added_latency_p50_us | 7551 | valid; gateway_c1_samples=790; direct_c1_samples=267951 | — |
| openai → cohere | added_latency_p99_us | 8314 | valid; gateway_c1_samples=790; direct_c1_samples=267951 | — |
| openai → cohere | gateway_c1_p99_us | 8341 | valid; gateway_c1_samples=790 | — |
| openai → cohere | direct_c1_p99_us | 27 | valid; direct_c1_samples=267951 | — |
| openai → cohere | cpu_us_per_request | 7593.984962406015 | valid | — |
| openai → cohere | rps_per_cpu_second | 131.68316831683168 | valid | — |
| openai → cohere | cost_window_conc | 8 | valid | — |
| openai → cohere | cost_core_utilisation | 0.2502569364427439 | valid | — |
| openai → cohere | cost_window_ok | 798 | valid | — |
| openai → cohere | cost_window_rps | 132 | valid | — |
| openai → cohere | cost_threads | 13 | valid | — |
| openai → cohere | cost_nonvol_ctxt_per_request | 0.03132832080200501 | valid | — |
| openai → cohere | cost_majflt | 0 | valid | — |
| openai → cohere | added_ttft_p50_us | untestable | untestable | — |
| openai → cohere | added_ttft_p99_us | untestable | untestable | — |
| openai → cohere | added_gap_p50_us | untestable | untestable | — |
| openai → cohere | added_gap_p99_us | untestable | untestable | — |
| openai → cohere | streams_sustained_fps | untestable | untestable | — |
| openai → cohere | streams_sustained | untestable | untestable | — |
| openai → cohere | idle_rss_mib | 592.56640625 | valid | — |
| openai → cohere | steady_state_rss_mib | 666.06640625 | valid | — |
| openai → cohere | recovered_rss_mib | 666.0703125 | valid | — |
| openai → cohere | peak_rss_mib | 666.0703125 | valid | — |
| openai → cohere | peak_rss_hwm_mib | 666.0703125 | valid | — |
| openai → cohere | time_to_plateau_s | 61 | valid | — |
| openai → cohere | growth_rate_mib_per_min | 0.0020496859180001844 | valid | — |
| openai → cohere | load_s | 301 | valid | — |
| openai → cohere | shape | not_measured | not_measured | — |
| openai → cohere | idle_shape | not_measured | not_measured | — |
| openai → cohere | idle_static | 1 | valid | — |
| openai → cohere | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → cohere | throughput_rps | 131 req/s | concurrency 1; samples 787 | 10 |
| openai → cohere | throughput_rps | 131 req/s | concurrency 1; samples 787 | 50 |
| openai → cohere | throughput_rps | 131 req/s | concurrency 1; samples 787 | 100 |
| openai → cohere | throughput_rps | 138 req/s | concurrency 64; samples 829 | unbounded |
| bedrock → cohere | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| anthropic → cohere | added_latency_p50_us | 8248 | valid; gateway_c1_samples=723; direct_c1_samples=260556 | — |
| anthropic → cohere | added_latency_p99_us | 9054 | valid; gateway_c1_samples=723; direct_c1_samples=260556 | — |
| anthropic → cohere | gateway_c1_p99_us | 9082 | valid; gateway_c1_samples=723 | — |
| anthropic → cohere | direct_c1_p99_us | 28 | valid; direct_c1_samples=260556 | — |
| anthropic → cohere | cpu_us_per_request | 8407.202216066482 | valid | — |
| anthropic → cohere | rps_per_cpu_second | 118.94563426688632 | valid | — |
| anthropic → cohere | cost_window_conc | 8 | valid | — |
| anthropic → cohere | cost_core_utilisation | 0.2504942972604469 | valid | — |
| anthropic → cohere | cost_window_ok | 722 | valid | — |
| anthropic → cohere | cost_window_rps | 120 | valid | — |
| anthropic → cohere | cost_threads | 13 | valid | — |
| anthropic → cohere | cost_nonvol_ctxt_per_request | 0.024930747922437674 | valid | — |
| anthropic → cohere | cost_majflt | 0 | valid | — |
| anthropic → cohere | added_ttft_p50_us | untestable | untestable | — |
| anthropic → cohere | added_ttft_p99_us | untestable | untestable | — |
| anthropic → cohere | added_gap_p50_us | untestable | untestable | — |
| anthropic → cohere | added_gap_p99_us | untestable | untestable | — |
| anthropic → cohere | streams_sustained_fps | untestable | untestable | — |
| anthropic → cohere | streams_sustained | untestable | untestable | — |
| anthropic → cohere | idle_rss_mib | 592.1796875 | valid | — |
| anthropic → cohere | steady_state_rss_mib | 665.765625 | valid | — |
| anthropic → cohere | recovered_rss_mib | 665.765625 | valid | — |
| anthropic → cohere | peak_rss_mib | 665.765625 | valid | — |
| anthropic → cohere | peak_rss_hwm_mib | 665.765625 | valid | — |
| anthropic → cohere | time_to_plateau_s | 61 | valid | — |
| anthropic → cohere | growth_rate_mib_per_min | 0 | valid | — |
| anthropic → cohere | load_s | 302 | valid | — |
| anthropic → cohere | shape | not_measured | not_measured | — |
| anthropic → cohere | idle_shape | not_measured | not_measured | — |
| anthropic → cohere | idle_static | 1 | valid | — |
| anthropic → cohere | idle_growth_rate_mib_per_min | 0 | valid | — |
| anthropic → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| anthropic → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| anthropic → cohere | throughput_rps | 118 req/s | concurrency 1; samples 713 | 10 |
| anthropic → cohere | throughput_rps | 120 req/s | concurrency 2; samples 722 | 50 |
| anthropic → cohere | throughput_rps | 120 req/s | concurrency 2; samples 722 | 100 |
| anthropic → cohere | throughput_rps | 120 req/s | concurrency 2; samples 722 | unbounded |
| openai-responses → cohere | added_latency_p50_us | 9710 | valid; gateway_c1_samples=615; direct_c1_samples=263786 | — |
| openai-responses → cohere | added_latency_p99_us | 10423 | valid; gateway_c1_samples=615; direct_c1_samples=263786 | — |
| openai-responses → cohere | gateway_c1_p99_us | 10451 | valid; gateway_c1_samples=615 | — |
| openai-responses → cohere | direct_c1_p99_us | 28 | valid; direct_c1_samples=263786 | — |
| openai-responses → cohere | cpu_us_per_request | 9886.363636363636 | valid | — |
| openai-responses → cohere | rps_per_cpu_second | 101.14942528735632 | valid | — |
| openai-responses → cohere | cost_window_conc | 8 | valid | — |
| openai-responses → cohere | cost_core_utilisation | 0.25085857203068684 | valid | — |
| openai-responses → cohere | cost_window_ok | 616 | valid | — |
| openai-responses → cohere | cost_window_rps | 102 | valid | — |
| openai-responses → cohere | cost_threads | 13 | valid | — |
| openai-responses → cohere | cost_nonvol_ctxt_per_request | 0.04707792207792208 | valid | — |
| openai-responses → cohere | cost_majflt | 0 | valid | — |
| openai-responses → cohere | added_ttft_p50_us | untestable | untestable | — |
| openai-responses → cohere | added_ttft_p99_us | untestable | untestable | — |
| openai-responses → cohere | added_gap_p50_us | untestable | untestable | — |
| openai-responses → cohere | added_gap_p99_us | untestable | untestable | — |
| openai-responses → cohere | streams_sustained_fps | untestable | untestable | — |
| openai-responses → cohere | streams_sustained | untestable | untestable | — |
| openai-responses → cohere | idle_rss_mib | 592.70703125 | valid | — |
| openai-responses → cohere | steady_state_rss_mib | 667.8046875 | valid | — |
| openai-responses → cohere | recovered_rss_mib | 667.8046875 | valid | — |
| openai-responses → cohere | peak_rss_mib | 667.8046875 | valid | — |
| openai-responses → cohere | peak_rss_hwm_mib | 667.8046875 | valid | — |
| openai-responses → cohere | time_to_plateau_s | 62 | valid | — |
| openai-responses → cohere | growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → cohere | load_s | 304 | valid | — |
| openai-responses → cohere | shape | not_measured | not_measured | — |
| openai-responses → cohere | idle_shape | not_measured | not_measured | — |
| openai-responses → cohere | idle_static | 1 | valid | — |
| openai-responses → cohere | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai-responses → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai-responses → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai-responses → cohere | throughput_rps | 104 req/s | concurrency 2; samples 630 | 50 |
| openai-responses → cohere | throughput_rps | 104 req/s | concurrency 2; samples 630 | 100 |
| openai-responses → cohere | throughput_rps | 106 req/s | concurrency 64; samples 637 | unbounded |
| gemini → cohere | added_latency_p50_us | 13881 | valid; gateway_c1_samples=428; direct_c1_samples=262194 | — |
| gemini → cohere | added_latency_p99_us | 14770 | valid; gateway_c1_samples=428; direct_c1_samples=262194 | — |
| gemini → cohere | gateway_c1_p99_us | 14798 | valid; gateway_c1_samples=428 | — |
| gemini → cohere | direct_c1_p99_us | 28 | valid; direct_c1_samples=262194 | — |
| gemini → cohere | cpu_us_per_request | 14342.723004694835 | valid | — |
| gemini → cohere | rps_per_cpu_second | 69.72176759410802 | valid | — |
| gemini → cohere | cost_window_conc | 8 | valid | — |
| gemini → cohere | cost_core_utilisation | 0.2507360988510164 | valid | — |
| gemini → cohere | cost_window_ok | 426 | valid | — |
| gemini → cohere | cost_window_rps | 70 | valid | — |
| gemini → cohere | cost_threads | 13 | valid | — |
| gemini → cohere | cost_nonvol_ctxt_per_request | 0.06338028169014084 | valid | — |
| gemini → cohere | cost_majflt | 0 | valid | — |
| gemini → cohere | added_ttft_p50_us | untestable | untestable | — |
| gemini → cohere | added_ttft_p99_us | untestable | untestable | — |
| gemini → cohere | added_gap_p50_us | untestable | untestable | — |
| gemini → cohere | added_gap_p99_us | untestable | untestable | — |
| gemini → cohere | streams_sustained_fps | untestable | untestable | — |
| gemini → cohere | streams_sustained | untestable | untestable | — |
| gemini → cohere | idle_rss_mib | 593.359375 | valid | — |
| gemini → cohere | steady_state_rss_mib | 670.828125 | valid | — |
| gemini → cohere | recovered_rss_mib | 670.9921875 | valid | — |
| gemini → cohere | peak_rss_mib | 670.9921875 | valid | — |
| gemini → cohere | peak_rss_hwm_mib | 670.9921875 | valid | — |
| gemini → cohere | time_to_plateau_s | 63 | valid | — |
| gemini → cohere | growth_rate_mib_per_min | 0.09640675558624215 | valid | — |
| gemini → cohere | load_s | 302 | valid | — |
| gemini → cohere | shape | not_measured | not_measured | — |
| gemini → cohere | idle_shape | not_measured | not_measured | — |
| gemini → cohere | idle_static | 1 | valid | — |
| gemini → cohere | idle_growth_rate_mib_per_min | 0 | valid | — |
| gemini → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| gemini → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| gemini → cohere | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| gemini → cohere | throughput_rps | 71 req/s | concurrency 1; samples 427 | 50 |
| gemini → cohere | throughput_rps | 71 req/s | concurrency 1; samples 427 | 100 |
| gemini → cohere | throughput_rps | 73 req/s | concurrency 256; samples 439 | unbounded |
| cohere → anthropic | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| bedrock → anthropic | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| openai-responses → anthropic | added_latency_p50_us | 13061 | valid; gateway_c1_samples=457; direct_c1_samples=263938 | — |
| openai-responses → anthropic | added_latency_p99_us | 13826 | valid; gateway_c1_samples=457; direct_c1_samples=263938 | — |
| openai-responses → anthropic | gateway_c1_p99_us | 13854 | valid; gateway_c1_samples=457 | — |
| openai-responses → anthropic | direct_c1_p99_us | 28 | valid; direct_c1_samples=263938 | — |
| openai-responses → anthropic | cpu_us_per_request | 13414.096916299559 | valid | — |
| openai-responses → anthropic | rps_per_cpu_second | 74.54844006568145 | valid | — |
| openai-responses → anthropic | cost_window_conc | 8 | valid | — |
| openai-responses → anthropic | cost_core_utilisation | 0.25025039298228663 | valid | — |
| openai-responses → anthropic | cost_window_ok | 454 | valid | — |
| openai-responses → anthropic | cost_window_rps | 75 | valid | — |
| openai-responses → anthropic | cost_threads | 13 | valid | — |
| openai-responses → anthropic | cost_nonvol_ctxt_per_request | 0.05506607929515418 | valid | — |
| openai-responses → anthropic | cost_majflt | 0 | valid | — |
| openai-responses → anthropic | added_ttft_p50_us | untestable | untestable | — |
| openai-responses → anthropic | added_ttft_p99_us | untestable | untestable | — |
| openai-responses → anthropic | added_gap_p50_us | untestable | untestable | — |
| openai-responses → anthropic | added_gap_p99_us | untestable | untestable | — |
| openai-responses → anthropic | streams_sustained_fps | untestable | untestable | — |
| openai-responses → anthropic | streams_sustained | untestable | untestable | — |
| openai-responses → anthropic | idle_rss_mib | 592.65625 | valid | — |
| openai-responses → anthropic | steady_state_rss_mib | 667.640625 | valid | — |
| openai-responses → anthropic | recovered_rss_mib | 667.640625 | valid | — |
| openai-responses → anthropic | peak_rss_mib | 667.640625 | valid | — |
| openai-responses → anthropic | peak_rss_hwm_mib | 667.640625 | valid | — |
| openai-responses → anthropic | time_to_plateau_s | 62 | valid | — |
| openai-responses → anthropic | growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → anthropic | load_s | 302 | valid | — |
| openai-responses → anthropic | shape | not_measured | not_measured | — |
| openai-responses → anthropic | idle_shape | not_measured | not_measured | — |
| openai-responses → anthropic | idle_static | 1 | valid | — |
| openai-responses → anthropic | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai-responses → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai-responses → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai-responses → anthropic | throughput_rps | 76 req/s | concurrency 1; samples 458 | 50 |
| openai-responses → anthropic | throughput_rps | 76 req/s | concurrency 1; samples 458 | 100 |
| openai-responses → anthropic | throughput_rps | 76 req/s | concurrency 1; samples 458 | unbounded |
| gemini → anthropic | added_latency_p50_us | 16324 | valid; gateway_c1_samples=364; direct_c1_samples=262033 | — |
| gemini → anthropic | added_latency_p99_us | 17226 | valid; gateway_c1_samples=364; direct_c1_samples=262033 | — |
| gemini → anthropic | gateway_c1_p99_us | 17254 | valid; gateway_c1_samples=364 | — |
| gemini → anthropic | direct_c1_p99_us | 28 | valid; direct_c1_samples=262033 | — |
| gemini → anthropic | cpu_us_per_request | 16831.955922865014 | valid | — |
| gemini → anthropic | rps_per_cpu_second | 59.41080196399345 | valid | — |
| gemini → anthropic | cost_window_conc | 8 | valid | — |
| gemini → anthropic | cost_core_utilisation | 0.2501531621486448 | valid | — |
| gemini → anthropic | cost_window_ok | 363 | valid | — |
| gemini → anthropic | cost_window_rps | 60 | valid | — |
| gemini → anthropic | cost_threads | 13 | valid | — |
| gemini → anthropic | cost_nonvol_ctxt_per_request | 0.0743801652892562 | valid | — |
| gemini → anthropic | cost_majflt | 0 | valid | — |
| gemini → anthropic | added_ttft_p50_us | untestable | untestable | — |
| gemini → anthropic | added_ttft_p99_us | untestable | untestable | — |
| gemini → anthropic | added_gap_p50_us | untestable | untestable | — |
| gemini → anthropic | added_gap_p99_us | untestable | untestable | — |
| gemini → anthropic | streams_sustained_fps | untestable | untestable | — |
| gemini → anthropic | streams_sustained | untestable | untestable | — |
| gemini → anthropic | idle_rss_mib | 591.91796875 | valid | — |
| gemini → anthropic | steady_state_rss_mib | 671.87109375 | valid | — |
| gemini → anthropic | recovered_rss_mib | 671.87109375 | valid | — |
| gemini → anthropic | peak_rss_mib | 671.87109375 | valid | — |
| gemini → anthropic | peak_rss_hwm_mib | 671.87109375 | valid | — |
| gemini → anthropic | time_to_plateau_s | 63 | valid | — |
| gemini → anthropic | growth_rate_mib_per_min | 0 | valid | — |
| gemini → anthropic | load_s | 305 | valid | — |
| gemini → anthropic | shape | not_measured | not_measured | — |
| gemini → anthropic | idle_shape | not_measured | not_measured | — |
| gemini → anthropic | idle_static | 1 | valid | — |
| gemini → anthropic | idle_growth_rate_mib_per_min | 0 | valid | — |
| gemini → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| gemini → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| gemini → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| gemini → anthropic | throughput_rps | 60 req/s | concurrency 2; samples 362 | 50 |
| gemini → anthropic | throughput_rps | 60 req/s | concurrency 2; samples 362 | 100 |
| gemini → anthropic | throughput_rps | 60 req/s | concurrency 2; samples 362 | unbounded |
| openai → anthropic | added_latency_p50_us | 10313 | valid; gateway_c1_samples=579; direct_c1_samples=265135 | — |
| openai → anthropic | added_latency_p99_us | 11125 | valid; gateway_c1_samples=579; direct_c1_samples=265135 | — |
| openai → anthropic | gateway_c1_p99_us | 11153 | valid; gateway_c1_samples=579 | — |
| openai → anthropic | direct_c1_p99_us | 28 | valid; direct_c1_samples=265135 | — |
| openai → anthropic | cpu_us_per_request | 10411.663807890223 | valid | — |
| openai → anthropic | rps_per_cpu_second | 96.04612850082371 | valid | — |
| openai → anthropic | cost_window_conc | 8 | valid | — |
| openai → anthropic | cost_core_utilisation | 0.2500487063572403 | valid | — |
| openai → anthropic | cost_window_ok | 583 | valid | — |
| openai → anthropic | cost_window_rps | 97 | valid | — |
| openai → anthropic | cost_threads | 28 | valid | — |
| openai → anthropic | cost_nonvol_ctxt_per_request | 0.03773584905660377 | valid | — |
| openai → anthropic | cost_majflt | 0 | valid | — |
| openai → anthropic | added_ttft_p50_us | 13428 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → anthropic | added_ttft_p99_us | 14151 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → anthropic | added_gap_p50_us | 3 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → anthropic | added_gap_p99_us | 1348 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → anthropic | streams_sustained_fps | 1232.8623220270053 | valid | — |
| openai → anthropic | streams_sustained | 117 | valid | — |
| openai → anthropic | idle_rss_mib | 592.48828125 | valid | — |
| openai → anthropic | steady_state_rss_mib | 667.12109375 | valid | — |
| openai → anthropic | recovered_rss_mib | 667.12109375 | valid | — |
| openai → anthropic | peak_rss_mib | 667.12109375 | valid | — |
| openai → anthropic | peak_rss_hwm_mib | 667.12109375 | valid | — |
| openai → anthropic | time_to_plateau_s | 62 | valid | — |
| openai → anthropic | growth_rate_mib_per_min | 0 | valid | — |
| openai → anthropic | load_s | 304 | valid | — |
| openai → anthropic | shape | not_measured | not_measured | — |
| openai → anthropic | idle_shape | not_measured | not_measured | — |
| openai → anthropic | idle_static | 1 | valid | — |
| openai → anthropic | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → anthropic | throughput_rps | 97 req/s | concurrency 1; samples 586 | 50 |
| openai → anthropic | throughput_rps | 98 req/s | concurrency 8; samples 589 | 100 |
| openai → anthropic | throughput_rps | 98 req/s | concurrency 8; samples 589 | unbounded |
| anthropic → anthropic | added_latency_p50_us | 10994 | valid; gateway_c1_samples=544; direct_c1_samples=263642 | — |
| anthropic → anthropic | added_latency_p99_us | 11745 | valid; gateway_c1_samples=544; direct_c1_samples=263642 | — |
| anthropic → anthropic | gateway_c1_p99_us | 11773 | valid; gateway_c1_samples=544 | — |
| anthropic → anthropic | direct_c1_p99_us | 28 | valid; direct_c1_samples=263642 | — |
| anthropic → anthropic | cpu_us_per_request | 11322.160148975792 | valid | — |
| anthropic → anthropic | rps_per_cpu_second | 88.32236842105263 | valid | — |
| anthropic → anthropic | cost_window_conc | 8 | valid | — |
| anthropic → anthropic | cost_core_utilisation | 0.2501294799366864 | valid | — |
| anthropic → anthropic | cost_window_ok | 537 | valid | — |
| anthropic → anthropic | cost_window_rps | 89 | valid | — |
| anthropic → anthropic | cost_threads | 268 | valid | — |
| anthropic → anthropic | cost_nonvol_ctxt_per_request | 0.03910614525139665 | valid | — |
| anthropic → anthropic | cost_majflt | 0 | valid | — |
| anthropic → anthropic | added_ttft_p50_us | 14510 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| anthropic → anthropic | added_ttft_p99_us | 15815 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| anthropic → anthropic | added_gap_p50_us | 0 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| anthropic → anthropic | added_gap_p99_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| anthropic → anthropic | streams_sustained_fps | 3187.580899184528 | valid | — |
| anthropic → anthropic | streams_sustained | 255 | valid | — |
| anthropic → anthropic | idle_rss_mib | 592.48828125 | valid | — |
| anthropic → anthropic | steady_state_rss_mib | 658.05859375 | valid | — |
| anthropic → anthropic | recovered_rss_mib | 658.05859375 | valid | — |
| anthropic → anthropic | peak_rss_mib | 658.05859375 | valid | — |
| anthropic → anthropic | peak_rss_hwm_mib | 658.05859375 | valid | — |
| anthropic → anthropic | time_to_plateau_s | 62 | valid | — |
| anthropic → anthropic | growth_rate_mib_per_min | 0 | valid | — |
| anthropic → anthropic | load_s | 300 | valid | — |
| anthropic → anthropic | shape | not_measured | not_measured | — |
| anthropic → anthropic | idle_shape | not_measured | not_measured | — |
| anthropic → anthropic | idle_static | 1 | valid | — |
| anthropic → anthropic | idle_growth_rate_mib_per_min | 0 | valid | — |
| anthropic → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| anthropic → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| anthropic → anthropic | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| anthropic → anthropic | throughput_rps | 89 req/s | concurrency 2; samples 538 | 50 |
| anthropic → anthropic | throughput_rps | 89 req/s | concurrency 2; samples 538 | 100 |
| anthropic → anthropic | throughput_rps | 91 req/s | concurrency 64; samples 550 | unbounded |
| openai-responses → bedrock | added_latency_p50_us | 17646 | valid; gateway_c1_samples=339; direct_c1_samples=267586 | — |
| openai-responses → bedrock | added_latency_p99_us | 18361 | valid; gateway_c1_samples=339; direct_c1_samples=267586 | — |
| openai-responses → bedrock | gateway_c1_p99_us | 18388 | valid; gateway_c1_samples=339 | — |
| openai-responses → bedrock | direct_c1_p99_us | 27 | valid; direct_c1_samples=267586 | — |
| openai-responses → bedrock | cpu_us_per_request | 18244.04761904762 | valid | — |
| openai-responses → bedrock | rps_per_cpu_second | 54.81239804241436 | valid | — |
| openai-responses → bedrock | cost_window_conc | 8 | valid | — |
| openai-responses → bedrock | cost_core_utilisation | 0.2502884887591485 | valid | — |
| openai-responses → bedrock | cost_window_ok | 336 | valid | — |
| openai-responses → bedrock | cost_window_rps | 55 | valid | — |
| openai-responses → bedrock | cost_threads | 17 | valid | — |
| openai-responses → bedrock | cost_nonvol_ctxt_per_request | 0.0744047619047619 | valid | — |
| openai-responses → bedrock | cost_majflt | 0 | valid | — |
| openai-responses → bedrock | added_ttft_p50_us | untestable | untestable | — |
| openai-responses → bedrock | added_ttft_p99_us | untestable | untestable | — |
| openai-responses → bedrock | added_gap_p50_us | untestable | untestable | — |
| openai-responses → bedrock | added_gap_p99_us | untestable | untestable | — |
| openai-responses → bedrock | streams_sustained_fps | untestable | untestable | — |
| openai-responses → bedrock | streams_sustained | untestable | untestable | — |
| openai-responses → bedrock | idle_rss_mib | 592.97265625 | valid | — |
| openai-responses → bedrock | steady_state_rss_mib | 689.4375 | valid | — |
| openai-responses → bedrock | recovered_rss_mib | 689.4375 | valid | — |
| openai-responses → bedrock | peak_rss_mib | 689.4375 | valid | — |
| openai-responses → bedrock | peak_rss_hwm_mib | 689.4375 | valid | — |
| openai-responses → bedrock | time_to_plateau_s | 64 | valid | — |
| openai-responses → bedrock | growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → bedrock | load_s | 301 | valid | — |
| openai-responses → bedrock | shape | not_measured | not_measured | — |
| openai-responses → bedrock | idle_shape | not_measured | not_measured | — |
| openai-responses → bedrock | idle_static | 1 | valid | — |
| openai-responses → bedrock | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai-responses → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai-responses → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai-responses → bedrock | throughput_rps | 56 req/s | concurrency 1; samples 338 | 50 |
| openai-responses → bedrock | throughput_rps | 56 req/s | concurrency 1; samples 338 | 100 |
| openai-responses → bedrock | throughput_rps | 57 req/s | concurrency 2048; samples 345 | unbounded |
| cohere → bedrock | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| anthropic → bedrock | added_latency_p50_us | 16315 | valid; gateway_c1_samples=366; direct_c1_samples=262394 | — |
| anthropic → bedrock | added_latency_p99_us | 17156 | valid; gateway_c1_samples=366; direct_c1_samples=262394 | — |
| anthropic → bedrock | gateway_c1_p99_us | 17184 | valid; gateway_c1_samples=366 | — |
| anthropic → bedrock | direct_c1_p99_us | 28 | valid; direct_c1_samples=262394 | — |
| anthropic → bedrock | cpu_us_per_request | 18047.337278106508 | valid | — |
| anthropic → bedrock | rps_per_cpu_second | 55.40983606557377 | valid | — |
| anthropic → bedrock | cost_window_conc | 8 | valid | — |
| anthropic → bedrock | cost_core_utilisation | 0.25012322825186667 | valid | — |
| anthropic → bedrock | cost_window_ok | 338 | valid | — |
| anthropic → bedrock | cost_window_rps | 56 | valid | — |
| anthropic → bedrock | cost_threads | 17 | valid | — |
| anthropic → bedrock | cost_nonvol_ctxt_per_request | 0.07396449704142012 | valid | — |
| anthropic → bedrock | cost_majflt | 0 | valid | — |
| anthropic → bedrock | added_ttft_p50_us | untestable | untestable | — |
| anthropic → bedrock | added_ttft_p99_us | untestable | untestable | — |
| anthropic → bedrock | added_gap_p50_us | untestable | untestable | — |
| anthropic → bedrock | added_gap_p99_us | untestable | untestable | — |
| anthropic → bedrock | streams_sustained_fps | untestable | untestable | — |
| anthropic → bedrock | streams_sustained | untestable | untestable | — |
| anthropic → bedrock | idle_rss_mib | 592.72265625 | valid | — |
| anthropic → bedrock | steady_state_rss_mib | 687.4609375 | valid | — |
| anthropic → bedrock | recovered_rss_mib | 687.4609375 | valid | — |
| anthropic → bedrock | peak_rss_mib | 687.4609375 | valid | — |
| anthropic → bedrock | peak_rss_hwm_mib | 687.4609375 | valid | — |
| anthropic → bedrock | time_to_plateau_s | 63 | valid | — |
| anthropic → bedrock | growth_rate_mib_per_min | 0 | valid | — |
| anthropic → bedrock | load_s | 305 | valid | — |
| anthropic → bedrock | shape | not_measured | not_measured | — |
| anthropic → bedrock | idle_shape | not_measured | not_measured | — |
| anthropic → bedrock | idle_static | 1 | valid | — |
| anthropic → bedrock | idle_growth_rate_mib_per_min | 0 | valid | — |
| anthropic → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| anthropic → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| anthropic → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| anthropic → bedrock | throughput_rps | 62 req/s | concurrency 2; samples 376 | 50 |
| anthropic → bedrock | throughput_rps | 62 req/s | concurrency 2; samples 376 | 100 |
| anthropic → bedrock | throughput_rps | 63 req/s | concurrency 64; samples 382 | unbounded |
| gemini → bedrock | added_latency_p50_us | 21239 | valid; gateway_c1_samples=281; direct_c1_samples=267100 | — |
| gemini → bedrock | added_latency_p99_us | 22150 | valid; gateway_c1_samples=281; direct_c1_samples=267100 | — |
| gemini → bedrock | gateway_c1_p99_us | 22177 | valid; gateway_c1_samples=281 | — |
| gemini → bedrock | direct_c1_p99_us | 27 | valid; direct_c1_samples=267100 | — |
| gemini → bedrock | cpu_us_per_request | 22086.330935251797 | valid | — |
| gemini → bedrock | rps_per_cpu_second | 45.276872964169385 | valid | — |
| gemini → bedrock | cost_window_conc | 8 | valid | — |
| gemini → bedrock | cost_core_utilisation | 0.2506164650746462 | valid | — |
| gemini → bedrock | cost_window_ok | 278 | valid | — |
| gemini → bedrock | cost_window_rps | 46 | valid | — |
| gemini → bedrock | cost_threads | 17 | valid | — |
| gemini → bedrock | cost_nonvol_ctxt_per_request | 0.07913669064748201 | valid | — |
| gemini → bedrock | cost_majflt | 0 | valid | — |
| gemini → bedrock | added_ttft_p50_us | untestable | untestable | — |
| gemini → bedrock | added_ttft_p99_us | untestable | untestable | — |
| gemini → bedrock | added_gap_p50_us | untestable | untestable | — |
| gemini → bedrock | added_gap_p99_us | untestable | untestable | — |
| gemini → bedrock | streams_sustained_fps | untestable | untestable | — |
| gemini → bedrock | streams_sustained | untestable | untestable | — |
| gemini → bedrock | idle_rss_mib | 591.921875 | valid | — |
| gemini → bedrock | steady_state_rss_mib | 692.7734375 | valid | — |
| gemini → bedrock | recovered_rss_mib | 692.77734375 | valid | — |
| gemini → bedrock | peak_rss_mib | 692.77734375 | valid | — |
| gemini → bedrock | peak_rss_hwm_mib | 692.77734375 | valid | — |
| gemini → bedrock | time_to_plateau_s | 64 | valid | — |
| gemini → bedrock | growth_rate_mib_per_min | 0.005149277000641089 | valid | — |
| gemini → bedrock | load_s | 302 | valid | — |
| gemini → bedrock | shape | not_measured | not_measured | — |
| gemini → bedrock | idle_shape | not_measured | not_measured | — |
| gemini → bedrock | idle_static | 1 | valid | — |
| gemini → bedrock | idle_growth_rate_mib_per_min | 0 | valid | — |
| gemini → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| gemini → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| gemini → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| gemini → bedrock | throughput_rps | 46 req/s | concurrency 1; samples 280 | 50 |
| gemini → bedrock | throughput_rps | 46 req/s | concurrency 1; samples 280 | 100 |
| gemini → bedrock | throughput_rps | 46 req/s | concurrency 1; samples 280 | unbounded |
| bedrock → bedrock | added_latency_p50_us | 10032 | valid; gateway_c1_samples=595; direct_c1_samples=267885 | — |
| bedrock → bedrock | added_latency_p99_us | 10847 | valid; gateway_c1_samples=595; direct_c1_samples=267885 | — |
| bedrock → bedrock | gateway_c1_p99_us | 10874 | valid; gateway_c1_samples=595 | — |
| bedrock → bedrock | direct_c1_p99_us | 27 | valid; direct_c1_samples=267885 | — |
| bedrock → bedrock | cpu_us_per_request | 10134.003350083753 | valid | — |
| bedrock → bedrock | rps_per_cpu_second | 98.67768595041322 | valid | — |
| bedrock → bedrock | cost_window_conc | 8 | valid | — |
| bedrock → bedrock | cost_core_utilisation | 0.24972426540447598 | valid | — |
| bedrock → bedrock | cost_window_ok | 597 | valid | — |
| bedrock → bedrock | cost_window_rps | 99 | valid | — |
| bedrock → bedrock | cost_threads | 13 | valid | — |
| bedrock → bedrock | cost_nonvol_ctxt_per_request | 0.04522613065326633 | valid | — |
| bedrock → bedrock | cost_majflt | 0 | valid | — |
| bedrock → bedrock | added_ttft_p50_us | untestable | untestable | — |
| bedrock → bedrock | added_ttft_p99_us | untestable | untestable | — |
| bedrock → bedrock | added_gap_p50_us | untestable | untestable | — |
| bedrock → bedrock | added_gap_p99_us | untestable | untestable | — |
| bedrock → bedrock | streams_sustained_fps | untestable | untestable | — |
| bedrock → bedrock | streams_sustained | untestable | untestable | — |
| bedrock → bedrock | idle_rss_mib | 592.5078125 | valid | — |
| bedrock → bedrock | steady_state_rss_mib | 671.40234375 | valid | — |
| bedrock → bedrock | recovered_rss_mib | 671.40234375 | valid | — |
| bedrock → bedrock | peak_rss_mib | 671.40234375 | valid | — |
| bedrock → bedrock | peak_rss_hwm_mib | 671.40234375 | valid | — |
| bedrock → bedrock | time_to_plateau_s | 61 | valid | — |
| bedrock → bedrock | growth_rate_mib_per_min | 0 | valid | — |
| bedrock → bedrock | load_s | 303 | valid | — |
| bedrock → bedrock | shape | not_measured | not_measured | — |
| bedrock → bedrock | idle_shape | not_measured | not_measured | — |
| bedrock → bedrock | idle_static | 1 | valid | — |
| bedrock → bedrock | idle_growth_rate_mib_per_min | 0 | valid | — |
| bedrock → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| bedrock → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| bedrock → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| bedrock → bedrock | throughput_rps | 101 req/s | concurrency 2; samples 607 | 50 |
| bedrock → bedrock | throughput_rps | 101 req/s | concurrency 2; samples 607 | 100 |
| bedrock → bedrock | throughput_rps | 104 req/s | concurrency 128; samples 625 | unbounded |
| openai → bedrock | added_latency_p50_us | 14005 | valid; gateway_c1_samples=427; direct_c1_samples=272464 | — |
| openai → bedrock | added_latency_p99_us | 14745 | valid; gateway_c1_samples=427; direct_c1_samples=272464 | — |
| openai → bedrock | gateway_c1_p99_us | 14772 | valid; gateway_c1_samples=427 | — |
| openai → bedrock | direct_c1_p99_us | 27 | valid; direct_c1_samples=272464 | — |
| openai → bedrock | cpu_us_per_request | 14262.295081967213 | valid | — |
| openai → bedrock | rps_per_cpu_second | 70.11494252873564 | valid | — |
| openai → bedrock | cost_window_conc | 8 | valid | — |
| openai → bedrock | cost_core_utilisation | 0.24985541302136363 | valid | — |
| openai → bedrock | cost_window_ok | 427 | valid | — |
| openai → bedrock | cost_window_rps | 71 | valid | — |
| openai → bedrock | cost_threads | 17 | valid | — |
| openai → bedrock | cost_nonvol_ctxt_per_request | 0.0468384074941452 | valid | — |
| openai → bedrock | cost_majflt | 0 | valid | — |
| openai → bedrock | added_ttft_p50_us | untestable | untestable | — |
| openai → bedrock | added_ttft_p99_us | untestable | untestable | — |
| openai → bedrock | added_gap_p50_us | untestable | untestable | — |
| openai → bedrock | added_gap_p99_us | untestable | untestable | — |
| openai → bedrock | streams_sustained_fps | untestable | untestable | — |
| openai → bedrock | streams_sustained | untestable | untestable | — |
| openai → bedrock | idle_rss_mib | 593.1875 | valid | — |
| openai → bedrock | steady_state_rss_mib | 689.1015625 | valid | — |
| openai → bedrock | recovered_rss_mib | 689.1015625 | valid | — |
| openai → bedrock | peak_rss_mib | 689.1015625 | valid | — |
| openai → bedrock | peak_rss_hwm_mib | 689.1015625 | valid | — |
| openai → bedrock | time_to_plateau_s | 63 | valid | — |
| openai → bedrock | growth_rate_mib_per_min | 0 | valid | — |
| openai → bedrock | load_s | 302 | valid | — |
| openai → bedrock | shape | not_measured | not_measured | — |
| openai → bedrock | idle_shape | not_measured | not_measured | — |
| openai → bedrock | idle_static | 1 | valid | — |
| openai → bedrock | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → bedrock | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → bedrock | throughput_rps | 72 req/s | concurrency 2; samples 434 | 50 |
| openai → bedrock | throughput_rps | 72 req/s | concurrency 2; samples 434 | 100 |
| openai → bedrock | throughput_rps | 72 req/s | concurrency 2; samples 434 | unbounded |
| openai → openai | added_latency_p50_us | 8512 | valid; gateway_c1_samples=701; direct_c1_samples=267297 | — |
| openai → openai | added_latency_p99_us | 9522 | valid; gateway_c1_samples=701; direct_c1_samples=267297 | — |
| openai → openai | gateway_c1_p99_us | 9549 | valid; gateway_c1_samples=701 | — |
| openai → openai | direct_c1_p99_us | 27 | valid; direct_c1_samples=267297 | — |
| openai → openai | cpu_us_per_request | 8644.793152639088 | valid | — |
| openai → openai | rps_per_cpu_second | 115.67656765676568 | valid | — |
| openai → openai | cost_window_conc | 8 | valid | — |
| openai → openai | cost_core_utilisation | 0.24990558455671374 | valid | — |
| openai → openai | cost_window_ok | 701 | valid | — |
| openai → openai | cost_window_rps | 116 | valid | — |
| openai → openai | cost_threads | 32 | valid | — |
| openai → openai | cost_nonvol_ctxt_per_request | 0.028530670470756064 | valid | — |
| openai → openai | cost_majflt | 0 | valid | — |
| openai → openai | added_ttft_p50_us | 11154 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_ttft_p99_us | 11974 | valid; ttft_gw_samples=100; ttft_direct_samples=100 | — |
| openai → openai | added_gap_p50_us | below_resolution | below_resolution; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | added_gap_p99_us | 9 | valid; gap_gw_samples=63; gap_direct_samples=63 | — |
| openai → openai | streams_sustained_fps | 1207.9948436569616 | valid | — |
| openai → openai | streams_sustained | 62 | valid | — |
| openai → openai | idle_rss_mib | 592.95703125 | valid | — |
| openai → openai | steady_state_rss_mib | 628.60546875 | valid | — |
| openai → openai | recovered_rss_mib | 628.60546875 | valid | — |
| openai → openai | peak_rss_mib | 628.60546875 | valid | — |
| openai → openai | peak_rss_hwm_mib | 628.60546875 | valid | — |
| openai → openai | time_to_plateau_s | 61 | valid | — |
| openai → openai | growth_rate_mib_per_min | 0 | valid | — |
| openai → openai | load_s | 302 | valid | — |
| openai → openai | shape | not_measured | not_measured | — |
| openai → openai | idle_shape | not_measured | not_measured | — |
| openai → openai | idle_static | 1 | valid | — |
| openai → openai | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → openai | throughput_rps | 117 req/s | concurrency 1; samples 708 | 10 |
| openai → openai | throughput_rps | 119 req/s | concurrency 2; samples 718 | 50 |
| openai → openai | throughput_rps | 119 req/s | concurrency 2; samples 718 | 100 |
| openai → openai | throughput_rps | 119 req/s | concurrency 2; samples 718 | unbounded |
| anthropic → openai | — | — | not_served: probed and answered HTTP 200, but the body is not a usable anthropic response (see body_snippet) | — |
| openai-responses → openai | added_latency_p50_us | 10676 | valid; gateway_c1_samples=559; direct_c1_samples=265389 | — |
| openai-responses → openai | added_latency_p99_us | 11606 | valid; gateway_c1_samples=559; direct_c1_samples=265389 | — |
| openai-responses → openai | gateway_c1_p99_us | 11634 | valid; gateway_c1_samples=559 | — |
| openai-responses → openai | direct_c1_p99_us | 28 | valid; direct_c1_samples=265389 | — |
| openai-responses → openai | cpu_us_per_request | 10555.555555555555 | valid | — |
| openai-responses → openai | rps_per_cpu_second | 94.73684210526315 | valid | — |
| openai-responses → openai | cost_window_conc | 8 | valid | — |
| openai-responses → openai | cost_core_utilisation | 0.2503730077247171 | valid | — |
| openai-responses → openai | cost_window_ok | 576 | valid | — |
| openai-responses → openai | cost_window_rps | 95 | valid | — |
| openai-responses → openai | cost_threads | 13 | valid | — |
| openai-responses → openai | cost_nonvol_ctxt_per_request | 0.059027777777777776 | valid | — |
| openai-responses → openai | cost_majflt | 0 | valid | — |
| openai-responses → openai | added_ttft_p50_us | untestable | untestable | — |
| openai-responses → openai | added_ttft_p99_us | untestable | untestable | — |
| openai-responses → openai | added_gap_p50_us | untestable | untestable | — |
| openai-responses → openai | added_gap_p99_us | untestable | untestable | — |
| openai-responses → openai | streams_sustained_fps | untestable | untestable | — |
| openai-responses → openai | streams_sustained | untestable | untestable | — |
| openai-responses → openai | idle_rss_mib | 591.90625 | valid | — |
| openai-responses → openai | steady_state_rss_mib | 629.95703125 | valid | — |
| openai-responses → openai | recovered_rss_mib | 629.96875 | valid | — |
| openai-responses → openai | peak_rss_mib | 629.96875 | valid | — |
| openai-responses → openai | peak_rss_hwm_mib | 629.96875 | valid | — |
| openai-responses → openai | time_to_plateau_s | 62 | valid | — |
| openai-responses → openai | growth_rate_mib_per_min | 0.00555168581824748 | valid | — |
| openai-responses → openai | load_s | 304 | valid | — |
| openai-responses → openai | shape | not_measured | not_measured | — |
| openai-responses → openai | idle_shape | not_measured | not_measured | — |
| openai-responses → openai | idle_static | 1 | valid | — |
| openai-responses → openai | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai-responses → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai-responses → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai-responses → openai | throughput_rps | 95 req/s | concurrency 2; samples 572 | 50 |
| openai-responses → openai | throughput_rps | 95 req/s | concurrency 2; samples 572 | 100 |
| openai-responses → openai | throughput_rps | 95 req/s | concurrency 2; samples 572 | unbounded |
| cohere → openai | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| bedrock → openai | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| gemini → openai | added_latency_p50_us | 14407 | valid; gateway_c1_samples=412; direct_c1_samples=263451 | — |
| gemini → openai | added_latency_p99_us | 15293 | valid; gateway_c1_samples=412; direct_c1_samples=263451 | — |
| gemini → openai | gateway_c1_p99_us | 15321 | valid; gateway_c1_samples=412 | — |
| gemini → openai | direct_c1_p99_us | 28 | valid; direct_c1_samples=263451 | — |
| gemini → openai | cpu_us_per_request | 15086.41975308642 | valid | — |
| gemini → openai | rps_per_cpu_second | 66.28477905073649 | valid | — |
| gemini → openai | cost_window_conc | 8 | valid | — |
| gemini → openai | cost_core_utilisation | 0.25038840963155784 | valid | — |
| gemini → openai | cost_window_ok | 405 | valid | — |
| gemini → openai | cost_window_rps | 67 | valid | — |
| gemini → openai | cost_threads | 13 | valid | — |
| gemini → openai | cost_nonvol_ctxt_per_request | 0.044444444444444446 | valid | — |
| gemini → openai | cost_majflt | 0 | valid | — |
| gemini → openai | added_ttft_p50_us | untestable | untestable | — |
| gemini → openai | added_ttft_p99_us | untestable | untestable | — |
| gemini → openai | added_gap_p50_us | untestable | untestable | — |
| gemini → openai | added_gap_p99_us | untestable | untestable | — |
| gemini → openai | streams_sustained_fps | untestable | untestable | — |
| gemini → openai | streams_sustained | untestable | untestable | — |
| gemini → openai | idle_rss_mib | 592.00390625 | valid | — |
| gemini → openai | steady_state_rss_mib | 632.30859375 | valid | — |
| gemini → openai | recovered_rss_mib | 632.31640625 | valid | — |
| gemini → openai | peak_rss_mib | 632.31640625 | valid | — |
| gemini → openai | peak_rss_hwm_mib | 632.31640625 | valid | — |
| gemini → openai | time_to_plateau_s | 63 | valid | — |
| gemini → openai | growth_rate_mib_per_min | 0.003008508378215942 | valid | — |
| gemini → openai | load_s | 302 | valid | — |
| gemini → openai | shape | not_measured | not_measured | — |
| gemini → openai | idle_shape | not_measured | not_measured | — |
| gemini → openai | idle_static | 1 | valid | — |
| gemini → openai | idle_growth_rate_mib_per_min | 0 | valid | — |
| gemini → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| gemini → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| gemini → openai | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| gemini → openai | throughput_rps | 68 req/s | concurrency 1; samples 410 | 50 |
| gemini → openai | throughput_rps | 68 req/s | concurrency 1; samples 410 | 100 |
| gemini → openai | throughput_rps | 68 req/s | concurrency 1; samples 410 | unbounded |
| cohere → openai-responses | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| bedrock → openai-responses | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| gemini → openai-responses | added_latency_p50_us | 17577 | valid; gateway_c1_samples=338; direct_c1_samples=262004 | — |
| gemini → openai-responses | added_latency_p99_us | 18499 | valid; gateway_c1_samples=338; direct_c1_samples=262004 | — |
| gemini → openai-responses | gateway_c1_p99_us | 18527 | valid; gateway_c1_samples=338 | — |
| gemini → openai-responses | direct_c1_p99_us | 28 | valid; direct_c1_samples=262004 | — |
| gemini → openai-responses | cpu_us_per_request | 18005.865102639294 | valid | — |
| gemini → openai-responses | rps_per_cpu_second | 55.537459283387626 | valid | — |
| gemini → openai-responses | cost_window_conc | 8 | valid | — |
| gemini → openai-responses | cost_core_utilisation | 0.250691676136018 | valid | — |
| gemini → openai-responses | cost_window_ok | 341 | valid | — |
| gemini → openai-responses | cost_window_rps | 56 | valid | — |
| gemini → openai-responses | cost_threads | 13 | valid | — |
| gemini → openai-responses | cost_nonvol_ctxt_per_request | 0.08211143695014662 | valid | — |
| gemini → openai-responses | cost_majflt | 0 | valid | — |
| gemini → openai-responses | added_ttft_p50_us | untestable | untestable | — |
| gemini → openai-responses | added_ttft_p99_us | untestable | untestable | — |
| gemini → openai-responses | added_gap_p50_us | untestable | untestable | — |
| gemini → openai-responses | added_gap_p99_us | untestable | untestable | — |
| gemini → openai-responses | streams_sustained_fps | untestable | untestable | — |
| gemini → openai-responses | streams_sustained | untestable | untestable | — |
| gemini → openai-responses | idle_rss_mib | 592.40625 | valid | — |
| gemini → openai-responses | steady_state_rss_mib | 674.6484375 | valid | — |
| gemini → openai-responses | recovered_rss_mib | 674.6484375 | valid | — |
| gemini → openai-responses | peak_rss_mib | 674.6484375 | valid | — |
| gemini → openai-responses | peak_rss_hwm_mib | 674.6484375 | valid | — |
| gemini → openai-responses | time_to_plateau_s | 63 | valid | — |
| gemini → openai-responses | growth_rate_mib_per_min | 0 | valid | — |
| gemini → openai-responses | load_s | 306 | valid | — |
| gemini → openai-responses | shape | not_measured | not_measured | — |
| gemini → openai-responses | idle_shape | not_measured | not_measured | — |
| gemini → openai-responses | idle_static | 1 | valid | — |
| gemini → openai-responses | idle_growth_rate_mib_per_min | 0 | valid | — |
| gemini → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| gemini → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| gemini → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| gemini → openai-responses | throughput_rps | 56 req/s | concurrency 1; samples 337 | 50 |
| gemini → openai-responses | throughput_rps | 56 req/s | concurrency 1; samples 337 | 100 |
| gemini → openai-responses | throughput_rps | 59 req/s | concurrency 2048; samples 360 | unbounded |
| anthropic → openai-responses | added_latency_p50_us | 13287 | valid; gateway_c1_samples=449; direct_c1_samples=260854 | — |
| anthropic → openai-responses | added_latency_p99_us | 14010 | valid; gateway_c1_samples=449; direct_c1_samples=260854 | — |
| anthropic → openai-responses | gateway_c1_p99_us | 14038 | valid; gateway_c1_samples=449 | — |
| anthropic → openai-responses | direct_c1_p99_us | 28 | valid; direct_c1_samples=260854 | — |
| anthropic → openai-responses | cpu_us_per_request | 13738.738738738739 | valid | — |
| anthropic → openai-responses | rps_per_cpu_second | 72.78688524590164 | valid | — |
| anthropic → openai-responses | cost_window_conc | 8 | valid | — |
| anthropic → openai-responses | cost_core_utilisation | 0.25008979839096346 | valid | — |
| anthropic → openai-responses | cost_window_ok | 444 | valid | — |
| anthropic → openai-responses | cost_window_rps | 73 | valid | — |
| anthropic → openai-responses | cost_threads | 13 | valid | — |
| anthropic → openai-responses | cost_nonvol_ctxt_per_request | 0.07207207207207207 | valid | — |
| anthropic → openai-responses | cost_majflt | 0 | valid | — |
| anthropic → openai-responses | added_ttft_p50_us | untestable | untestable | — |
| anthropic → openai-responses | added_ttft_p99_us | untestable | untestable | — |
| anthropic → openai-responses | added_gap_p50_us | untestable | untestable | — |
| anthropic → openai-responses | added_gap_p99_us | untestable | untestable | — |
| anthropic → openai-responses | streams_sustained_fps | untestable | untestable | — |
| anthropic → openai-responses | streams_sustained | untestable | untestable | — |
| anthropic → openai-responses | idle_rss_mib | 591.65234375 | valid | — |
| anthropic → openai-responses | steady_state_rss_mib | 661.2265625 | valid | — |
| anthropic → openai-responses | recovered_rss_mib | 661.2265625 | valid | — |
| anthropic → openai-responses | peak_rss_mib | 661.2265625 | valid | — |
| anthropic → openai-responses | peak_rss_hwm_mib | 661.2265625 | valid | — |
| anthropic → openai-responses | time_to_plateau_s | 63 | valid | — |
| anthropic → openai-responses | growth_rate_mib_per_min | 0 | valid | — |
| anthropic → openai-responses | load_s | 303 | valid | — |
| anthropic → openai-responses | shape | not_measured | not_measured | — |
| anthropic → openai-responses | idle_shape | not_measured | not_measured | — |
| anthropic → openai-responses | idle_static | 1 | valid | — |
| anthropic → openai-responses | idle_growth_rate_mib_per_min | 0 | valid | — |
| anthropic → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| anthropic → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| anthropic → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| anthropic → openai-responses | throughput_rps | 74 req/s | concurrency 1; samples 450 | 50 |
| anthropic → openai-responses | throughput_rps | 74 req/s | concurrency 1; samples 450 | 100 |
| anthropic → openai-responses | throughput_rps | 74 req/s | concurrency 1; samples 450 | unbounded |
| openai → openai-responses | added_latency_p50_us | 10726 | valid; gateway_c1_samples=557; direct_c1_samples=261925 | — |
| openai → openai-responses | added_latency_p99_us | 11474 | valid; gateway_c1_samples=557; direct_c1_samples=261925 | — |
| openai → openai-responses | gateway_c1_p99_us | 11502 | valid; gateway_c1_samples=557 | — |
| openai → openai-responses | direct_c1_p99_us | 28 | valid; direct_c1_samples=261925 | — |
| openai → openai-responses | cpu_us_per_request | 10799.289520426288 | valid | — |
| openai → openai-responses | rps_per_cpu_second | 92.59868421052632 | valid | — |
| openai → openai-responses | cost_window_conc | 8 | valid | — |
| openai → openai-responses | cost_core_utilisation | 0.2502330796843856 | valid | — |
| openai → openai-responses | cost_window_ok | 563 | valid | — |
| openai → openai-responses | cost_window_rps | 93 | valid | — |
| openai → openai-responses | cost_threads | 13 | valid | — |
| openai → openai-responses | cost_nonvol_ctxt_per_request | 0.05328596802841918 | valid | — |
| openai → openai-responses | cost_majflt | 0 | valid | — |
| openai → openai-responses | added_ttft_p50_us | untestable | untestable | — |
| openai → openai-responses | added_ttft_p99_us | untestable | untestable | — |
| openai → openai-responses | added_gap_p50_us | untestable | untestable | — |
| openai → openai-responses | added_gap_p99_us | untestable | untestable | — |
| openai → openai-responses | streams_sustained_fps | untestable | untestable | — |
| openai → openai-responses | streams_sustained | untestable | untestable | — |
| openai → openai-responses | idle_rss_mib | 591.7890625 | valid | — |
| openai → openai-responses | steady_state_rss_mib | 669.40234375 | valid | — |
| openai → openai-responses | recovered_rss_mib | 669.40234375 | valid | — |
| openai → openai-responses | peak_rss_mib | 669.40234375 | valid | — |
| openai → openai-responses | peak_rss_hwm_mib | 669.40234375 | valid | — |
| openai → openai-responses | time_to_plateau_s | 62 | valid | — |
| openai → openai-responses | growth_rate_mib_per_min | 0 | valid | — |
| openai → openai-responses | load_s | 305 | valid | — |
| openai → openai-responses | shape | not_measured | not_measured | — |
| openai → openai-responses | idle_shape | not_measured | not_measured | — |
| openai → openai-responses | idle_static | 1 | valid | — |
| openai → openai-responses | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai → openai-responses | throughput_rps | 92 req/s | concurrency 1; samples 556 | 50 |
| openai → openai-responses | throughput_rps | 92 req/s | concurrency 1; samples 556 | 100 |
| openai → openai-responses | throughput_rps | 93 req/s | concurrency 16; samples 559 | unbounded |
| openai-responses → openai-responses | added_latency_p50_us | 11554 | valid; gateway_c1_samples=518; direct_c1_samples=265921 | — |
| openai-responses → openai-responses | added_latency_p99_us | 12280 | valid; gateway_c1_samples=518; direct_c1_samples=265921 | — |
| openai-responses → openai-responses | gateway_c1_p99_us | 12308 | valid; gateway_c1_samples=518 | — |
| openai-responses → openai-responses | direct_c1_p99_us | 28 | valid; direct_c1_samples=265921 | — |
| openai-responses → openai-responses | cpu_us_per_request | 11692.307692307691 | valid | — |
| openai-responses → openai-responses | rps_per_cpu_second | 85.52631578947368 | valid | — |
| openai-responses → openai-responses | cost_window_conc | 8 | valid | — |
| openai-responses → openai-responses | cost_core_utilisation | 0.2497786700112885 | valid | — |
| openai-responses → openai-responses | cost_window_ok | 520 | valid | — |
| openai-responses → openai-responses | cost_window_rps | 86 | valid | — |
| openai-responses → openai-responses | cost_threads | 13 | valid | — |
| openai-responses → openai-responses | cost_nonvol_ctxt_per_request | 0.06346153846153846 | valid | — |
| openai-responses → openai-responses | cost_majflt | 0 | valid | — |
| openai-responses → openai-responses | added_ttft_p50_us | untestable | untestable | — |
| openai-responses → openai-responses | added_ttft_p99_us | untestable | untestable | — |
| openai-responses → openai-responses | added_gap_p50_us | untestable | untestable | — |
| openai-responses → openai-responses | added_gap_p99_us | untestable | untestable | — |
| openai-responses → openai-responses | streams_sustained_fps | untestable | untestable | — |
| openai-responses → openai-responses | streams_sustained | untestable | untestable | — |
| openai-responses → openai-responses | idle_rss_mib | 592.46484375 | valid | — |
| openai-responses → openai-responses | steady_state_rss_mib | 627.4453125 | valid | — |
| openai-responses → openai-responses | recovered_rss_mib | 629.47265625 | valid | — |
| openai-responses → openai-responses | peak_rss_mib | 629.47265625 | valid | — |
| openai-responses → openai-responses | peak_rss_hwm_mib | 629.47265625 | valid | — |
| openai-responses → openai-responses | time_to_plateau_s | 62 | valid | — |
| openai-responses → openai-responses | growth_rate_mib_per_min | 3.0355931092621313 | valid | — |
| openai-responses → openai-responses | load_s | 300 | valid | — |
| openai-responses → openai-responses | shape | not_measured | not_measured | — |
| openai-responses → openai-responses | idle_shape | not_measured | not_measured | — |
| openai-responses → openai-responses | idle_static | 1 | valid | — |
| openai-responses → openai-responses | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai-responses → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai-responses → openai-responses | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai-responses → openai-responses | throughput_rps | 87 req/s | concurrency 2; samples 524 | 50 |
| openai-responses → openai-responses | throughput_rps | 87 req/s | concurrency 2; samples 524 | 100 |
| openai-responses → openai-responses | throughput_rps | 88 req/s | concurrency 32; samples 529 | unbounded |
| openai-responses → gemini | added_latency_p50_us | 10486 | valid; gateway_c1_samples=570; direct_c1_samples=265239 | — |
| openai-responses → gemini | added_latency_p99_us | 11201 | valid; gateway_c1_samples=570; direct_c1_samples=265239 | — |
| openai-responses → gemini | gateway_c1_p99_us | 11229 | valid; gateway_c1_samples=570 | — |
| openai-responses → gemini | direct_c1_p99_us | 28 | valid; direct_c1_samples=265239 | — |
| openai-responses → gemini | cpu_us_per_request | 10628.272251308901 | valid | — |
| openai-responses → gemini | rps_per_cpu_second | 94.08866995073892 | valid | — |
| openai-responses → gemini | cost_window_conc | 8 | valid | — |
| openai-responses → gemini | cost_core_utilisation | 0.25043632937727195 | valid | — |
| openai-responses → gemini | cost_window_ok | 573 | valid | — |
| openai-responses → gemini | cost_window_rps | 95 | valid | — |
| openai-responses → gemini | cost_threads | 13 | valid | — |
| openai-responses → gemini | cost_nonvol_ctxt_per_request | 0.041884816753926704 | valid | — |
| openai-responses → gemini | cost_majflt | 0 | valid | — |
| openai-responses → gemini | added_ttft_p50_us | untestable | untestable | — |
| openai-responses → gemini | added_ttft_p99_us | untestable | untestable | — |
| openai-responses → gemini | added_gap_p50_us | untestable | untestable | — |
| openai-responses → gemini | added_gap_p99_us | untestable | untestable | — |
| openai-responses → gemini | streams_sustained_fps | untestable | untestable | — |
| openai-responses → gemini | streams_sustained | untestable | untestable | — |
| openai-responses → gemini | idle_rss_mib | 592.390625 | valid | — |
| openai-responses → gemini | steady_state_rss_mib | 669.41015625 | valid | — |
| openai-responses → gemini | recovered_rss_mib | 669.4140625 | valid | — |
| openai-responses → gemini | peak_rss_mib | 669.4140625 | valid | — |
| openai-responses → gemini | peak_rss_hwm_mib | 669.4140625 | valid | — |
| openai-responses → gemini | time_to_plateau_s | 62 | valid | — |
| openai-responses → gemini | growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → gemini | load_s | 304 | valid | — |
| openai-responses → gemini | shape | not_measured | not_measured | — |
| openai-responses → gemini | idle_shape | not_measured | not_measured | — |
| openai-responses → gemini | idle_static | 1 | valid | — |
| openai-responses → gemini | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai-responses → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai-responses → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai-responses → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| openai-responses → gemini | throughput_rps | 96 req/s | concurrency 2; samples 578 | 50 |
| openai-responses → gemini | throughput_rps | 96 req/s | concurrency 2; samples 578 | 100 |
| openai-responses → gemini | throughput_rps | 96 req/s | concurrency 2; samples 578 | unbounded |
| gemini → gemini | added_latency_p50_us | 10073 | valid; gateway_c1_samples=592; direct_c1_samples=267671 | — |
| gemini → gemini | added_latency_p99_us | 10889 | valid; gateway_c1_samples=592; direct_c1_samples=267671 | — |
| gemini → gemini | gateway_c1_p99_us | 10916 | valid; gateway_c1_samples=592 | — |
| gemini → gemini | direct_c1_p99_us | 27 | valid; direct_c1_samples=267671 | — |
| gemini → gemini | cpu_us_per_request | 10236.486486486487 | valid | — |
| gemini → gemini | rps_per_cpu_second | 97.6897689768977 | valid | — |
| gemini → gemini | cost_window_conc | 8 | valid | — |
| gemini → gemini | cost_core_utilisation | 0.2501839420461313 | valid | — |
| gemini → gemini | cost_window_ok | 592 | valid | — |
| gemini → gemini | cost_window_rps | 98 | valid | — |
| gemini → gemini | cost_threads | 13 | valid | — |
| gemini → gemini | cost_nonvol_ctxt_per_request | 0.048986486486486486 | valid | — |
| gemini → gemini | cost_majflt | 0 | valid | — |
| gemini → gemini | added_ttft_p50_us | untestable | untestable | — |
| gemini → gemini | added_ttft_p99_us | untestable | untestable | — |
| gemini → gemini | added_gap_p50_us | untestable | untestable | — |
| gemini → gemini | added_gap_p99_us | untestable | untestable | — |
| gemini → gemini | streams_sustained_fps | untestable | untestable | — |
| gemini → gemini | streams_sustained | untestable | untestable | — |
| gemini → gemini | idle_rss_mib | 591.546875 | valid | — |
| gemini → gemini | steady_state_rss_mib | 627.6875 | valid | — |
| gemini → gemini | recovered_rss_mib | 627.6875 | valid | — |
| gemini → gemini | peak_rss_mib | 627.6875 | valid | — |
| gemini → gemini | peak_rss_hwm_mib | 627.6875 | valid | — |
| gemini → gemini | time_to_plateau_s | 61 | valid | — |
| gemini → gemini | growth_rate_mib_per_min | 0 | valid | — |
| gemini → gemini | load_s | 302 | valid | — |
| gemini → gemini | shape | not_measured | not_measured | — |
| gemini → gemini | idle_shape | not_measured | not_measured | — |
| gemini → gemini | idle_static | 1 | valid | — |
| gemini → gemini | idle_growth_rate_mib_per_min | 0 | valid | — |
| gemini → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| gemini → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| gemini → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 10ms, so this gateway carried no measurable throughput under that bound | 10 |
| gemini → gemini | throughput_rps | 101 req/s | concurrency 2; samples 612 | 50 |
| gemini → gemini | throughput_rps | 101 req/s | concurrency 2; samples 612 | 100 |
| gemini → gemini | throughput_rps | 101 req/s | concurrency 2; samples 612 | unbounded |
| cohere → gemini | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| bedrock → gemini | — | — | not_declared: Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json | — |
| openai → gemini | added_latency_p50_us | 8225 | valid; gateway_c1_samples=725; direct_c1_samples=265737 | — |
| openai → gemini | added_latency_p99_us | 8995 | valid; gateway_c1_samples=725; direct_c1_samples=265737 | — |
| openai → gemini | gateway_c1_p99_us | 9023 | valid; gateway_c1_samples=725 | — |
| openai → gemini | direct_c1_p99_us | 28 | valid; direct_c1_samples=265737 | — |
| openai → gemini | cpu_us_per_request | 8281.036834924966 | valid | — |
| openai → gemini | rps_per_cpu_second | 120.75782537067545 | valid | — |
| openai → gemini | cost_window_conc | 8 | valid | — |
| openai → gemini | cost_core_utilisation | 0.25021912323444456 | valid | — |
| openai → gemini | cost_window_ok | 733 | valid | — |
| openai → gemini | cost_window_rps | 122 | valid | — |
| openai → gemini | cost_threads | 13 | valid | — |
| openai → gemini | cost_nonvol_ctxt_per_request | 0.0286493860845839 | valid | — |
| openai → gemini | cost_majflt | 0 | valid | — |
| openai → gemini | added_ttft_p50_us | untestable | untestable | — |
| openai → gemini | added_ttft_p99_us | untestable | untestable | — |
| openai → gemini | added_gap_p50_us | untestable | untestable | — |
| openai → gemini | added_gap_p99_us | untestable | untestable | — |
| openai → gemini | streams_sustained_fps | untestable | untestable | — |
| openai → gemini | streams_sustained | untestable | untestable | — |
| openai → gemini | idle_rss_mib | 591.8359375 | valid | — |
| openai → gemini | steady_state_rss_mib | 665.40625 | valid | — |
| openai → gemini | recovered_rss_mib | 665.40625 | valid | — |
| openai → gemini | peak_rss_mib | 665.40625 | valid | — |
| openai → gemini | peak_rss_hwm_mib | 665.40625 | valid | — |
| openai → gemini | time_to_plateau_s | 61 | valid | — |
| openai → gemini | growth_rate_mib_per_min | 0 | valid | — |
| openai → gemini | load_s | 302 | valid | — |
| openai → gemini | shape | not_measured | not_measured | — |
| openai → gemini | idle_shape | not_measured | not_measured | — |
| openai → gemini | idle_static | 1 | valid | — |
| openai → gemini | idle_growth_rate_mib_per_min | 0 | valid | — |
| openai → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| openai → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| openai → gemini | throughput_rps | 121 req/s | concurrency 1; samples 732 | 10 |
| openai → gemini | throughput_rps | 123 req/s | concurrency 2; samples 740 | 50 |
| openai → gemini | throughput_rps | 123 req/s | concurrency 2; samples 740 | 100 |
| openai → gemini | throughput_rps | 127 req/s | concurrency 128; samples 767 | unbounded |
| anthropic → gemini | added_latency_p50_us | 8964 | valid; gateway_c1_samples=666; direct_c1_samples=263751 | — |
| anthropic → gemini | added_latency_p99_us | 9686 | valid; gateway_c1_samples=666; direct_c1_samples=263751 | — |
| anthropic → gemini | gateway_c1_p99_us | 9714 | valid; gateway_c1_samples=666 | — |
| anthropic → gemini | direct_c1_p99_us | 28 | valid; direct_c1_samples=263751 | — |
| anthropic → gemini | cpu_us_per_request | 9144.144144144144 | valid | — |
| anthropic → gemini | rps_per_cpu_second | 109.35960591133005 | valid | — |
| anthropic → gemini | cost_window_conc | 8 | valid | — |
| anthropic → gemini | cost_core_utilisation | 0.25073457424131973 | valid | — |
| anthropic → gemini | cost_window_ok | 666 | valid | — |
| anthropic → gemini | cost_window_rps | 110 | valid | — |
| anthropic → gemini | cost_threads | 13 | valid | — |
| anthropic → gemini | cost_nonvol_ctxt_per_request | 0.03453453453453453 | valid | — |
| anthropic → gemini | cost_majflt | 0 | valid | — |
| anthropic → gemini | added_ttft_p50_us | untestable | untestable | — |
| anthropic → gemini | added_ttft_p99_us | untestable | untestable | — |
| anthropic → gemini | added_gap_p50_us | untestable | untestable | — |
| anthropic → gemini | added_gap_p99_us | untestable | untestable | — |
| anthropic → gemini | streams_sustained_fps | untestable | untestable | — |
| anthropic → gemini | streams_sustained | untestable | untestable | — |
| anthropic → gemini | idle_rss_mib | 593.02734375 | valid | — |
| anthropic → gemini | steady_state_rss_mib | 666.55078125 | valid | — |
| anthropic → gemini | recovered_rss_mib | 666.5546875 | valid | — |
| anthropic → gemini | peak_rss_mib | 666.5546875 | valid | — |
| anthropic → gemini | peak_rss_hwm_mib | 666.5546875 | valid | — |
| anthropic → gemini | time_to_plateau_s | 61 | valid | — |
| anthropic → gemini | growth_rate_mib_per_min | 0.0057259229747044565 | valid | — |
| anthropic → gemini | load_s | 302 | valid | — |
| anthropic → gemini | shape | not_measured | not_measured | — |
| anthropic → gemini | idle_shape | not_measured | not_measured | — |
| anthropic → gemini | idle_static | 1 | valid | — |
| anthropic → gemini | idle_growth_rate_mib_per_min | 0 | valid | — |
| anthropic → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 1ms, so this gateway carried no measurable throughput under that bound | 1 |
| anthropic → gemini | throughput_rps | bound_not_met req/s | concurrency not recorded; every cleanly-served rung that reported a tail had it at or above 5ms, so this gateway carried no measurable throughput under that bound | 5 |
| anthropic → gemini | throughput_rps | 111 req/s | concurrency 1; samples 667 | 10 |
| anthropic → gemini | throughput_rps | 113 req/s | concurrency 2; samples 682 | 50 |
| anthropic → gemini | throughput_rps | 113 req/s | concurrency 2; samples 682 | 100 |
| anthropic → gemini | throughput_rps | 113 req/s | concurrency 2; samples 682 | unbounded |

## Probe evidence and instrument provenance

Historical capability notes may reference an earlier product version. They are not a new probe or a current support claim. Full RSS samples remain in the immutable raw JSON.

| Ingress → upstream | Cell state | Probe HTTP status | Recorded reason | Recorded detail | Evidence scope | Response excerpt | Tested build |
| --- | --- | --- | --- | --- | --- | --- | --- |
| cohere → cohere | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai → cohere | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| bedrock → cohere | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| anthropic → cohere | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai-responses → cohere | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| gemini → cohere | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| cohere → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| bedrock → anthropic | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai-responses → anthropic | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| gemini → anthropic | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai → anthropic | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| anthropic → anthropic | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai-responses → bedrock | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| cohere → bedrock | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| anthropic → bedrock | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| gemini → bedrock | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| bedrock → bedrock | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai → bedrock | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai → openai | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| anthropic → openai | not_served | 200 | failed | probed and answered HTTP 200, but the body is not a usable anthropic response (see body_snippet) | Recorded run evidence | `the 2xx body is not a response of the anthropic dialect with assistant text at content[type=text].text: {"id":"resp_bGl0ZWxsbTpjdXN0b21fbGxtX3Byb3ZpZGVyOm9wZW5haTttb2RlbF9pZDo2MTk3ZDQ1OWNlMzc1OTE2NzVlMTIyMTg3ZDAwOTViNzIyNDhmMTRmYTUwNzQzMDZkMzFlMzlhODkzMTM2OWQ4O3Jlc3BvbnNlX2lkOmNoYXRjbXBsLXg=","type":"message","role":"assistant","model":"gpt-4o-mini","stop_sequence":null,"usage":{"input_tokens":10,"output_tokens":2},"content":[],"stop_reason":"end_turn","stop_details":null}` | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai-responses → openai | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| cohere → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| bedrock → openai | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| gemini → openai | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| cohere → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| bedrock → openai-responses | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| gemini → openai-responses | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| anthropic → openai-responses | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai → openai-responses | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai-responses → openai-responses | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai-responses → gemini | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| gemini → gemini | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| cohere → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| bedrock → gemini | not_declared | — | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 1](https://onthebench.ai/#note-1) | [recorded note 2](https://onthebench.ai/#note-2) | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| openai → gemini | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |
| anthropic → gemini | served | — | — | — | Recorded run evidence | — | ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5 |

**Harness commit**: `d240badd966933fa19eec0606b2d1f3945f18ad2`

**Mock SHA-256**: `99d01f709d8d9c1f49055aa265d5eb7475264d6b513911fe317aee74a71190f8`

**Load generator SHA-256**: `not recorded`

**somaxconn**: 65535

**tcp_max_syn_backlog**: 65535

## Stream sweep diagnostics

Per streaming-sweep concurrency, as recorded by the engine: lanes by final HTTP status, and lanes that received no response.

| Ingress → upstream | Concurrency | Lanes by HTTP status | Lanes with no response |
| --- | --- | --- | --- |
| openai → anthropic | 1 | {"200":1} | 0 |
| openai → anthropic | 2 | {"200":2} | 0 |
| openai → anthropic | 4 | {"200":4} | 0 |
| openai → anthropic | 8 | {"200":8} | 0 |
| openai → anthropic | 16 | {"200":16} | 0 |
| openai → anthropic | 32 | {"200":32} | 0 |
| openai → anthropic | 64 | {"200":64} | 0 |
| openai → anthropic | 128 | {"200":128} | 0 |
| openai → anthropic | 96 | {"200":96} | 0 |
| openai → anthropic | 112 | {"200":112} | 0 |
| openai → anthropic | 120 | {"200":120} | 0 |
| openai → anthropic | 116 | {"200":116} | 0 |
| openai → anthropic | 118 | {"200":118} | 0 |
| openai → anthropic | 117 | {"200":117} | 0 |
| openai → anthropic | 117 | {"200":117} | 0 |
| openai → anthropic | 117 | {"200":117} | 0 |
| anthropic → anthropic | 1 | {"200":1} | 0 |
| anthropic → anthropic | 2 | {"200":2} | 0 |
| anthropic → anthropic | 4 | {"200":4} | 0 |
| anthropic → anthropic | 8 | {"200":8} | 0 |
| anthropic → anthropic | 16 | {"200":16} | 0 |
| anthropic → anthropic | 32 | {"200":32} | 0 |
| anthropic → anthropic | 64 | {"200":64} | 0 |
| anthropic → anthropic | 128 | {"200":128} | 0 |
| anthropic → anthropic | 256 | {"200":256} | 0 |
| anthropic → anthropic | 192 | {"200":192} | 0 |
| anthropic → anthropic | 224 | {"200":224} | 0 |
| anthropic → anthropic | 240 | {"200":240} | 0 |
| anthropic → anthropic | 248 | {"200":248} | 0 |
| anthropic → anthropic | 252 | {"200":252} | 0 |
| anthropic → anthropic | 254 | {"200":254} | 0 |
| anthropic → anthropic | 255 | {"200":255} | 0 |
| anthropic → anthropic | 255 | {"200":255} | 0 |
| anthropic → anthropic | 255 | {"200":255} | 0 |
| openai → openai | 1 | {"200":1} | 0 |
| openai → openai | 2 | {"200":2} | 0 |
| openai → openai | 4 | {"200":4} | 0 |
| openai → openai | 8 | {"200":8} | 0 |
| openai → openai | 16 | {"200":16} | 0 |
| openai → openai | 32 | {"200":32} | 0 |
| openai → openai | 64 | {"200":64} | 0 |
| openai → openai | 48 | {"200":48} | 0 |
| openai → openai | 56 | {"200":56} | 0 |
| openai → openai | 60 | {"200":60} | 0 |
| openai → openai | 62 | {"200":62} | 0 |
| openai → openai | 63 | {"200":63} | 0 |
| openai → openai | 62 | {"200":62} | 0 |
| openai → openai | 62 | {"200":62} | 0 |

## Recorded notes

Text recorded with the cells above. Each note is shown once; rows refer to it by number.

- Pinned v1.108.3 source b7d579fab2818968322ef96e5b8e98dc8178fafc: Chat, Responses, Messages and Gemini ingress translate to all six configured upstream wires. The per-model upstream setting selects Chat versus Responses without changing workload parameters. The default Messages-to-Chat bridge reaches the correct wire but returns empty content; it is configured as shipped, without a global bridge override, so its failure is recorded. Native Cohere and Bedrock ingress use the shipped /cohere and /bedrock provider-prefixed passthrough routes in litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py; passthrough is same-dialect only. Configured pairs that fail are recorded as failures. Evidence: results/diagnostics/refresh-configuration/litellm-python-final-config-correctness-preflight.json
- Historical capability research, not a fresh probe of the tested build. Referenced versions may differ from gateway_build; do not infer current capability.

## Evidence for the current measurements

- [run-litellm-python-10bb57507b26d310b59f12956034785b615cd5338cdaf5cee0e587c350c53acf](https://onthebench.ai/gateways/runs/run-litellm-python-10bb57507b26d310b59f12956034785b615cd5338cdaf5cee0e587c350c53acf)  ·  [complete raw record including RSS series](https://onthebench.ai/api/v1/runs/run-litellm-python-10bb57507b26d310b59f12956034785b615cd5338cdaf5cee0e587c350c53acf/raw.json)

## Configuration recorded by the run

```
model_list:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      use_chat_completions_api: true
      api_base: http://127.0.0.1:8000/v1
      api_key: dummy
  - model_name: gpt-4o-mini-responses
    litellm_params:
      model: openai/responses/gpt-4o-mini
      api_base: http://127.0.0.1:8000/v1
      api_key: dummy
  - model_name: gpt-4o-mini-anthropic
    litellm_params:
      model: anthropic/claude-3-5-sonnet-20241022
      api_base: http://127.0.0.1:8000
      api_key: dummy
  - model_name: gpt-4o-mini-gemini
    litellm_params:
      model: gemini/gemini-1.5-flash
      api_base: http://127.0.0.1:8000
      api_key: dummy
  - model_name: command-r
    litellm_params:
      model: cohere_chat/command-r
      api_base: http://127.0.0.1:8000/v2/chat
      api_key: dummy
  - model_name: gpt-4o-mini-bedrock
    litellm_params:
      model: bedrock/converse/anthropic.claude-3-sonnet-20240229-v1:0
      aws_bedrock_runtime_endpoint: http://127.0.0.1:8000
      aws_access_key_id: AKIAMOCKACCESSKEY
      aws_secret_access_key: mock-secret-access-key
      aws_region_name: us-east-1

# Public test token for the local mock rig, never a real provider credential.
general_settings:
  master_key: sk-otb-local-mock-benchmark

```

## Compare a shared protocol cell

- [LiteLLM · Python vs agentgateway](https://onthebench.ai/gateways/compare/litellm-python-vs-agentgateway)
- [LiteLLM · Python vs AISIX (api7)](https://onthebench.ai/gateways/compare/litellm-python-vs-aisix)
- [LiteLLM · Python vs APISIX](https://onthebench.ai/gateways/compare/litellm-python-vs-apisix)
- [LiteLLM · Python vs Bifrost](https://onthebench.ai/gateways/compare/litellm-python-vs-bifrost)
- [LiteLLM · Python vs Busbar](https://onthebench.ai/gateways/compare/litellm-python-vs-busbar)
- [LiteLLM · Python vs GoModel](https://onthebench.ai/gateways/compare/litellm-python-vs-gomodel)
- [LiteLLM · Python vs Helicone](https://onthebench.ai/gateways/compare/litellm-python-vs-helicone)
- [LiteLLM · Python vs Higress](https://onthebench.ai/gateways/compare/litellm-python-vs-higress)
- [LiteLLM · Python vs Kong](https://onthebench.ai/gateways/compare/litellm-python-vs-kong)
- [LiteLLM · Python vs LiteLLM · Rust](https://onthebench.ai/gateways/compare/litellm-python-vs-litellm-rust)
- [LiteLLM · Python vs One-API](https://onthebench.ai/gateways/compare/litellm-python-vs-one-api)
- [LiteLLM · Python vs Plano](https://onthebench.ai/gateways/compare/litellm-python-vs-plano)
- [LiteLLM · Python vs Portkey](https://onthebench.ai/gateways/compare/litellm-python-vs-portkey)
- [LiteLLM · Python vs TensorZero](https://onthebench.ai/gateways/compare/litellm-python-vs-tensorzero)

## Historical measurements

- [2026-11-10T17:39:41Z — ghcr.io/berriai/litellm:v1.108.3@sha256:e2bce769eb29b100d53ce90cbf2d6bba00dce82cb894c79d092ef6377523d7e5](https://onthebench.ai/gateways/runs/run-litellm-python-10bb57507b26d310b59f12956034785b615cd5338cdaf5cee0e587c350c53acf)
- [2026-08-26T19:08:49Z — ghcr.io/berriai/litellm:v1.98.0](https://onthebench.ai/gateways/runs/run-litellm-python-f479bedd357f332ad66a2fb412beb3c3ab9dc6e00dcf7c41a019f156c375db90)
