LiteLLM · Rust history
Aug 2026 → Nov 2026
openai → openai cell. The engine changed between these runs (f3adbb1 → d240bad), so differences can include instrument changes, not only gateway changes. Changes within ±5% show as roughly unchanged (—); this is a display tolerance, not a statistical margin.
| Reading | Aug 2026 | Nov 2026 | Change | Rank Aug 2026 → Nov 2026 |
|---|---|---|---|---|
| Added latency p50 (µs) | n/a | 249 µs | · | — → 6 of 15 |
| Added latency p99 (µs) | n/a | 279 µs | · | — → 5 of 15 |
| Throughput, p99 ≤ 1 ms (req/s) | n/a | 8,404 req/s | · | — → 3 of 3 |
| Throughput, p99 ≤ 5 ms (req/s) | n/a | 10,049 req/s | · | — → 1 of 1 |
| Throughput, p99 ≤ 10 ms (req/s) | n/a | 10,049 req/s | · | — → 1 of 2 |
| Throughput, p99 unbounded (req/s) | n/a | 10,049 req/s | · | — → 1 of 3 |
| CPU per request (µs) | n/a | 272.6 | · | — → 7 of 14 |
| Idle RSS (MiB) | n/a | 12.74 MiB | · | — → — |
| Peak RSS (MiB) | n/a | 19.88 MiB | · | — → — |
Runs
2 published runs · Added latency p99 (µs), openai → openai · values describe the recorded local-mock configuration; a missing reading is never zero. Hover a trend for its comparability details.
| Measured | Build | Engine | Result | Trend | Field placement | Evidence |
|---|---|---|---|---|---|---|
| src/litellm-rust/target/release/litellm-gateway | d240bad | 279 µs | Engine changed | 5 of 15 Current field | Run evidence | |
| litellm-ai-gateway | f3adbb1 | Not measured | First run | Not ranked | Run evidence |