LiteLLM · Python history
Aug 2026 → Nov 2026
openai → openai cell. The engine changed between these runs (f3adbb1 → d240bad), so differences can include instrument changes, not only gateway changes. Changes within ±5% show as roughly unchanged (—); this is a display tolerance, not a statistical margin.
| Reading | Aug 2026 | Nov 2026 | Change | Rank Aug 2026 → Nov 2026 |
|---|---|---|---|---|
| Added latency p50 (µs) | 6,453 µs | 8,512 µs | ↓ +31.9% | 11 of 14 → 12 of 15 |
| Added latency p99 (µs) | 8,143 µs | 9,522 µs | ↓ +16.9% | 11 of 14 → 12 of 15 |
| Throughput, p99 ≤ 10 ms (req/s) | 163 req/s | 117 req/s | ↓ -28.2% | 1 of 1 → 1 of 1 |
| Throughput, p99 unbounded (req/s) | 192 req/s | 119 req/s | ↓ -38.0% | 2 of 2 → 1 of 1 |
| CPU per request (µs) | 6,414.9 | 8,644.8 | ↓ +34.8% | 12 of 14 → 13 of 14 |
| Idle RSS (MiB) | 744.8 MiB | 593 MiB | ↑ -20.4% | — → — |
| Peak RSS (MiB) | 778.9 MiB | 628.6 MiB | ↑ -19.3% | — → — |
Runs
2 published runs · Added latency p99 (µs), openai → openai · values describe the recorded local-mock configuration; a missing reading is never zero. Hover a trend for its comparability details.
| Measured | Build | Engine | Result | Trend | Field placement | Evidence |
|---|---|---|---|---|---|---|
| ghcr.io/berriai/litellm:v1.108.3 | d240bad | 9,522 µs | Engine changed | 12 of 15 Current field | Run evidence | |
| ghcr.io/berriai/litellm:v1.98.0 | f3adbb1 | 8,143 µs | First run | 11 of 14 Aug 2026 field | Run evidence |