GoModel benchmark run

Operated by the Busbar team; Busbar is also an entrant. Gateway overhead against a local mock is not end-to-end model latency. Methodology and limitations.

Measured
Recorded build
enterpilot/gomodel:0.1.82
Hardware
AWS m7g.4xlarge (Graviton3, 16 cores / 64 GB). Gateway-under-test pinned to 4 cores (the comparable basis); mock and load generator on 6 cores each so the mock never bottlenecks the streaming sweep. Ubuntu 24.04. One dedicated box per gateway.
Architecture
arm64

SHA-256 of exact raw JSON bytes: d5a944f96686f17fd771628164aa2872bb733a541b28d2b40dbbfd3876e24b19.

Full immutable raw result, all protocol cells and samples · Run metadata JSON · Current gateway evidence

Recorded metric definitions

memory
See matrix.memory.protocol and each cell's memory.protocol, which state the durations and both plateau thresholds formatted from the constants that ran.
perf.added_latency
What the gateway adds, at concurrency 1. The gateway leg's percentile minus the same request taken straight to the mock, one at a time, both legs in the same window. No search and no threshold: there is nothing to stop early and nothing to decide. A difference at or below what the rig can resolve publishes below_resolution - the best result the comparison can express - and every surface renders that apart from a never-measured hole.
perf.cost
What a request cost, in microseconds of gateway CPU. User plus system time, summed across the gateway's whole process tree (a gateway that forks workers keeps its CPU in the children, and a parent-only reading would report a busy gateway as idle), sampled as the difference across one load window and divided by the requests that window actually completed. Why it exists beside throughput. Peak rps is a saturation number: once a gateway fills the cores it is pinned to, the ladder stops describing the gateway and starts describing the box, and two gateways at that wall read the same however different they are. Cost per request has no such ceiling - at saturation both serve the same rate by definition, and the one doing less work per request still reads lower. It is also the figure that maps to money, since half the CPU serves the same traffic on half the instance. Measured at one concurrency, c=8, held identical for every gateway and published as cost_window_conc. No single concurrency is below saturation for a field spanning double-digit to five-figure rps, so matched load is impossible and matched concurrency is the honest substitute - and it is stated rather than assumed. Refused, not estimated, in two cases. A window with any failure publishes no cost at all: CPU spent refusing requests is real, but dividing it by only the successes would describe the failures rather than the work. And a window with major page faults means the box was swapping, so what was timed is the disk - those numbers publish with cost_majflt non-zero and the cost itself marked a harness fault, so a reader sees why the row looks wrong instead of finding a hole. cost_core_utilisation is the fraction of the pinned cores busy across that same window. It is what makes a peak interpretable: near 1.0 the gateway had filled the cores it was given and its throughput number is a real ceiling; well below it, the limit is somewhere else and the peak means something else.
perf.frontier
Throughput at a tail latency you accept. For each declared bound, the most requests/sec this cell carried while 99% of requests finished under that bound and it failed none it accepted. Bounds: 1ms, 5ms, 10ms, 50ms, 100ms plus one reading with no latency bound at all (failures only). One measurement, read six ways: a single concurrency sweep, published as sweep_max_proxy, so every reading is re-derivable from the rungs rather than taken on trust. Monotone non-decreasing across the bounds by construction - relaxing a bound only adds rungs to the set a maximum is taken over - so a reading cannot exceed a looser one. Zero failures, not a tolerance: the load generator counts connections the rig could not open separately (rig_refused, and those windows are discarded), so a failure reaching a reading is the gateway losing a request it had accepted. Termination: the sweep climbs a doubling ladder from the floor and stops at the first concurrency where no window served cleanly, because more concurrency cannot un-fail those requests. Each reading names first_disqualified_conc, the lowest concurrency above it that stopped qualifying - that pair is the proof it is a boundary. When nothing above it disqualified, the sweep ran out of range instead of finding a limit, and the reading sets lower_bound: true: the rate is real, it is simply a floor rather than a ceiling. The p99_us on a reading is the tail the winning rung actually produced, never the bound - 4ms under a 100ms bound is not the same finding as 99ms.
stream.added_ttft_and_gap
What the gateway adds to a stream, at concurrency 1: time to the first event, and the gaps between frames inside one stream, each as the gateway leg minus the same stream taken straight to the mock. Time-to-first-event rather than first token, deliberately and by the same definition on both legs: a frame budget of one is satisfied by a dialect's opening scaffolding (openai sends a role delta, anthropic a message_start), so the quantity is the same on both sides and the difference still isolates what the gateway added.
stream.streams_sustained
The most concurrent SSE streams this cell carries cleanly, and the frames/sec at that point. A concurrency holds the gate when every expected content frame arrives (ratio 1.0, i.e. no tolerance - a proxy that drops a frame has dropped a user's token), no inter-frame gap exceeds 10x the mock's own pacing interval, and under 0.1% of streams error. Termination: the gate is monotone in concurrency, so the ceiling is a boundary found by bisection - a pass proven at n and a failure measured above it. sweep_streams carries every rung probed. A range whose top still passed publishes an absence rather than a ceiling, because the top of the range is our choice and not the gateway's answer.

Instrument provenance

Architecture
arm64
Release URL
https://github.com/GetBusbar/benchmarking/releases/download/rig
Engine (harness)
Commit
f3adbb1315b26129f5e317af5279decefb1cea8f
Uncommitted changes
No
Mock upstream
Origin
release
SHA-256
c32b9ff470eddf87d477098dac6e19a6c75eeef6e594dfc57005310d69e9049d
Asset updated
2026-08-03T21:23:53Z
Load generator
Not recorded
Box qualification
Band (%)
20
Baseline (req/s)
500,705
Baseline samples
1
Concurrency
32
Drift (%)
0.561
Observation absence detail
Not recorded
Observation absence reason
Not recorded
Observed (req/s)
503,516
Outcome
pass

The exact recorded JSON is in the full immutable raw result.