How a model is scored
Every model builds the same real app from a fixed spec; we run what it built, and measure four things — quality, cost, speed and tokens.
One page for every reader — the role changes what leads, the depth changes how much shows. Nothing is held back: every section can show the rest in place.
The task
Every model gets the same real assignment: build a working online store — 9 small services, written in 5 programming languages, that must all work together as one shop.
The run
Every job is run 5 times, because the same model can do brilliantly on one try and badly on the next — and the spread between tries is the risk you would be buying.
The four measures
Four things are measured for every result: quality (does the software work?), cost, speed and tokens.
What you can observe
This section goes deeper than the current view.
Kept honest
The rules were sealed before any model ran, no model or provider is special-cased, no model judges another, and the raw results are released — how each of these is enforced.
No scored round has run yet — this page describes the method, not a result.
Reproduce it
This section goes deeper than the current view.