Why you can trust it
A benchmark is only as good as what stops it being tuned or favoured — here is each safeguard, the conflict it answers, and what is still unbuilt.
One page for every reader — the role changes what leads, the depth changes how much shows. Nothing is held back: every section can show the rest in place.
Sealed before the run
The rules were locked in public before any model ran: which models compete, how many times each job runs, and the scoring formula. Nothing can be tuned to a result after the fact.
No favourites
No model and no provider is special-cased. Every model runs through the same code path, and no model judges another — scoring is plain arithmetic over real builds and test runs.
The disclosed conflict
There is a real conflict of interest, stated up front: the person who runs the benchmark also wrote the software that runs and scores it.
The spread, not just a score
Every score is shown with its spread, not as a lone number — because an inconsistent model is a risk, even when its average looks good.
Building it, not remembering it
The online store is a public open-source project, so a model may have seen it. The task that decides the ranking is kept non-public, so it cannot have been memorized.
The human check
Human experts check the machine's scores before anything is published: they can agree, add a note, or overrule.
What is still unbuilt
Stated plainly: no scored round has run yet. Still being finished before it does — the blind human sample, the tie-break rules for the team bracket, the fairness audit of the bug check, and a fix so the coaching and context variants are delivered as designed.