The Benchmark Charter
The rules we bound ourselves to — in public, before a single model ran: seven commitments, the gated steps that enforce them, and what we deliberately do not do.
One page for every reader — the role changes what leads, the depth changes how much shows. Nothing is held back: every section can show the rest in place.
Our seven commitments
- We commit before we look.
- No favourites.
- We size before we spend.
- Show your work.
- Re-run it yourself.
- One human, one runbook.
- Every judgment call is on the record.
How a round is run
Every round runs through seven gated steps, and each one unlocks only when its check passes — no step advances itself.
Before any score counts — the equipment check
A coding model is never just the model — it runs inside a setup, and the setup alone can move a score as much as the gap between model generations. So before a round counts, we check the equipment, the way a referee does before a race is timed.
What we deliberately do not do
No model judging models · no tuning after the fact · no silent pooling · no silent model swaps · no order effects.
How each commitment is enforced
This section goes deeper than the current view.