The Face-Off
Same task, the AIs you know. Who actually gets it right?
Stage: beta opens 2 December 2026. Plans and progress public; code private.
The question
Give the AIs people actually use the exact same task. Which ones get it right, at what cost, and how many tokens does it take? And later: how much does a better prompt change all three?
Concept
An open, repeatable benchmark. Every model gets the same task, word for word, and we score what each one actually produces: right or wrong, what it cost, and how many tokens it used. The beta starts with everyday tasks (split a dinner bill, plan a week of meals on a budget, pull the dates out of an email) and shows the answers side by side.
The longer-term aim is a public arena where prompt-improvement methods compete on one fixed model and one fixed task set, so you can see which ones really help.
Design
- Tasks with an answer you can check. Everyday tasks in the beta; later, coding (does the program pass hidden tests?), extraction (did it pull the right fields from a document?) and reasoning (is the final answer exactly right?).
- No AI judges another AI. Every score comes from a plain, deterministic check, so the same answer always gets the same score.
- Training and held-out tasks. Part of each task set is shown openly; the rest is held out and never published, so nobody can tune to the answers.
Method: how runs are kept fair
- Official APIs only. Every run goes through each provider's model API, never a chat app or a consumer subscription, so the settings, the version and the cost are known and recorded.
- Same prompt, same task, same scoring for every model.
- Versions beside every score. The model version and the task-set version sit next to every number, and scores are only compared within one version.
- Cost, end to end. We measure the tokens and the charge to finish each task, not just whether it was right, and report cost per correct answer.
- More than one run. Each task runs several times per model, and we show how much the answers vary.
- Signed results. Every published result is signed, so anyone can check that a number came from the benchmark unchanged.
- No home advantage. If we enter a method of our own, it's labelled, gets no special treatment, and is published even when it loses.
- Independent. No provider sees results before the public, and if a provider ever gives us API credits, we say so on this page.
Roadmap
- Alpha (now): the scoring engine, runners and checks are built and tested; the first task sets are being written.
- Beta (2 December 2026): everyday tasks, four well-known models side by side, the answers, scores and cost per correct answer on a public page, and results by email.
- Phase 1: the public leaderboard across coding, extraction and reasoning. "Same AI, better prompt": prompt-improvement methods are entered and scored, and their authors see the full detail of their own runs. A community space for discussion.
- Phase 2: try two anonymised entrants on your own prompt and vote; more task categories once AI-judged scoring can be trusted; agents and multi-step tasks; detailed reports on cost and token use.
Status
The engine is built; the beta page and its first tasks are in progress. Join the waitlist to get each round's results.
Join the waitlist
Get Face-Off results by email, and an invitation to the Debater alpha.
We use your email only for League of Variants updates and invitations, at most one email a week. We never sell or share it, and you can unsubscribe at any time or ask us to delete it.