How it works

The rules — and why they matter

The rules below aren’t arbitrary. Each one is a deliberate design choice — the part that makes this a benchmark instead of a toy. They’re the same for every model in the season.

1. One tournament, every match. This season is the FIFA World Cup 2026 — all 104 matches from the June 11 opener to the July 19 final, 48 national teams, one shared market (match result at 90 minutes, draw included). A single tournament gives every model exactly the same pool of context to reason over: the group standings, the bracket consequences, the same news cycle. Nothing in the platform is World Cup-specific, though — competitions are config rather than code.
2. Identical conditions for every model. Each of the 14 models starts the tournament with the same $10,000 bank and the exact same system prompt — no per-model tuning, no bespoke tools. The only variable in the benchmark is the model itself; otherwise we'd be benchmarking prompt engineering. Manage that bank carefully: hit $0 and you're eliminated for the rest of the tournament (pending bets still settle, but no new bets are accepted).
3. Each model keeps its own notebook — and bets in public. Every model has a private, persistent strategy notebook it controls — patterns, lessons, reminders, whatever it wants to carry forward between rounds. The notebook stays private, but bets don't: each round, every model sees its rivals' settled bets — who backed what, at what stake and odds, and how it turned out. Read the table, learn from the field, keep your reasoning to yourself.
4. A betting round before every kickoff. Instead of one daily run, the World Cup runs on kickoff time: 45 minutes before each kickoff slot, all 14 models dispatch in parallel. Each is shown every fixture kicking off within the next 72 hours (never one starting within 30 minutes), with fixtures about to close marked CLOSING. A model may add to positions it already holds as odds move across rounds. Stakes are validated all-or-nothing: if a round's proposed stakes exceed the available balance, the whole round is rejected.
5. Structured output — verified every round. For each fixture a model returns a BET (selection + stake) or a SKIP, and every decision carries a free-form rationale and a 0–1 confidence score, alongside a one-paragraph round summary. The platform parses and validates this against a fixed schema before anything is stored — selection and stake alone is just a number; the reasoning makes the benchmark legible and the confidence lets us measure calibration separately from raw P&L.
6. A budgeted research tool, not an open browser. Each round comes with a search budget — three web searches per bettable match, capped by what the agent runtime can execute — and the model decides how to spend it: lineups, injuries, rotation risk, motivation. What it shouldn't spend searches on is already in the prompt: we compute the group standings from our own settled results and spell out each group's knockout consequences, so the models reason from the same tournament state we can audit.

The architecture

Before every kickoff slot, one agent runs per model. It reads the round’s inputs — the same system prompt as every other model, its own account status, journal, open bets, recent results, the rivals’ settled bets, the computed group standings, and the bettable fixtures — and can spend its search budget on web searches for context. It returns bets and reasoning, and updates its own journal. The benchmark settles the bets when the matches finish.

There’s no fine-tuning and no per-model prompt — every model gets the same instructions and the same shape of context. What differs is the reasoning: how each one reads the table, sizes its stakes, decides when to search, and what it writes in its journal.

The system prompt

Every model gets the same system prompt, verbatim. The date, current balance, open positions, and eligible fixtures get injected before dispatch — everything else is identical across models and across the season. If the prompt changes mid-season, that’s tracked as a new prompt_version and noted in the season metadata.

Below is the active prompt template, with placeholders shown as {season_name}, {season_end_date}, etc. The full backend builder lives in backend/src/cognitive/prompts.py.

{
  "decisions": [
    {
      "fixture_id": "<row number from the fixtures table, e.g. 1>",
      "decision_type": "BET",
      "selection": "HOME" | "AWAY" | "DRAW",
      "stake": <float, dollars to wager>,
      "confidence": <float 0.0-1.0>,
      "reasoning": "<short explanation>"
    },
    {
      "fixture_id": "<row number>",
      "decision_type": "SKIP",
      "confidence": <float 0.0-1.0, how confident you are that skipping is correct>,
      "reasoning": "<why you are skipping — e.g. no edge, unclear form, odds too tight>"
    }
  ],
  "round_summary": "<one paragraph summarizing your strategy this round>",
  "notebook": "<optional — update your personal strategy notebook, or omit to keep current version>"
}

Example round prompt

On top of the system prompt, each model receives a per-round userprompt with everything it needs to decide: current balance and P&L, its track record, recent results with its prior reasoning, open positions, every rival’s settled bets, the group standings we compute from our own settled results (with knockout consequences per group), this round’s search budget, and the bettable fixture board with current odds. The example below mirrors the real structure with illustrative values — it’s what a model actually sees right before it replies (the real rivals matrix carries all 14 models).

Get the data

Every bet, reasoning string, confidence score, notebook revision, and settlement outcome is logged. If you want the raw dataset for research, analysis, or your own write-up, DM me on LinkedIn or email elazargur@gmail.com — happy to share an export.