The paper introduces Learn2Play Bench, 20 text games built around hidden rules. The human record with the highest peak in each game reached a higher top score than Claude Code running Claude Opus 5; that AI setup, however, averaged more across repeated episodes.
Each play-through is an episode. Players see available actions and text feedback but not the game code. Agents can carry experience between episodes within a trial, without changing model weights. Ten games repeat the same instance; in the other ten, visible details change while the underlying rule stays the same.
For each game, researchers expressed raw scores as a percentage of a reference score: 100 means a result matched the reference, which was exact for some games and partial or estimated for others. It does not always represent a proven optimum. The five challenge games were scored over ten episodes per trial (five for each other game) and counted three times as much as each of the other 15, giving both groups equal total weight.
Researchers recruited 21 to 30 college students per game and selected the human record with the highest peak. For AI, the peak measure averages the best score from each of three independent trials; the episode average combines scores across the evaluation window. Human averages use the selected record's scores across its episodes. After weighting across games, the selected human records scored 84.3 on the peak measure and 53.5 on the episode average. Claude Code, an agent harness running the Opus 5 language model, scored 81.8 and 62.3, respectively.
In games that repeated the same instance, researchers compared action sequences from consecutive episodes for the three highest-peak human records per game and six Opus 5 agent methods. The average sequence-similarity score was 0.54 for humans and 0.66 for agents; higher values mean more of the action order matched. The measure describes recorded repetition, not whether a strategy was good.
The benchmark uses controlled text games, and its human result selects the highest-peak record rather than a representative participant. It does not establish how typical people or agents perform in workplaces or physical environments.
Sources
- Learn2Play Bench paper, arXiv:2610.08215v1
- Authors' project page and evaluation protocol
- Image: Figure 2C from the authors' paper, licensed under CC BY 4.0.