Reinforcement Learning in Super Mario Bros: Curriculum, Pedagogy, and Optimal Level Design in World 1-1
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization
Abstract (arXiv)
World 1-1 of Super Mario Bros is widely celebrated as a masterclass in game design: its progressive structure is credited with teaching players core mechanics through the level itself. We ask whether that structure is empirically measurable using reinforcement learning. We implement World 1-1 from scratch as a fully discrete environment and compare four algorithms -- Q-Learning, SARSA, Monte Carlo, and Deep Q-Network (DQN) -- across three progressively complex versions of the same level. Monte Carlo emerges as the strongest agent (94.9% ± 1.5% win rate), outperforming DQN (76.4% ± 3.4%) by learning to maximize intermediate rewards along winning paths rather than taking the most direct route. We then use Monte Carlo in a curriculum experiment permuting World 1-1's six canonical segments across twelve conditions. Canonical ordering converges fastest, achieves the highest learning efficiency, and is the only condition with zero catastrophic failures; no random permutation matches all three criteria simultaneously. These results provide, to the best of our knowledge, the first empirical validation that World 1-1's canonical design encodes genuine pedagogical structure: one that measurably accelerates learning and cannot be replicated by chance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 0 of its 9 references that are in the map. First posted on arXiv 28 Jun 2026.
Builds on
0 of 9 references.
No references here yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Gym4ReaL: A Suite for Benchmarking Real-World Reinforcement Learning
- 2026Reinforcement Learning: From Algorithms To Foundation Models
- 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning
- 2026Quantum-Inspired Episode Selection for Monte Carlo Reinforcement Learning via QUBO Optimization
- 2026Player-1: Learning to Play from Synthetic Experience
- 2024Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search
- 2025DiffFP: Learning Behaviors from Scratch via Diffusion-based Fictitious Play
- 2019ZPD Teaching Strategies for Deep Reinforcement Learning from Demonstrations