# GameCraft-Bench: Can Agents Build Playable Games End-to-End in a Real Game Engine?

arXiv: https://arxiv.org/abs/2606.17861 · HTML: https://mlsanity.com/paper/2606.17861/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 16 Jun 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 17 of 17)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023)
- [CogAgent: A Visual Language Model for GUI Agents](https://mlsanity.com/paper/2312.08914/) (2023)
- [OS-ATLAS: A Foundation Action Model for Generalist GUI Agents](https://mlsanity.com/paper/2410.23218/) (2024)
- [GameDevBench: Evaluating Agentic Capabilities Through Game Development](https://mlsanity.com/paper/2602.11103/) (2026)
- [OpenGame: Open Agentic Coding for Games](https://mlsanity.com/paper/2604.18394/) (2026)
- [WebGameBench: Requirement-to-Application Evaluation for Coding Agents via Browser-Native Games](https://mlsanity.com/paper/2605.17637/) (2026)
- [GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injection](https://mlsanity.com/paper/2605.07442/) (2026)
- [AutoUE: Automated Generation of 3D Games in Unreal Engine via Multi-Agent Systems](https://mlsanity.com/paper/2603.07106/) (2026)
- [90% Faster, 100% Code-Free: MLLM-Driven Zero-Code 3D Game Development](https://mlsanity.com/paper/2509.26161/) (2025)

## Built on by (most cited first, 22 of 22)

- [GameXpert-Bench: How Far Are Coding Agents from Expert Game Development?](https://mlsanity.com/paper/2608.21833/) (2026)
- [RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments](https://mlsanity.com/paper/2609.15364/) (2026)
- [Harness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement](https://mlsanity.com/paper/2609.01481/) (2026)
- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [GameASG-Bench: Benchmarking Autonomous Software Generation for Game Development](https://mlsanity.com/paper/2609.21293/) (2026)
- [RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents](https://mlsanity.com/paper/2609.22000/) (2026)
- [DashArena: Benchmarking LLMs on Interactive Analytic Dashboard Generation](https://mlsanity.com/paper/2608.10567/) (2026)
- [CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering](https://mlsanity.com/paper/2609.32600/) (2026)
- [World Agent: Can Language Models Keep a World Running?](https://mlsanity.com/paper/2609.32692/) (2026)
- [SWE-Game: Can Coding Agents Build the Games We Want?](https://mlsanity.com/paper/2609.33678/) (2026)
- [A2Z GameSpec-Bench: How Faithfully Can Coding Agents Generate Games from Game Design Specifications?](https://mlsanity.com/paper/2609.39564/) (2026)
- [World Editing: Intervening on Executable Worlds at Increasing Depth](https://mlsanity.com/paper/2610.02331/) (2026)
- [OpenGameEval: Benchmarking Agentic Programming and Exploration in a Stateful Game Engine](https://mlsanity.com/paper/2610.02563/) (2026)
- [Code2Games: Enabling Coding Agents for Gaming World Generation](https://mlsanity.com/paper/2610.05033/) (2026)
- [GAMEGO: Training Game-Dev Agents with Synthetic Trajectories Anchored in Real-World Assets](https://mlsanity.com/paper/2610.06910/) (2026)
- [Recursive Game Creator: An Agentic Product-Level Experience-Oriented Game Harness](https://mlsanity.com/paper/2610.08621/) (2026)
- [WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?](https://mlsanity.com/paper/2610.08720/) (2026)
- [Don't Guess, Just Ask: Evaluating Interactive Requirement Clarification for Repository-Level Software Development](https://mlsanity.com/paper/2610.13022/) (2026)
- [ProgramGym: Scaling Training Environments for Long-Horizon Program Reproduction Agents](https://mlsanity.com/paper/2610.13395/) (2026)
- [Game-Creation-Agents: Autonomous Game Production](https://mlsanity.com/paper/2610.16776/) (2026)
- [Interaction-as-a-Judge](https://mlsanity.com/paper/2610.17895/) (2026)
- [Player-1: Learning to Play from Synthetic Experience](https://mlsanity.com/paper/2610.20584/) (2026)

## Abstract (arXiv)

Game generation is an emerging application of coding agents, requiring models to transform natural-language specifications into playable interactive systems. Unlike traditional coding tasks, game generation takes place within a game engine, where scripts, scenes, assets, rendering, and runtime interactions must jointly produce coherent gameplay. We formalize end-to-end game generation as the problem of producing a complete game artifact that realizes a specification through observable player-game interaction in a target environment. We argue that evaluating this setting requires three desiderata: Engine Grounding, Artifact Completeness, and Interactive Verification. We propose an interaction-grounded evaluation framework that assesses executable gameplay through replayed demonstrations and rubric-guided multimodal judging. We instantiate this framework as GameCraft-Bench, a benchmark comprising 140 Godot tasks across 15 game families. Evaluations of frontier coding agents show that end-to-end game generation remains highly challenging: the strongest agent achieves only 41.46%, and most agents score below 40%. Further analysis reveals that while agents often implement recognizable mechanics, they struggle to deliver complete games with sufficient content, functional visual feedback, and coherent presentation. See https://tongxuluo.github.io/gamecraft-bench-website for demos, code, and data.