# TextQuests: How Good are LLMs at Text-Based Video Games?

arXiv: https://arxiv.org/abs/2507.23701 · HTML: https://mlsanity.com/paper/2507.23701/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 31 Jul 2025.

## Builds on (most cited first, 17 of 17)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://mlsanity.com/paper/2504.12516/) (2025)
- [PaperBench: Evaluating AI's Ability to Replicate AI Research](https://mlsanity.com/paper/2504.01848/) (2025)
- [MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMs](https://mlsanity.com/paper/2501.17399/) (2025)
- [Interactive Fiction Games: A Colossal Adventure](https://mlsanity.com/paper/1909.05398/) (2019)
- [NoLiMa: Long-Context Evaluation Beyond Literal Matching](https://mlsanity.com/paper/2502.05167/) (2025)
- [Keep CALM and Explore: Language Models for Action Generation in Text-based Games](https://mlsanity.com/paper/2010.02903/) (2020)
- [Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following](https://mlsanity.com/paper/2410.15553/) (2024)
- [Graph Constrained Reinforcement Learning for Natural Language Action Spaces](https://mlsanity.com/paper/2001.08837/) (2020)
- [What Would Jiminy Cricket Do? Towards Agents That Behave Morally](https://mlsanity.com/paper/2110.13136/) (2021)
- [Finding Flawed Fictions: Evaluating Complex Reasoning in Language Models via Plot Hole Detection](https://mlsanity.com/paper/2504.11900/) (2025)

## Built on by (most cited first, 12 of 12)

- [Vox Deorum: A Hybrid LLM Architecture for 4X / Grand Strategy Game AI -- Lessons from Civilization V](https://mlsanity.com/paper/2512.18564/) (2025)
- [What Do LLM Agents Know About Their World? Task2Quiz: A Paradigm for Studying Environment Understanding](https://mlsanity.com/paper/2601.09503/) (2026)
- [Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse](https://mlsanity.com/paper/2605.09965/) (2026)
- [Dual-Scale World Models for LLM Agents Towards Hard-Exploration Problems](https://mlsanity.com/paper/2509.24116/) (2025)
- [Mini Amusement Parks (MAPs): A Testbed for Modelling Business Decisions](https://mlsanity.com/paper/2511.15830/) (2025)
- [Muse Spark Safety & Preparedness Report](https://mlsanity.com/paper/2606.12429/) (2026)
- [SysAdmin: Measuring Instrumental Power-Seeking in Frontier AI](https://mlsanity.com/paper/2607.18239/) (2026)
- [The Token Games: Evaluating Language Model Reasoning with Puzzle Duels](https://mlsanity.com/paper/2602.17831/) (2026)
- [Towards More Standardized AI Evaluation: From Models to Agents](https://mlsanity.com/paper/2602.18029/) (2026)
- [RPA-Check: A Multi-Stage Automated Framework for Evaluating Dynamic LLM-based Role-Playing Agents](https://mlsanity.com/paper/2604.11655/) (2026)
- [Rushes: A Human Preference Dataset for Pluralistic Alignment](https://mlsanity.com/paper/2607.20767/) (2026)
- [Schema: Discovering Unknown Environments via Agentic Program Induction](https://mlsanity.com/paper/2609.39140/) (2026)

## Abstract (arXiv)

Evaluating AI agents within complex, interactive environments that mirror real-world challenges is critical for understanding their practical capabilities. While existing agent benchmarks effectively assess skills like tool use or performance on structured tasks, they often do not fully capture an agent's ability to operate autonomously in exploratory environments that demand sustained, self-directed reasoning over a long and growing context. To enable a more accurate assessment of AI agents in challenging exploratory environments, we introduce TextQuests, a benchmark based on the Infocom suite of interactive fiction games. These text-based adventures, which can take human players over 30 hours and require hundreds of precise actions to solve, serve as an effective proxy for evaluating AI agents on focused, stateful tasks. The benchmark is specifically designed to assess an LLM agent's capacity for self-contained problem-solving by precluding the use of external tools, thereby focusing on intrinsic long-context reasoning capabilities in an exploratory environment characterized by the need for trial-and-error learning and sustained problem-solving within a single interactive session. We release TextQuests at https://textquests.ai.