LUMINA: Long-horizon Understanding for Multi-turn Interactive Agents
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Large language models can perform well on many isolated tasks, yet they continue to struggle on multi-turn, long-horizon agentic problems that require skills such as planning, state tracking, and long context processing. In this work, we aim to better understand the relative importance of advancing these underlying capabilities for success on such tasks. We develop an oracle counterfactual framework for multi-turn problems that asks: how would an agent perform if it could leverage an oracle to perfectly perform a specific task? The change in the agent's performance due to this oracle assistance allows us to measure the criticality of such oracle skill in the future advancement of AI agents. We introduce a suite of procedurally generated, game-like tasks with tunable complexity. These controlled environments allow us to provide precise oracle interventions, such as perfect planning or flawless state tracking, and make it possible to isolate the contribution of each oracle without confounding effects present in real-world benchmarks. Our results show that while some interventions (e.g., planning) consistently improve performance across settings, the usefulness of other skills is dependent on the properties of the environment and language model. Our work sheds light on the challenges of multi-turn agentic environments to guide the future efforts in the development of AI agents and language models.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 13 of its 22 references that are in the map. First posted on arXiv 23 Jan 2026.
Builds on
13 of 22 references.
Closest ancestors · 1
Same direction, newest first
Other references · 12
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
- 2023Reasoning with Language Model is Planning with World Modelcited by 846
- 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
- 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 359
- 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 84
- 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMscited by 62
- 2023Memory Sandbox: Transparent and Interactive Memory Management for Conversational Agentscited by 36
- 2024Your Context Is Not an Array: Unveiling Random Access Limitations in Transformerscited by 11
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions
- 2025A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning
- 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward
- 2025Limits of Emergent Reasoning of Large Language Models in Agentic Frameworks for Deterministic Games
- 2026SKILL.state: Scalable Long-Horizon Agent Skills
- 2026From Task Solving to Robust Real-World Adaptation in LLM Agents
- 2026SkillAlign: Aligning Skill Interfaces for LLM-based Agents
- 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applications