mlsanityData to 14 Oct 2026

MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces

arXiv 2502.07709 ↗First posted 11 Feb 2025cs.AIcited by 7

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Open-ended learning agents must efficiently prioritize goals in vast possibility spaces, focusing on those that maximize learning progress (LP). When such autotelic exploration is achieved by LLM agents trained with online RL in high-dimensional and evolving goal spaces, a key challenge for LP prediction is modeling one's own competence, a form of metacognitive monitoring. Traditional approaches either require extensive sampling or rely on brittle expert-defined goal groupings. We introduce MAGELLAN, a metacognitive framework that lets LLM agents learn to predict their competence and LP online. By capturing semantic relationships between goals, MAGELLAN enables sample-efficient LP estimation and dynamic adaptation to evolving goal spaces through generalization. In an interactive learning environment, we show that MAGELLAN improves LP prediction efficiency and goal prioritization, being the only method allowing the agent to fully master a large and evolving goal space. These results demonstrate how augmenting LLM agents with a metacognitive ability for LP predictions can effectively scale curriculum learning to open-ended goal spaces.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 37 of its 70 references that are in the map. First posted on arXiv 11 Feb 2025.

Builds on

37 of 70 references.

References · 37

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  7. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  8. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  9. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  10. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  11. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  12. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
25 more
  1. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  2. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  3. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  4. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
  5. 2021Replay-Guided Adversarial Environment Designcited by 111
  6. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  7. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  8. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 85
  9. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 83
  10. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 79
  11. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 70
  12. 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
  13. 2013Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robotscited by 58
  14. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 45
  15. 2021Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraftcited by 35
  16. 2024Practice Makes Perfect: Planning to Learn Skill Parameter Policiescited by 23
  17. 2013Multi-Armed Bandits for Intelligent Tutoring Systemscited by 21
  18. 2022General Intelligence Requires Rethinking Explorationcited by 20
  19. 2021TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RLcited by 18
  20. 2018Curiosity Driven Exploration of Learned Disentangled Goal Spacescited by 16
  21. 2018Accuracy-based Curriculum Learning in Deep Reinforcement Learningcited by 15
  22. 2020WordCraft: An Environment for Benchmarking Commonsense Agentscited by 14
  23. 2022Language and Culture Internalisation for Human-Like Autotelic AIcited by 14
  24. 2022Stein Variational Goal Generation for adaptive Exploration in Multi-Goal Reinforcement Learningcited by 6
  25. 2024SAC-GLAM: Improving Online RL for LLM agents with Soft Actor-Critic and Hindsight Relabelingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 356
  2. 2026Interestingness as an Inductive Heuristic for Future Compression Progresscited by 3
  3. 2026The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillationcited by 3
  4. 2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Modelscited by 2
  5. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  6. 2025HERAKLES: Hierarchical Skill Compilation for Open-ended LLM Agentscited by 0
  7. 2026LLMs learn different forms of metacognition when trained to predict their own accuracycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Competence-Aware AI Agents with Metacognition for Unknown Situations and Environments (MUSE)
  2. 2025Truly Self-Improving Agents Require Intrinsic Metacognitive Learning
  3. 2026Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs
  4. 2026MetaCogAgent: A Metacognitive Multi-Agent LLM Framework with Self-Aware Task Delegation
  5. 2025From Experience to Strategy: Empowering LLM Agents with Trainable Graph Memory
  6. 2026Aspire: Can Models Self-Evolve from Vague Goals?
  7. 2024Metacognition is all you need? Using Introspection in Generative Agents to Improve Goal-directed Behavior
  8. 2026Learn Like Humans: Use Meta-cognitive Reflection for Efficient Self-Improvement