mlsanityData to 14 Oct 2026

Open-ended Multi-agent Autocurricula via Visual Inspection of Policies with Multi-modal LLMs

arXiv 2607.08193 ↗First posted 9 Jul 2026cs.LG cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Open-ended curricula in Reinforcement Learning (RL) aim to train generally-capable agents by identifying tasks that facilitate learning increasingly complex skills. A major challenge when designing such curricula is assessing task difficulty relative to the agent's current learning progress. While previous work has explored using scalar task scores or textual summaries of the agent's behavior, here we study a different approach: directly inspecting policy behavior via recorded episode videos. We introduce a simple yet effective instantiation of this approach which leverages a Video Language Model (VLM) to both process these videos and provide curriculum recommendations, which we call Visual Inspection of Policies (VIP). Since videos can naturally contain any number of controllable agents, we empirically study VIP on the StarCraft Multi-Agent Challenge (SMAC). We show that even with a lightweight and openly accessible VLM (VideoLLaMa2-7B), VIP can use policy videos to generate more effective curricula than both its text-only ablation and methods that rely on scalar task scores.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 43 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

35 of 43 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  4. 2023Mastering Diverse Domains through World Modelscited by 1,253
  5. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  6. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 771
  7. 2017Deep Reinforcement Learning that Matterscited by 754
  8. 2023A Comprehensive Overview of Large Language Modelscited by 648
  9. 2019The StarCraft Multi-Agent Challengecited by 597
  10. 2024Large Language Models: A Surveycited by 525
  11. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
  12. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
23 more
  1. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  2. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 274
  3. 2018An Introduction to Deep Reinforcement Learningcited by 206
  4. 2020Prioritized Level Replaycited by 198
  5. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  6. 2022Evolving Curricula with Regret-Based Environment Designcited by 173
  7. 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 170
  8. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  9. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  10. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 137
  11. 2021Replay-Guided Adversarial Environment Designcited by 111
  12. 2020Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutionscited by 106
  13. 2024OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Codecited by 74
  14. 2023OMNI: Open-endedness via Models of human Notions of Interestingnesscited by 70
  15. 2019From Few to More: Large-scale Dynamic Multiagent Curriculum Learningcited by 36
  16. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 31
  17. 2024No Regrets: Investigating and Improving Regret Approximations for Curriculum Discoverycited by 31
  18. 2021DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learningcited by 26
  19. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  20. 2025Foundation Model Self-Play: Open-Ended Strategy Innovation via Foundation Modelscited by 6
  21. 2024Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Taskscited by 4
  22. 2023Learning Curricula in Open-Ended Worldscited by 2
  23. 2024Game On: Towards Language Models as RL Experimenterscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning Instruction-Following Policies through Open-Ended Instruction Relabeling with Large Language Models
  2. 2025VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learning
  3. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization
  4. 2026V-ICAL Bench: Evaluating Video In-Context Learning for Multimodal Agents in Interactive Environments
  5. 2026ViSkill: Reinforcing VLM Agents with Evolving Visual-Native Skills
  6. 2022Learning to Guide Multiple Heterogeneous Actors from a Single Human Demonstration via Automatic Curriculum Learning in StarCraft II
  7. 2025Refined Policy Distillation: From VLA Generalists to RL Experts
  8. 2024Syllabus: Portable Curricula for Reinforcement Learning Agents