mlsanityData to 7 Oct 2026

CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning

arXiv 1810.06284 ↗First posted 15 Oct 2018cs.AIcited by 78

Not on the map. Why?

Abstract (arXiv)

In open-ended environments, autonomous learning agents must set their own goals and build their own curriculum through an intrinsically motivated exploration. They may consider a large diversity of goals, aiming to discover what is controllable in their environments, and what is not. Because some goals might prove easy and some impossible, agents must actively select which goal to practice at any moment, to maximize their overall mastery on the set of learnable goals. This paper proposes CURIOUS, an algorithm that leverages 1) a modular Universal Value Function Approximator with hindsight learning to achieve a diversity of goals of different kinds within a unique policy and 2) an automated curriculum learning mechanism that biases the attention of the agent towards goals maximizing the absolute learning progress. Agents focus sequentially on goals of increasing complexity, and focus back on goals that are being forgotten. Experiments conducted in a new modular-goal robotic environment show the resulting developmental self-organization of a learning curriculum, and demonstrate properties of robustness to distracting goals, forgetting and changes in body properties.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 23; 2023: 8; 2024: 7; 2025: 12; 2026: 10.

78 papers.

Sort:

By area:

  1. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  2. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  3. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  4. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  5. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
  6. 2021Causal Influence Detection for Improving Efficiency in Reinforcement Learningcited by 70
  7. 2022Semantic Exploration from Language Abstractions and Pretrained Representationscited by 51
  8. 2021MADE: Exploration via Maximizing Deviation from Explored Regionscited by 40
  9. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulationcited by 29
  10. 2023Augmenting Autotelic Agents with Large Language Modelscited by 29
  11. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  12. 2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
38 more
  1. 2021MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learningcited by 24
  2. 2021Return-based Scaling: Yet Another Normalisation Trick for Deep RLcited by 22
  3. 2024Practice Makes Perfect: Planning to Learn Skill Parameter Policiescited by 22
  4. 2022Intelligent problem-solving as integrated hierarchical reinforcement learningcited by 21
  5. 2021TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RLcited by 18
  6. 2022Embedding Synthetic Off-Policy Experience for Autonomous Driving via Zero-Shot Curriculacited by 17
  7. 2021Self-supervised Reinforcement Learning with Independently Controllable Subgoalscited by 16
  8. 2021Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learningcited by 15
  9. 2022Language and Culture Internalisation for Human-Like Autotelic AIcited by 14
  10. 2026Teaching Models to Teach Themselves: Reasoning at the Edge of Learnabilitycited by 13
  11. 2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 12
  12. 2021DisCo RL: Distribution-Conditioned Reinforcement Learning for General-Purpose Policiescited by 9
  13. 2022Continual Auxiliary Task Learningcited by 9
  14. 2024Discovering Sensorimotor Agency in Cellular Automata using Diversity Searchcited by 9
  15. 2025SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Modelscited by 9
  16. 2023Embodied Active Learning of Relational State Abstractions for Bilevel Planningcited by 8
  17. 2025The impact of intrinsic rewards on exploration in Reinforcement Learningcited by 8
  18. 2021CLAIM: Curriculum Learning Policy for Influence Maximization in Unknown Social Networkscited by 7
  19. 2021Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learningcited by 7
  20. 2021Hindsight Goal Ranking on Replay Buffer for Sparse Reward Environmentcited by 7
  21. 2023Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learningcited by 7
  22. 2025MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spacescited by 7
  23. 2023Goal Space Abstraction in Hierarchical Reinforcement Learning via Set-Based Reachability Analysiscited by 6
  24. 2022Cell-Free Latent Go-Explorecited by 5
  25. 2023Regularity as Intrinsic Reward for Free Playcited by 5
  26. 2021Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer Learning to Discover Task Hierarchycited by 4
  27. 2021Reinforcement Learning with a Disentangled Universal Value Function for Item Recommendationcited by 4
  28. 2021Automatic Curricula via Expert Demonstrationscited by 4
  29. 2021Adaptive Multi-Goal Explorationcited by 4
  30. 2022Help Me Explore: Minimal Social Interventions for Graph-Based Autotelic Agentscited by 3
  31. 2022Robots Learn Increasingly Complex Tasks with Intrinsic Motivation and Automatic Curriculum Learningcited by 3
  32. 2022Learning Progress Driven Multi-Agent Curriculumcited by 3
  33. 2023Learning, Fast and Slow: A Goal-Directed Memory-Based Approach for Dynamic Environmentscited by 3
  34. 2023Modularity in Deep Learning: A Surveycited by 3
  35. 2024Diversity Progress for Goal Selection in Discriminability-Motivated RLcited by 2
  36. 2025Interleaved Multitask Learning with Energy Modulated Learning Progresscited by 2
  37. 2025VendiRL: A Framework for Self-Supervised Reinforcement Learning of Diversely Diverse Skillscited by 2
  38. 2022When to Go, and When to Explore: The Benefit of Post-Exploration in Intrinsic Motivationcited by 1

50 most cited of 78 All 78 (JSON)