# Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots

arXiv: https://arxiv.org/abs/1301.4862 · HTML: https://mlsanity.com/paper/1301.4862/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2013.

## Built on by (most cited first, 30 of 58)

- [Guiding Pretraining in Reinforcement Learning with Large Language Models](https://mlsanity.com/paper/2302.06692/) (2023)
- [Asymmetric self-play for automatic goal discovery in robotic manipulation](https://mlsanity.com/paper/2101.04882/) (2021)
- [OMNI: Open-endedness via Models of human Notions of Interestingness](https://mlsanity.com/paper/2306.01711/) (2023)
- [Self-Challenging Language Model Agents](https://mlsanity.com/paper/2506.01716/) (2025)
- [Temporal Abstraction in Reinforcement Learning with the Successor Representation](https://mlsanity.com/paper/2110.05740/) (2021)
- [How do language models learn facts? Dynamics, curricula and hallucinations](https://mlsanity.com/paper/2503.21676/) (2025)
- [Boosted Prompt Ensembles for Large Language Models](https://mlsanity.com/paper/2304.05970/) (2023)
- [Planning Goals for Exploration](https://mlsanity.com/paper/2303.13002/) (2023)
- [What Can I Do Here? Learning New Skills by Imagining Visual Affordances](https://mlsanity.com/paper/2106.00671/) (2021)
- [Practice Makes Perfect: Planning to Learn Skill Parameter Policies](https://mlsanity.com/paper/2402.15025/) (2024)
- [Proximal Curriculum for Reinforcement Learning Agents](https://mlsanity.com/paper/2304.12877/) (2023)
- [HyperNCA: Growing Developmental Networks with Neural Cellular Automata](https://mlsanity.com/paper/2204.11674/) (2022)
- [Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning](https://mlsanity.com/paper/2104.05043/) (2021)
- [Self-supervised Reinforcement Learning with Independently Controllable Subgoals](https://mlsanity.com/paper/2109.04150/) (2021)
- [Reward-Respecting Subtasks for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2202.03466/) (2022)
- [Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Vision](https://mlsanity.com/paper/2210.15206/) (2022)
- [WorldLLM: Improving LLMs' world modeling using curiosity-driven theory-making](https://mlsanity.com/paper/2506.06725/) (2025)
- [Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivation](https://mlsanity.com/paper/2102.11051/) (2021)
- [Discovering Sensorimotor Agency in Cellular Automata using Diversity Search](https://mlsanity.com/paper/2402.10236/) (2024)
- [Hierarchical Empowerment: Towards Tractable Empowerment-Based Skill Learning](https://mlsanity.com/paper/2307.02728/) (2023)
- [Proximal Curriculum with Task Correlations for Deep Reinforcement Learning](https://mlsanity.com/paper/2405.02481/) (2024)
- [MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces](https://mlsanity.com/paper/2502.07709/) (2025)
- [ACES: Generating Diverse Programming Puzzles with with Autotelic Generative Models](https://mlsanity.com/paper/2310.10692/) (2023)
- [World Models with Hints of Large Language Models for Goal Achieving](https://mlsanity.com/paper/2406.07381/) (2024)
- [Causally Aligned Curriculum Learning](https://mlsanity.com/paper/2503.16799/) (2025)
- [Discovering and Exploiting Sparse Rewards in a Learned Behavior Space](https://mlsanity.com/paper/2111.01919/) (2021)
- [Behavior Self-Organization Supports Task Inference for Continual Robot Learning](https://mlsanity.com/paper/2107.04533/) (2021)
- [Neural Inverse Kinematics](https://mlsanity.com/paper/2205.10837/) (2022)
- [Parametric-Task MAP-Elites](https://mlsanity.com/paper/2402.01275/) (2024)
- [Help Me Explore: Minimal Social Interventions for Graph-Based Autotelic Agents](https://mlsanity.com/paper/2202.05129/) (2022)

## Abstract (arXiv)

We introduce the Self-Adaptive Goal Generation - Robust Intelligent Adaptive Curiosity (SAGG-RIAC) architecture as an intrinsi- cally motivated goal exploration mechanism which allows active learning of inverse models in high-dimensional redundant robots. This allows a robot to efficiently and actively learn distributions of parameterized motor skills/policies that solve a corresponding distribution of parameterized tasks/goals. The architecture makes the robot sample actively novel parameterized tasks in the task space, based on a measure of competence progress, each of which triggers low-level goal-directed learning of the motor policy pa- rameters that allow to solve it. For both learning and generalization, the system leverages regression techniques which allow to infer the motor policy parameters corresponding to a given novel parameterized task, and based on the previously learnt correspondences between policy and task parameters. We present experiments with high-dimensional continuous sensorimotor spaces in three different robotic setups: 1) learning the inverse kinematics in a highly-redundant robotic arm, 2) learning omnidirectional locomotion with motor primitives in a quadruped robot, 3) an arm learning to control a fishing rod with a flexible wire. We show that 1) exploration in the task space can be a lot faster than exploration in the actuator space for learning inverse models in redundant robots; 2) selecting goals maximizing competence progress creates developmental trajectories driving the robot to progressively focus on tasks of increasing complexity and is statistically significantly more efficient than selecting tasks randomly, as well as more efficient than different standard active motor babbling methods; 3) this architecture allows the robot to actively discover which parts of its task space it can learn to reach and which part it cannot.