# Learning to Play with Intrinsically-Motivated Self-Aware Agents

arXiv: https://arxiv.org/abs/1802.07442 · HTML: https://mlsanity.com/paper/1802.07442/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Latent dynamics models and model-based RL, one of 11 directions in World models.
- Builds on 2 of its 46 references that are in the map.
- First posted on arXiv 21 Feb 2018.

## Placed in

- World models / [Latent dynamics models and model-based RL](https://mlsanity.com/wm/latent-dynamics-models-and-model-based-rl/)

## Builds on (most cited first, 2 of 2)

- [A Laplacian Framework for Option Discovery in Reinforcement Learning](https://mlsanity.com/paper/1703.00956/) (2017)
- [Data-efficient Deep Reinforcement Learning for Dexterous Manipulation](https://mlsanity.com/paper/1704.03073/) (2017)

## Built on by (most cited first, 30 of 35)

- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [Physion: Evaluating Physical Prediction from Vision in Humans and Machines](https://mlsanity.com/paper/2106.08261/) (2021)
- [Learning Formal Mathematics From Intrinsic Motivation](https://mlsanity.com/paper/2407.00695/) (2024)
- [Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning](https://mlsanity.com/paper/2107.12544/) (2021)
- [Neither hype nor gloom do DNNs justice](https://mlsanity.com/paper/2312.05355/) (2023)
- [TIDEE: Tidying Up Novel Rooms using Visuo-Semantic Commonsense Priors](https://mlsanity.com/paper/2207.10761/) (2022)
- [Pretraining in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2211.03959/) (2022)
- [World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry](https://mlsanity.com/paper/2604.01985/) (2026)
- [Shaping embodied agent behavior with activity-context priors from egocentric video](https://mlsanity.com/paper/2110.07692/) (2021)
- [Curious Replay for Model-based Adaptation](https://mlsanity.com/paper/2306.15934/) (2023)
- [Curiosity-Driven Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2501.11463/) (2025)
- [Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration](https://mlsanity.com/paper/2109.08603/) (2021)
- [Reward-Free Curricula for Training Robust World Models](https://mlsanity.com/paper/2306.09205/) (2023)
- [Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning](https://mlsanity.com/paper/2301.10886/) (2023)
- [Hierarchies of Planning and Reinforcement Learning for Robot Navigation](https://mlsanity.com/paper/2109.11178/) (2021)
- [MetaVIM: Meta Variationally Intrinsic Motivated Reinforcement Learning for Decentralized Traffic Signal Control](https://mlsanity.com/paper/2101.00746/) (2021)
- [Don't Do What Doesn't Matter: Intrinsic Motivation with Action Usefulness](https://mlsanity.com/paper/2105.09992/) (2021)
- [OPEn: An Open-ended Physics Environment for Learning Without a Task](https://mlsanity.com/paper/2110.06912/) (2021)
- [Reinforcement Learning with Exogenous States and Rewards](https://mlsanity.com/paper/2303.12957/) (2023)
- [Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning](https://mlsanity.com/paper/2309.04459/) (2023)
- [Intrinsic Goals for Autonomous Agents: Model-Based Exploration in Virtual Zebrafish Predicts Ethological Behavior and Whole-Brain Dynamics](https://mlsanity.com/paper/2506.00138/) (2025)
- [Continual Developmental Neurosimulation Using Embodied Computational Agents](https://mlsanity.com/paper/2103.05753/) (2021)
- [Development of collective behavior in newborn artificial agents](https://mlsanity.com/paper/2111.03796/) (2021)
- [CLR-GAM: Contrastive Point Cloud Learning with Guided Augmentation and Feature Mapping](https://mlsanity.com/paper/2302.14306/) (2023)
- [Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics](https://mlsanity.com/paper/2303.00077/) (2023)
- [Continual Learning of Visual Concepts for Robots through Limited Supervision](https://mlsanity.com/paper/2101.10509/) (2021)
- [Toward Human-Level Artificial Intelligence](https://mlsanity.com/paper/2108.03793/) (2021)
- [Parallel development of social preferences in fish and machines](https://mlsanity.com/paper/2305.11137/) (2023)
- [Developmental Curiosity and Social Interaction in Virtual Agents](https://mlsanity.com/paper/2305.13396/) (2023)
- [Measuring and Modeling Physical Intrinsic Motivation](https://mlsanity.com/paper/2305.13452/) (2023)

## Abstract (arXiv)

Infants are experts at playing, with an amazing ability to generate novel structured behaviors in unstructured environments that lack clear extrinsic reward signals. We seek to mathematically formalize these abilities using a neural network that implements curiosity-driven intrinsic motivation. Using a simple but ecologically naturalistic simulated environment in which an agent can move and interact with objects it sees, we propose a "world-model" network that learns to predict the dynamic consequences of the agent's actions. Simultaneously, we train a separate explicit "self-model" that allows the agent to track the error map of its own world-model, and then uses the self-model to adversarially challenge the developing world-model. We demonstrate that this policy causes the agent to explore novel and informative interactions with its environment, leading to the generation of a spectrum of complex behaviors, including ego-motion prediction, object attention, and object gathering. Moreover, the world-model that the agent learns supports improved performance on object dynamics prediction, detection, localization and recognition tasks. Taken together, our results are initial steps toward creating flexible autonomous agents that self-supervise in complex novel physical environments.