mlsanityData to 7 Oct 2026

Regularity as Intrinsic Reward for Free Play

arXiv 2312.01473 ↗First posted 3 Dec 2023cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

We propose regularity as a novel reward signal for intrinsically-motivated reinforcement learning. Taking inspiration from child development, we postulate that striving for structure and order helps guide exploration towards a subspace of tasks that are not favored by naive uncertainty-based intrinsic rewards. Our generalized formulation of Regularity as Intrinsic Reward (RaIR) allows us to operationalize it within model-based reinforcement learning. In a synthetic environment, we showcase the plethora of structured patterns that can emerge from pursuing this regularity objective. We also demonstrate the strength of our method in a multi-object robotic manipulation environment. We incorporate RaIR into free play and use it to complement the model's epistemic uncertainty as an intrinsic reward. Doing so, we witness the autonomous construction of towers and other regular structures during free play, which leads to a substantial improvement in zero-shot downstream task performance on assembly tasks.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 3 Dec 2023.

Builds on

30 of 51 references.

References · 30

  1. 2018Relational inductive biases, deep learning, and graph networkscited by 1,341
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  3. 2018Exploration by Random Network Distillationcited by 966
  4. 2018DeepMind Control Suitecited by 935
  5. 2020Object-Centric Learning with Slot Attentioncited by 865
  6. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  7. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  8. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  9. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  10. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  11. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  12. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
18 more
  1. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 307
  2. 2019Self-Supervised Exploration via Disagreementcited by 263
  3. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  4. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  5. 2021Discovering and Achieving Goals via World Modelscited by 131
  6. 2020Sample-efficient Cross-Entropy Method for Real-time Planningcited by 90
  7. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 78
  8. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  9. 2019Towards Practical Multi-Object Manipulation using Relational Reinforcement Learningcited by 54
  10. 2021Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planningcited by 43
  11. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulationcited by 29
  12. 2023Planning Goals for Explorationcited by 28
  13. 2020Active World Model Learning with Progress Curiositycited by 27
  14. 2021Capturing the objects of vision with neural networkscited by 21
  15. 2021Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Explorationcited by 14
  16. 2021Sparse Reward Exploration via Novelty Search and Emitterscited by 13
  17. 2019Curiosity-Driven Multi-Criteria Hindsight Experience Replaycited by 11
  18. 2022Factored World Models for Zero-Shot Generalization in Robotic Manipulationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2026Teaching Models to Teach Themselves: Reasoning at the Edge of Learnabilitycited by 13
  2. 2025SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Modelscited by 9
  3. 2025ReaLJam: Real-Time Human-AI Music Jamming with Reinforcement Learning-Tuned Transformerscited by 7
  4. 2025Skill Learning via Policy Diversity Yields Identifiable Representations for Reinforcement Learningcited by 4
  5. 2025VendiRL: A Framework for Self-Supervised Reinforcement Learning of Diversely Diverse Skillscited by 2