Towards Teachable Autotelic Agents
Not on the map. Why?
Abstract (arXiv)
Autonomous discovery and direct instruction are two distinct sources of learning in children but education sciences demonstrate that mixed approaches such as assisted discovery or guided play result in improved skill acquisition. In the field of Artificial Intelligence, these extremes respectively map to autonomous agents learning from their own signals and interactive learning agents fully taught by their teachers. In between should stand teachable autotelic agents (TAA): agents that learn from both internal and teaching signals to benefit from the higher efficiency of assisted discovery. Designing such agents will enable real-world non-expert users to orient the learning trajectories of agents towards their expectations. More fundamentally, this may also be a key step to build agents with human-level intelligence. This paper presents a roadmap towards the design of teachable autonomous agents. Building on developmental psychology and education sciences, we start by identifying key features enabling assisted discovery processes in child-tutor interactions. This leads to the production of a checklist of features that future TAA will need to demonstrate. The checklist allows us to precisely pinpoint the various limitations of current reinforcement learning agents and to identify the promising first steps towards TAA. It also shows the way forward by highlighting key research directions towards the design or autonomous agents that can be taught by ordinary people via natural pedagogy.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 139 references that are in the map. First posted on arXiv 25 May 2021.
Builds on
28 of 139 references.
References · 28
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2017Hindsight Experience Replaycited by 1,096
- 2018Behavioral Cloning from Observationcited by 482
- 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
- 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
- 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
- 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
- 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
- 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 141
- 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 119
- 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 79
- 2020Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Explorationcited by 68
16 more
- 2019Towards Practical Multi-Object Manipulation using Relational Reinforcement Learningcited by 54
- 2019Learning Human Objectives by Evaluating Hypothetical Behaviorcited by 41
- 2019Contextual Imagined Goals for Self-Supervised Robotic Learningcited by 40
- 2019Automated curricula through setter-solver interactionscited by 26
- 2021Interactive Learning from Activity Descriptioncited by 26
- 2020Feature Expansive Reward Learning: Rethinking Human Inputcited by 25
- 2021Symbolic Behaviour in Artificial Intelligencecited by 24
- 2020Inverse Reinforcement Learning with Natural Language Goalscited by 20
- 2020Hierarchically Organized Latent Modules for Exploratory Search in Morphogenetic Systemscited by 13
- 2019Curiosity-Driven Multi-Criteria Hindsight Experience Replaycited by 11
- 2019CLIC: Curriculum Learning and Imitation for object Control in non-rewarding environmentscited by 10
- 2017AI Challenges in Human-Robot Cognitive Teamingcited by 9
- 2019ZPD Teaching Strategies for Deep Reinforcement Learning from Demonstrationscited by 5
- 2020Theory of Mind with Guilt Aversion Facilitates Cooperative Reinforcement Learningcited by 5
- 2020Learning Compositional Neural Programs for Continuous Controlcited by 3
- 2020Explainable Agents Through Social Cues: A Reviewcited by 1
Built on by
Citing papers per year: 2023: 2.
2 papers.