Pragmatically Learning from Pedagogical Demonstrations in Multi-Goal Environments
On the map: Offline RL and imitation learning from data → Robot learning from demonstration
Abstract (arXiv)
Learning from demonstration methods usually leverage close to optimal demonstrations to accelerate training. By contrast, when demonstrating a task, human teachers deviate from optimal demonstrations and pedagogically modify their behavior by giving demonstrations that best disambiguate the goal they want to demonstrate. Analogously, human learners excel at pragmatically inferring the intent of the teacher, facilitating communication between the two agents. These mechanisms are critical in the few demonstrations regime, where inferring the goal is more difficult. In this paper, we implement pedagogy and pragmatism mechanisms by leveraging a Bayesian model of Goal Inference from demonstrations (BGI). We highlight the benefits of this model in multi-goal teacher-learner setups with two artificial agents that learn with goal-conditioned Reinforcement Learning. We show that combining BGI-agents (a pedagogical teacher and a pragmatic learner) results in faster learning and reduced goal ambiguity over standard learning from demonstrations, especially in the few demonstrations regime. We provide the code for our experiments (https://github.com/Caselles/NeurIPS22-demonstrations-pedagogy-pragmatism), as well as an illustrative video explaining our approach (https://youtu.be/V4n16IjkNyw).
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data. Builds on 15 of its 55 references that are in the map. First posted on arXiv 9 Jun 2022.
Builds on
15 of 55 references.
References · 15
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2017Neural Message Passing for Quantum Chemistrycited by 3,460
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2017Hindsight Experience Replaycited by 1,096
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
- 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
- 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
- 2020Online Bayesian Goal Inference for Boundedly-Rational Planning Agentscited by 67
- 2017Pragmatic-Pedagogic Value Alignmentcited by 36
- 2021Cognitive science as a source of forward and inverse models of human decisions for robotics and controlcited by 22
- 2019Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learningcited by 15
3 more
- 2019CLIC: Curriculum Learning and Imitation for object Control in non-rewarding environmentscited by 10
- 2018Active choice of teachers, learning strategies and goals for a socially guided intrinsic motivation learnercited by 5
- 2022Help Me Explore: Minimal Social Interventions for Graph-Based Autotelic Agentscited by 3
Built on by
Citing papers per year: 2023: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2022Pedagogical Demonstrations and Pragmatic Learning in Artificial Tutor-Learner Interactions
- 2026Learning from Demonstrations via Capability-Aware Goal Sampling
- 2021Supervised Bayesian Specification Inference from Demonstrations
- 2022Can In-context Learners Learn a Reasoning Concept from Demonstrations?
- 2024The intrinsic motivation of reinforcement and imitation learning for sequential tasks
- 2025When a Robot is More Capable than a Human: Learning from Constrained Demonstrators
- 2024Adaptive Teaching in Heterogeneous Agents: Balancing Surprise in Sparse Reward Scenarios
- 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regression