mlsanityData to 7 Oct 2026

Imitator Learning: Achieve Out-of-the-Box Imitation Ability in Variable Environments

arXiv 2310.05712 ↗First posted 9 Oct 2023cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning (IL) enables agents to mimic expert behaviors. Most previous IL techniques focus on precisely imitating one policy through mass demonstrations. However, in many applications, what humans require is the ability to perform various tasks directly through a few demonstrations of corresponding tasks, where the agent would meet many unexpected changes when deployed. In this scenario, the agent is expected to not only imitate the demonstration but also adapt to unforeseen environmental changes. This motivates us to propose a new topic called imitator learning (ItorL), which aims to derive an imitator module that can on-the-fly reconstruct the imitation policies based on very limited expert demonstrations for different unseen tasks, without any extra adjustment. In this work, we focus on imitator learning based on only one expert demonstration. To solve ItorL, we propose Demo-Attention Actor-Critic (DAAC), which integrates IL into a reinforcement-learning paradigm that can regularize policies' behaviors in unexpected situations. Besides, for autonomous imitation policy building, we design a demonstration-based attention architecture for imitator policy that can effectively output imitated actions by adaptively tracing the suitable states in demonstrations. We develop a new navigation benchmark and a robot environment for topic~and show that DAAC~outperforms previous imitation methods textitwith large margins both on seen and unseen tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 32 of its 47 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

32 of 47 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Planning for Sample Efficient Imitation Learning18 Oct
  2. 2021Imitation Learning by Reinforcement Learning10 Aug
  3. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  4. 2020Provable Representation Learning for Imitation Learning via Bi-level Optimization24 Feb
  5. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec

Other references · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2022Emergent Abilities of Large Language Modelscited by 2,483
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  8. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  9. 2019Solving Rubik's Cube with a Robot Handcited by 699
  10. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  11. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  12. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
15 more
  1. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  2. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 299
  3. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  4. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  5. 2017One-Shot Imitation Learningcited by 232
  6. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  7. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  8. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  9. 2020Transformers for One-Shot Visual Imitationcited by 77
  10. 2020Deep Imitation Learning for Bimanual Robotic Manipulationcited by 49
  11. 2019Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safetycited by 48
  12. 2021Towards More Generalizable One-shot Visual Imitation Learningcited by 48
  13. 2020Goal-Aware Prediction: Learning to Model What Matterscited by 45
  14. 2022Vision-based Autonomous Driving for Unstructured Environments Using Imitation Learningcited by 3
  15. 2021Stage Conscious Attention Network (SCAN) : A Demonstration-Conditioned Policy for Few-Shot Imitationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Diverse Imitation Learning via Self-Organizing Generative Models
  2. 2018Zero-Shot Visual Imitation
  3. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges
  4. 2020Imitating Unknown Policies via Exploration
  5. 2022Imitating, Fast and Slow: Robust learning from demonstrations via decision-time planning
  6. 2019State Alignment-based Imitation Learning
  7. 2024Online Adaptation for Enhancing Imitation Learning Policies
  8. 2025Self-evolved Imitation Learning in Simulated World