mlsanityData to 7 Oct 2026

Get Back Here: Robust Imitation by Return-to-Distribution Planning

arXiv 2305.01400 ↗First posted 2 May 2023cs.RO cs.AI cs.LGcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We consider the Imitation Learning (IL) setup where expert data are not collected on the actual deployment environment but on a different version. To address the resulting distribution shift, we combine behavior cloning (BC) with a planner that is tasked to bring the agent back to states visited by the expert whenever the agent deviates from the demonstration distribution. The resulting algorithm, POIR, can be trained offline, and leverages online interactions to efficiently fine-tune its planner to improve performance over time. We test POIR on a variety of human-generated manipulation demonstrations in a realistic robotic manipulation simulator and show robustness of the learned policy to different initial state distributions and noisy dynamics.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 53 of its 74 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

53 of 74 references.

Closest ancestors · 15

Same direction, newest first

  1. 2022Imitating, Fast and Slow: Robust learning from demonstrations via decision-time planning7 Apr
  2. 2021Implicit Behavioral Cloning1 Sep
  3. 2021What Matters for Adversarial Imitation Learning?1 Jun
  4. 2021No Need for Interactions: Robust Model-Based Imitation Learning using Neural ODE3 Apr
  5. 2020SAFARI: Safe and Active Robot Imitation Learning with Imagination18 Nov
  6. 2020Primal Wasserstein Imitation Learning8 Jun
  7. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  8. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  9. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation16 May
  10. 2019Uncertainty-Aware Data Aggregation for Deep Imitation Learning7 May
  11. 2018An Algorithmic Perspective on Imitation Learning16 Nov
  12. 2018Deep Imitative Models for Flexible Inference, Planning, and Control15 Oct
3 more
  1. 2018HG-DAgger: Interactive Imitation Learning with Human Experts5 Oct
  2. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep
  3. 2018Safe end-to-end imitation learning for model predictive control27 Mar

Other references · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  8. 2016End to End Learning for Self-Driving Carscited by 1,000
  9. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  10. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  11. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  12. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
26 more
  1. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  2. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  3. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  4. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  5. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  6. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  7. 2019Learning Latent Plans from Playcited by 310
  8. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  9. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  10. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  11. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  12. 2019Active Domain Randomizationcited by 161
  13. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  14. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  15. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  16. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 138
  17. 2020Model-Based Offline Planningcited by 126
  18. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  19. 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
  20. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  21. 2018Task-Embedded Control Networks for Few-Shot Imitation Learningcited by 65
  22. 2018Evaluating Reinforcement Learning Algorithms in Observational Health Settingscited by 60
  23. 2021Model-Based Offline Planning with Trajectory Pruningcited by 30
  24. 2021Learning Dynamics Models for Model Predictive Agentscited by 21
  25. 2019Uncertainty-Aware Imitation Learning using Kernelized Movement Primitivescited by 11
  26. 2019Model-based Behavioral Cloning with Future Image Similarity Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024IntervenGen: Interventional Data Generation for Robust and Data-Efficient Robot Imitation Learningcited by 24
  2. 2024Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulationcited by 13
  3. 2024OLLIE: Imitation Learning from Offline Pretraining to Online Finetuningcited by 6
  4. 2025EBT-Policy: Energy Unlocks Emergent Physical Reasoning Capabilitiescited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2022Robust Imitation of a Few Demonstrations with a Backwards Model
  2. 2022Robust Learning from Observation with Model Misspecification
  3. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation
  4. 2025A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search
  5. 2025Balance Equation-based Distributionally Robust Offline Imitation Learning
  6. 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning
  7. 2026Difference-Aware Retrieval Policies for Imitation Learning
  8. 2023Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior