mlsanityData to 7 Oct 2026

Back to the Manifold: Recovering from Out-of-Distribution States

arXiv 2207.08673 ↗First posted 18 Jul 2022cs.LG cs.ROcited by 19

Not on the map. Why?

Abstract (arXiv)

Learning from previously collected datasets of expert data offers the promise of acquiring robotic policies without unsafe and costly online explorations. However, a major challenge is a distributional shift between the states in the training dataset and the ones visited by the learned policy at the test time. While prior works mainly studied the distribution shift caused by the policy during the offline training, the problem of recovering from out-of-distribution states at the deployment time is not very well studied yet. We alleviate the distributional shift at the deployment time by introducing a recovery policy that brings the agent back to the training manifold whenever it steps out of the in-distribution states, e.g., due to an external perturbation. The recovery policy relies on an approximation of the training data density and a learned equivariant mapping that maps visual observations into a latent space in which translations correspond to the robot actions. We demonstrate the effectiveness of the proposed method through several manipulation experiments on a real robotic platform. Our results show that the recovery policy enables the agent to complete tasks while the behavioral cloning alone fails because of the distributional shift problem.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 18 Jul 2022.

Builds on

33 of 47 references.

References · 33

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2018World Modelscited by 1,444
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  6. 2016Group Equivariant Convolutional Networkscited by 1,154
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  9. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  10. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  11. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  12. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
21 more
  1. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  2. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  3. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  4. 2019Contrastive Learning of Structured World Modelscited by 206
  5. 2020Recovery RL: Safe Reinforcement Learning with Learned Recovery Zonescited by 156
  6. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 149
  7. 2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
  8. 2018State Representation Learning for Control: An Overviewcited by 125
  9. 2020Learning to be Safe: Deep RL with a Safety Criticcited by 109
  10. 2019Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Predictioncited by 96
  11. 2017Uncertainty-Aware Reinforcement Learning for Collision Avoidancecited by 86
  12. 2017Deep Predictive Policy Training using Reinforcement Learningcited by 30
  13. 2015Learning to Linearize Under Uncertaintycited by 25
  14. 2020Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learningcited by 22
  15. 2019Affordance Learning for End-to-End Visuomotor Robot Controlcited by 18
  16. 2022Equivariant Representation Learning via Class-Pose Decompositioncited by 17
  17. 2020NewtonianVAE: Proportional Control and Goal Identification from Pixels via Physical Latent Spacescited by 15
  18. 2021LS3: Latent Space Safe Sets for Long-Horizon Visuomotor Control of Sparse Reward Iterative Taskscited by 9
  19. 2019Adversarial Feature Training for Generalizable Robotic Visuomotor Controlcited by 7
  20. 2015Data-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical Modelscited by 6
  21. 2016A Sensorimotor Reinforcement Learning Framework for Physical Human-Robot Interactioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 5; 2025: 7; 2026: 3.

19 papers.

Sort:

By area:

  1. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 30
  2. 2023CCIL: Continuity-based Data Augmentation for Corrective Imitation Learningcited by 26
  3. 2023One ACT Play: Single Demonstration Behavior Cloning with Action Chunking Transformerscited by 17
  4. 2024SAFE-GIL: SAFEty Guided Imitation Learning for Robotic Systemscited by 13
  5. 2024RecoveryChaining: Learning Local Recovery Policies for Robust Manipulationcited by 8
  6. 2024Out-of-Distribution Recovery with Object-Centric Keypoint Inverse Policy for Visuomotor Imitation Learningcited by 6
  7. 2026Green-VLA: Staged Vision-Language-Action Model for Generalist Robotscited by 6
  8. 2024ARCADE: Scalable Demonstration Collection and Generation via Augmented Reality for Imitation Learningcited by 5
  9. 2025Real-Time Operator Takeover for Visuomotor Diffusion Policy Trainingcited by 5
  10. 2025SafeMimic: Towards Safe and Autonomous Human-to-Robot Imitation for Mobile Manipulationcited by 5
  11. 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learningcited by 5
  12. 2025Failure Forecasting Boosts Robustness of Sim2Real Rhythmic Insertion Policiescited by 2
7 more
  1. 2026Learning Robust Execution in Robotic Manipulation with Agentic Reinforcement Learningcited by 2
  2. 2022Bimanual rope manipulation skill synthesis through context dependent correction policy learning from human demonstrationcited by 1
  3. 2024Dense Dynamics-Aware Reward Synthesis: Integrating Prior Experience with Demonstrationscited by 1
  4. 2025Exploiting Policy Idling for Dexterous Manipulationcited by 1
  5. 2025Diffusing Trajectory Optimization Problems for Recovery During Multi-Finger Manipulationcited by 1
  6. 2023DEFENDER: DTW-Based Episode Filtering Using Demonstrations for Enhancing RL Safetycited by 0
  7. 2026PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data
  2. 2026Conservative Offline Robot Policy Learning via Posterior-Transition Reweighting
  3. 2026Kintsugi-VLA: Turning Failed Robot Rollouts into Recovery Data through Interventional Recoverability
  4. 2026How to Mitigate the Distribution Shift Problem in Robotics Control: A Robust and Adaptive Approach Based on Offline to Online Imitation Learning
  5. 2022Implicit Offline Reinforcement Learning via Supervised Learning
  6. 2021Efficient Robotic Manipulation Through Offline-to-Online Reinforcement Learning and Goal-Aware State Information
  7. 2024Contractive Dynamical Imitation Policies for Efficient Out-of-Sample Recovery
  8. 2024Problem Space Transformations for Out-of-Distribution Generalisation in Behavioural Cloning