# How to Mitigate the Distribution Shift Problem in Robotics Control: A Robust and Adaptive Approach Based on Offline to Online Imitation Learning

arXiv: https://arxiv.org/abs/2605.25414 · HTML: https://mlsanity.com/paper/2605.25414/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 25 May 2026.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 23 of 23)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation](https://mlsanity.com/paper/2401.02117/) (2024)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1604.00923/) (2016)
- [Positive-Unlabeled Learning with Non-Negative Risk Estimator](https://mlsanity.com/paper/1703.00593/) (2017)
- [DART: Noise Injection for Robust Imitation Learning](https://mlsanity.com/paper/1703.09327/) (2017)
- [Safe Policy Improvement with Baseline Bootstrapping](https://mlsanity.com/paper/1712.06924/) (2017)
- [Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations](https://mlsanity.com/paper/2207.10050/) (2022)
- [Class-prior Estimation for Learning from Positive and Unlabeled Data](https://mlsanity.com/paper/1611.01586/) (2016)
- [Mahalanobis++: Improving OOD Detection via Feature Normalization](https://mlsanity.com/paper/2505.18032/) (2025)
- [CCIL: Continuity-based Data Augmentation for Corrective Imitation Learning](https://mlsanity.com/paper/2310.12972/) (2023)
- [Multi-Task Imitation Learning for Linear Dynamical Systems](https://mlsanity.com/paper/2212.00186/) (2022)
- [Stable-BC: Controlling Covariate Shift with Stable Behavior Cloning](https://mlsanity.com/paper/2408.06246/) (2024)
- [SAFE-GIL: SAFEty Guided Imitation Learning for Robotic Systems](https://mlsanity.com/paper/2404.05249/) (2024)
- [A Likelihood Ratio-Based Approach to Segmenting Unknown Objects](https://mlsanity.com/paper/2409.06424/) (2024)
- [Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Tasks](https://mlsanity.com/paper/2301.00051/) (2022)
- [Improving Behavioural Cloning with Positive Unlabeled Learning](https://mlsanity.com/paper/2301.11734/) (2023)
- [Beyond Imitation: A Life-long Policy Learning Framework for Path Tracking Control of Autonomous Driving](https://mlsanity.com/paper/2404.17198/) (2024)
- [Dynamic Contrastive Skill Learning with State-Transition Based Skill Clustering and Dynamic Length Adjustment](https://mlsanity.com/paper/2504.14805/) (2025)

## Abstract (arXiv)

Distribution shift in imitation learning refers to the problem that the agent cannot plan proper actions for a state that has not been visited during the training. This problem can be largely attributed to the inherently narrow state-action coverage provided by expert demonstrations over the full environment. In this paper, we propose a robust offline to adaptive online imitation learning framework that handles the distribution shift problem in a lifelong, multi-phase scheme. In the offline learning phase, we leverage supplementary demonstrations to broaden the state-action coverage of the policy by utilizing a discriminator to effectively train the policy with supplementary demonstrations, thereby enhancing the robustness of the policy to distribution shift. In the subsequent online inference phase, our framework detects the occurrence of distribution shift and conducts self-supervised imitation learning from online experiences to adapt the policy to the online environments. Through extensive evaluations in MuJoCo environments, we demonstrate that our method exhibits better robustness to distribution shift and better adaptation performance to online environments than the baseline algorithms, which indicates superior performance of our framework against the distribution shift.