# The Pitfalls of Imitation Learning when Actions are Continuous arXiv: https://arxiv.org/abs/2503.09722 · HTML: https://mlsanity.com/paper/2503.09722/ ## Key facts - Cited by 26 papers in the map (data through 14 Oct 2026). - Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. - Builds on 15 of its 54 references that are in the map. - First posted on arXiv 12 Mar 2025. ## Placed in - Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/) ## Builds on (most cited first, 15 of 15) - [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016) - [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023) - [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024) - [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017) - [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023) - [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021) - [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022) - [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016) - [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018) - [IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies](https://mlsanity.com/paper/2304.10573/) (2023) - [Imitating Human Behaviour with Diffusion Models](https://mlsanity.com/paper/2301.10677/) (2023) - [MEGA-DAgger: Imitation Learning with Multiple Imperfect Experts](https://mlsanity.com/paper/2303.00638/) (2023) - [Diffusing States and Matching Scores: A New Framework for Imitation Learning](https://mlsanity.com/paper/2410.13855/) (2024) - [Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression](https://mlsanity.com/paper/2310.11428/) (2023) - [High-probability minimax lower bounds](https://mlsanity.com/paper/2406.13447/) (2024) ## Built on by (most cited first, 26 of 26) - [Much Ado About Noising: Dispelling the Myths of Generative Robotic Control](https://mlsanity.com/paper/2512.01809/) (2025) - [From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuning](https://mlsanity.com/paper/2603.10263/) (2026) - [Inverse Q-Learning Done Right: Offline Imitation Learning in Qπ-Realizable MDPs](https://mlsanity.com/paper/2505.19946/) (2025) - [Drift is a Sampling Error: SNR-Aware Power Distributions for Long-Horizon Robotic Planning](https://mlsanity.com/paper/2605.09537/) (2026) - [Ratatouille: Imitation Learning Ingredients for Real-world Social Robot Navigation](https://mlsanity.com/paper/2509.17204/) (2025) - [Unified Visuomotor Targets: Supervising VLAs Beyond Physical Actions](https://mlsanity.com/paper/2608.03563/) (2026) - [Latent Reasoning via Sentence Embedding Prediction](https://mlsanity.com/paper/2505.22202/) (2025) - [A Test-Function Approach to Incremental Stability](https://mlsanity.com/paper/2507.00695/) (2025) - [Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames](https://mlsanity.com/paper/2510.24194/) (2025) - [STRUCTUREDAGENT: Planning with AND/OR Trees for Long-Horizon Web Tasks](https://mlsanity.com/paper/2603.05294/) (2026) - [MoRI: Mixture of RL and IL Experts for Long-Horizon Manipulation Tasks](https://mlsanity.com/paper/2604.10165/) (2026) - [Fast Rates for Inverse Reinforcement Learning](https://mlsanity.com/paper/2605.14599/) (2026) - [Understanding Multimodal Failure in Action-Chunking Behavioral Cloning](https://mlsanity.com/paper/2605.22493/) (2026) - [Workspace Models: Lightweight Robotic Memory via Saliency-Driven Supervision](https://mlsanity.com/paper/2609.20820/) (2026) - [VRScout: Towards Real-Time, Autonomous Testing of Virtual Reality Games](https://mlsanity.com/paper/2511.00002/) (2025) - [Beyond State-Wise Mirror Descent: Offline Policy Optimization with Parametric Policies](https://mlsanity.com/paper/2602.23811/) (2026) - [Understanding Behavior Cloning with Action Quantization](https://mlsanity.com/paper/2603.20538/) (2026) - [Non-Adversarial Imitation Learning Provably Free of Compounding Errors: The Value Flow Mechanism](https://mlsanity.com/paper/2603.22713/) (2026) - [Behavior Cloning Under PD Control: A Finite-Horizon Theory of Gain-Dependent Error Amplification](https://mlsanity.com/paper/2604.14484/) (2026) - [Provable imitation learning for control of instability in partially-observed Vlasov--Poisson equations](https://mlsanity.com/paper/2605.05081/) (2026) - [Adversarial Dual On-Policy Distillation from Expressive Teacher](https://mlsanity.com/paper/2605.27095/) (2026) - [Robustness of Robotic Manipulation: Foundations and Frontiers](https://mlsanity.com/paper/2606.31494/) (2026) - [Beyond Patch Removal: Persistent Adversarial Effects in Vision-Language-Action Policies](https://mlsanity.com/paper/2609.19669/) (2026) - [Anatomy of a Closed-Loop Collapse: A Causal Case Study of a Compressed VLA Policy](https://mlsanity.com/paper/2609.23048/) (2026) - [Action Chunking Proximal Policy Optimization with Feedback Correction](https://mlsanity.com/paper/2609.36250/) (2026) - [Measuring the Stability Assumption Behind Action Chunking](https://mlsanity.com/paper/2610.01626/) (2026) ## Abstract (arXiv) We study the problem of imitating an expert demonstrator in a discrete-time, continuous state-and-action control system. We show that, even if the dynamics satisfy a control-theoretic property called exponential stability (i.e. the effects of perturbations decay exponentially quickly), and the expert is smooth and deterministic, any smooth, deterministic imitator policy necessarily suffers error on execution that is exponentially larger, as a function of problem horizon, than the error under the distribution of expert training data. Our negative result applies to any algorithm which learns solely from expert data, including both behavior cloning and offline-RL algorithms, unless the algorithm produces highly "improper" imitator policies--those which are non-smooth, non-Markovian, or which exhibit highly state-dependent stochasticity--or unless the expert trajectory distribution is sufficiently "spread." We provide experimental evidence of the benefits of these more complex policy parameterizations, explicating the benefits of today's popular policy parameterizations in robot learning (e.g. action-chunking and diffusion policies). We also establish a host of complementary negative and positive results for imitation in control systems.