mlsanityData to 14 Oct 2026

The Pitfalls of Imitation Learning when Actions are Continuous

arXiv 2503.09722 ↗First posted 12 Mar 2025cs.LG cs.SY eess.SYcited by 26

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We study the problem of imitating an expert demonstrator in a discrete-time, continuous state-and-action control system. We show that, even if the dynamics satisfy a control-theoretic property called exponential stability (i.e. the effects of perturbations decay exponentially quickly), and the expert is smooth and deterministic, any smooth, deterministic imitator policy necessarily suffers error on execution that is exponentially larger, as a function of problem horizon, than the error under the distribution of expert training data. Our negative result applies to any algorithm which learns solely from expert data, including both behavior cloning and offline-RL algorithms, unless the algorithm produces highly "improper" imitator policies--those which are non-smooth, non-Markovian, or which exhibit highly state-dependent stochasticity--or unless the expert trajectory distribution is sufficiently "spread." We provide experimental evidence of the benefits of these more complex policy parameterizations, explicating the benefits of today's popular policy parameterizations in robot learning (e.g. action-chunking and diffusion policies). We also establish a host of complementary negative and positive results for imitation in control systems.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 15 of its 54 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

15 of 54 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learning17 Oct
  2. 2023MEGA-DAgger: Imitation Learning with Multiple Imperfect Experts1 Mar

Other references · 13

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  4. 2017Decoupled Weight Decay Regularizationcited by 1,831
  5. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  7. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  8. 2016End to End Learning for Self-Driving Carscited by 1,009
  9. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 397
  10. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 325
  11. 2023Imitating Human Behaviour with Diffusion Modelscited by 263
  12. 2023Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregressioncited by 14
1 more
  1. 2024High-probability minimax lower boundscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 19.

26 papers.

Sort:

By area:

  1. 2025Much Ado About Noising: Dispelling the Myths of Generative Robotic Controlcited by 36
  2. 2026From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuningcited by 10
  3. 2025Inverse Q-Learning Done Right: Offline Imitation Learning in Qπ-Realizable MDPscited by 5
  4. 2026Drift is a Sampling Error: SNR-Aware Power Distributions for Long-Horizon Robotic Planningcited by 3
  5. 2025Ratatouille: Imitation Learning Ingredients for Real-world Social Robot Navigationcited by 2
  6. 2026Unified Visuomotor Targets: Supervising VLAs Beyond Physical Actionscited by 2
  7. 2025Latent Reasoning via Sentence Embedding Predictioncited by 1
  8. 2025A Test-Function Approach to Incremental Stabilitycited by 1
  9. 2025Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogamescited by 1
  10. 2026STRUCTUREDAGENT: Planning with AND/OR Trees for Long-Horizon Web Taskscited by 1
  11. 2026MoRI: Mixture of RL and IL Experts for Long-Horizon Manipulation Taskscited by 1
  12. 2026Fast Rates for Inverse Reinforcement Learningcited by 1
14 more
  1. 2026Understanding Multimodal Failure in Action-Chunking Behavioral Cloningcited by 1
  2. 2026Workspace Models: Lightweight Robotic Memory via Saliency-Driven Supervisioncited by 1
  3. 2025VRScout: Towards Real-Time, Autonomous Testing of Virtual Reality Gamescited by 0
  4. 2026Beyond State-Wise Mirror Descent: Offline Policy Optimization with Parametric Policiescited by 0
  5. 2026Understanding Behavior Cloning with Action Quantizationcited by 0
  6. 2026Non-Adversarial Imitation Learning Provably Free of Compounding Errors: The Value Flow Mechanismcited by 0
  7. 2026Behavior Cloning Under PD Control: A Finite-Horizon Theory of Gain-Dependent Error Amplificationcited by 0
  8. 2026Provable imitation learning for control of instability in partially-observed Vlasov--Poisson equationscited by 0
  9. 2026Adversarial Dual On-Policy Distillation from Expressive Teachercited by 0
  10. 2026Robustness of Robotic Manipulation: Foundations and Frontierscited by 0
  11. 2026Beyond Patch Removal: Persistent Adversarial Effects in Vision-Language-Action Policiescited by 0
  12. 2026Anatomy of a Closed-Loop Collapse: A Causal Case Study of a Compressed VLA Policycited by 0
  13. 2026Action Chunking Proximal Policy Optimization with Feedback Correctioncited by 0
  14. 2026Measuring the Stability Assumption Behind Action Chunkingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021On the Sample Complexity of Stability Constrained Imitation Learning
  2. 2023Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior
  3. 2025Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous Control
  4. 2021Imitation Learning by Reinforcement Learning
  5. 2026Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback
  6. 2022Imitation Learning by Estimating Expertise of Demonstrators
  7. 2020Toward the Fundamental Limits of Imitation Learning
  8. 2023Agnostic Interactive Imitation Learning: New Theory and Practical Algorithms