mlsanityData to 14 Oct 2026

Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning

arXiv 2407.15007 ↗First posted 20 Jul 2024cs.LG cs.AI math.STcited by 74

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning (IL) aims to mimic the behavior of an expert in a sequential decision making task by learning from demonstrations, and has been widely applied to robotics, autonomous driving, and autoregressive text generation. The simplest approach to IL, behavior cloning (BC), is thought to incur sample complexity with unfavorable quadratic dependence on the problem horizon, motivating a variety of different online algorithms that attain improved linear horizon dependence under stronger assumptions on the data and the learner's access to the expert. We revisit the apparent gap between offline and online IL from a learning-theoretic perspective, with a focus on the realizable/well-specified setting with general policy classes up to and including deep neural networks. Through a new analysis of behavior cloning with the logarithmic loss, we show that it is possible to achieve horizon-independent sample complexity in offline IL whenever (i) the range of the cumulative payoffs is controlled, and (ii) an appropriate notion of supervised learning complexity for the policy class is controlled. Specializing our results to deterministic, stationary policies, we show that the gap between offline and online IL is smaller than previously thought: (i) it is possible to achieve linear dependence on horizon in offline IL under dense rewards (matching what was previously only known to be achievable in online IL); and (ii) without further assumptions on the policy class, online IL cannot improve over offline IL with the logarithmic loss, even in benign MDPs. We complement our theoretical results with experiments on standard RL tasks and autoregressive language generation to validate the practical relevance of our findings.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. First posted on arXiv 20 Jul 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 33; 2026: 33.

74 papers.

Sort:

By area:

  1. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 108
  2. 2025Scaling Test-Time Compute Without Verification or RL is Suboptimalcited by 84
  3. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 74
  4. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulatorscited by 67
  5. 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
  6. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 23
  7. 2025A Theory of Learning with Autoregressive Chain of Thoughtcited by 23
  8. 2024Diffusing States and Matching Scores: A New Framework for Imitation Learningcited by 16
  9. 2025Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulationcited by 13
  10. 2025Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learningcited by 12
  11. 2025SPINE: Token-Selective Test-Time Reinforcement Learning with Entropy-Band Regularizationcited by 12
  12. 2024Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Boundscited by 11
38 more
  1. 2026Reinforcement Learning from Rich Feedback with Distributional DAggercited by 10
  2. 2024Active Fine-Tuning of Multi-Task Policiescited by 9
  3. 2024The Central Role of the Loss Function in Reinforcement Learningcited by 8
  4. 2025Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levelscited by 8
  5. 2025Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Predictioncited by 8
  6. 2025Computational Algebra with Attention: Transformer Oracles for Border Basis Algorithmscited by 7
  7. 2025Learning to Answer from Correct Demonstrationscited by 7
  8. 2024Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximationcited by 6
  9. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  10. 2025Inverse Q-Learning Done Right: Offline Imitation Learning in Qπ-Realizable MDPscited by 5
  11. 2025Behavioral Exploration: Learning to Explore via In-Context Adaptationcited by 5
  12. 2025Inverse Reinforcement Learning with Just Classification and a Few Regressionscited by 5
  13. 2025Improved Bounds for Private and Robust Alignmentcited by 5
  14. 2025Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learningcited by 4
  15. 2026Meet Dynamic Individual Preferences: Resolving Conflicting Human Value with Paired Fine-Tuningcited by 4
  16. 2026Learning Visual Feature-Based World Models via Residual Latent Actioncited by 4
  17. 2025EMA Without the Lag: Bias-Corrected Iterate Averaging Schemescited by 3
  18. 2025To Distill or Decide? Understanding the Algorithmic Trade-off in Partially Observable Reinforcement Learningcited by 3
  19. 2026When Does Predictive Inverse Dynamics Outperform Behavior Cloning?cited by 3
  20. 2026Learning from the Best: Smoothness-Driven Metrics for Data Quality in Imitation Learningcited by 3
  21. 2026When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizoncited by 3
  22. 2024Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloningcited by 2
  23. 2025On the Identifiability of Latent Action Policiescited by 2
  24. 2025Rate optimal learning of equilibria from datacited by 2
  25. 2025Near-Optimal Second-Order Guarantees for Model-Based Adversarial Imitation Learningcited by 2
  26. 2025Learning When to Quit in Sales Conversationscited by 2
  27. 2025A Model-Based Approach to Imitation Learning through Multi-Step Predictionscited by 1
  28. 2025How Ensembles of Distilled Policies Improve Generalisation in Reinforcement Learningcited by 1
  29. 2025An Affective-Taxis Hypothesis for Alignment and Interpretabilitycited by 1
  30. 2025Learning to Evaluate Autonomous Behaviour in Human-Robot Interactioncited by 1
  31. 2025Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogamescited by 1
  32. 2026Your Self-Play Algorithm is Secretly an Adversarial Imitator: Understanding LLM Self-Play through the Lens of Imitation Learningcited by 1
  33. 2026Multi-agent imitation learning with function approximation: Linear Markov games and beyondcited by 1
  34. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithmscited by 1
  35. 2026Fast Rates for Inverse Reinforcement Learningcited by 1
  36. 2024Problem Space Transformations for Out-of-Distribution Generalisation in Behavioural Cloningcited by 0
  37. 2025Learning to Choose or Choosing to Learn: Best-of-N vs. Supervised Fine-Tuning for Bit String Generationcited by 0
  38. 2025Comparing Behavioural Cloning and Reinforcement Learning for Spacecraft Guidance and Control Networkscited by 0

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning When to Stop: Selective Imitation Learning Under Arbitrary Dynamics Shift
  2. 2022Model-based Offline Imitation Learning with Non-expert Data
  3. 2020Imitating Unknown Policies via Exploration
  4. 2023Coherent Soft Imitation Learning
  5. 2020Offline Learning from Demonstrations and Unlabeled Experience
  6. 2023Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior
  7. 2024Stable-BC: Controlling Covariate Shift with Stable Behavior Cloning
  8. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations