mlsanityData to 14 Oct 2026

Disagreement-Regularized Imitation Learning for Image-Based Continuous Control with Gaussian and Beta Policies

arXiv 2609.38407 ↗First posted 29 Sep 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Purpose: Behavior cloning can accumulate errors when a learned controller visits states outside the demonstrated distribution. This study evaluates whether Disagreement-Regularized Imitation Learning (DRIL), which converts disagreement among cloned policies into a reinforcement-learning reward, improves image-based continuous control. Methods: A controlled CarRacing study combines Gaussian and Beta learner policies, demonstrations from either a clipped Gaussian expert or an intrinsically bounded Beta expert, one or 20 trajectories, deterministic and stochastic evaluation, and three retained stages: behavior cloning, the highest 10-episode training-score checkpoint, and the final DRIL checkpoint. The disagreement ensemble contains five Gaussian policies in every variant. Each retained policy is evaluated over 100 procedurally generated episodes. Results: Score-selected DRIL produced its largest gains in the few-demonstration setting, improving over the strongest behavior-cloning mean by 61% with clipped-action demonstrations and by 112% with bounded-action demonstrations. With 20 trajectories, the advantage of DRIL narrowed; in the bounded-action regime, Beta behavior cloning remained about 7% above the best DRIL checkpoint. The experiments also show that the informativeness of the disagreement reward changes with the learner representation and training stage. Conclusion: DRIL can substantially improve few-demonstration visual continuous control, while bounded Beta policies provide strong behavior-cloning performance when more demonstrations are available. The results highlight the joint importance of learner support,ensemble response, and checkpoint selection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 40 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

20 of 40 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning20 Jul
  2. 2021A Pragmatic Look at Deep Imitation Learning4 Aug
  3. 2019Causal Confusion in Imitation Learning28 May

Other references · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2016OpenAI Gymcited by 1,736
  7. 2016Generative Adversarial Imitation Learningcited by 1,597
  8. 2016End to End Learning for Self-Driving Carscited by 1,009
  9. 2017End-to-end Driving via Conditional Imitation Learningcited by 485
  10. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  11. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 194
  12. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 129
5 more
  1. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 69
  2. 2024Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learningcited by 35
  3. 2023Hierarchical Generative Adversarial Imitation Learning with Mid-level Input Generation for Autonomous Driving on Urban Environmentscited by 12
  4. 2021Proximal Policy Optimization with Continuous Bounded Action Space via the Beta Distributioncited by 10
  5. 2025Exploring multimodal implicit behavior learning for vehicle navigation in simulated citiescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Continuous Mean-Zero Disagreement-Regularized Imitation Learning (CMZ-DRIL)
  2. 2024Stable-BC: Controlling Covariate Shift with Stable Behavior Cloning
  3. 2023Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior
  4. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
  5. 2026Proximal Action Replacement for Behavior Cloning Actor-Critic in Offline Reinforcement Learning
  6. 2025Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations
  7. 2026Understanding Behavior Cloning with Action Quantization
  8. 2022Reliable Conditioning of Behavioral Cloning for Offline Reinforcement Learning