mlsanityData to 7 Oct 2026

Density-Ratio Weighted Behavioral Cloning: Learning Control Policies from Corrupted Datasets

arXiv 2510.01479 ↗First posted 1 Oct 2025cs.LG cs.SY eess.SYcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Offline reinforcement learning (RL) enables policy optimization from fixed datasets, making it suitable for safety-critical applications where online exploration is infeasible. However, these datasets are often contaminated by adversarial poisoning, system errors, or low-quality samples, leading to degraded policy performance in standard behavioral cloning (BC) and offline RL methods. This paper introduces Density-Ratio Weighted Behavioral Cloning (Weighted BC), a robust imitation learning approach that uses a small, verified clean reference set to estimate trajectory-level density ratios via a binary discriminator. These ratios are clipped and used as weights in the BC objective to prioritize clean expert behavior while down-weighting or discarding corrupted data, without requiring knowledge of the contamination mechanism. We establish theoretical guarantees showing convergence to the clean expert policy with finite-sample bounds that are independent of the contamination rate. A comprehensive evaluation framework is established, which incorporates various poisoning protocols (reward, state, transition, and action) on continuous control benchmarks. Experiments demonstrate that Weighted BC maintains near-optimal performance even at high contamination ratios outperforming baselines such as traditional BC, batch-constrained Q-learning (BCQ) and behavior regularized actor-critic (BRAC).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 27 of its 37 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

27 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations20 Jul
  2. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  3. 2018An Algorithmic Perspective on Imitation Learning16 Nov

Other references · 24

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2016Generative Adversarial Imitation Learningcited by 1,576
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  6. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  7. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  8. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  9. 2017Robust Adversarial Reinforcement Learningcited by 439
  10. 2017Adversarial Attacks on Neural Network Policiescited by 314
  11. 2017Certified Defenses for Data Poisoning Attackscited by 288
  12. 2019DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Correctionscited by 254
12 more
  1. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  2. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  3. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  4. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  5. 2019AlgaeDICE: Policy Gradient from Arbitrary Experiencecited by 187
  6. 2020GenDICE: Generalized Offline Estimation of Stationary Valuescited by 122
  7. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  8. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  9. 2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
  10. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
  11. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  12. 2025Implicit Constraint-Aware Off-Policy Correction for Offline Reinforcement Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Reliable Conditioning of Behavioral Cloning for Offline Reinforcement Learning
  2. 2022Improving TD3-BC: Relaxed Policy Constraint for Offline Learning and Stable Online Fine-Tuning
  3. 2025An Optimal Discriminator Weighted Imitation Perspective for Reinforcement Learning
  4. 2026Proximal Action Replacement for Behavior Cloning Actor-Critic in Offline Reinforcement Learning
  5. 2022Know Your Boundaries: The Necessity of Explicit Behavioral Cloning in Offline RL
  6. 2025Fine-tuning Behavioral Cloning Policies with Preference-Based Reinforcement Learning
  7. 2022Model-based Offline Imitation Learning with Non-expert Data
  8. 2021A Closer Look at Advantage-Filtered Behavioral Cloning in High-Noise Datasets