mlsanityData to 7 Oct 2026

Improving Behavioural Cloning with Positive Unlabeled Learning

arXiv 2301.11734 ↗First posted 27 Jan 2023cs.LG cs.ROcited by 8

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Learning control policies offline from pre-recorded datasets is a promising avenue for solving challenging real-world problems. However, available datasets are typically of mixed quality, with a limited number of the trajectories that we would consider as positive examples; i.e., high-quality demonstrations. Therefore, we propose a novel iterative learning algorithm for identifying expert trajectories in unlabeled mixed-quality robotics datasets given a minimal set of positive examples, surpassing existing algorithms in terms of accuracy. We show that applying behavioral cloning to the resulting filtered dataset outperforms several competitive offline reinforcement learning and imitation learning baselines. We perform experiments on a range of simulated locomotion tasks and on two challenging manipulation tasks on a real robotic system; in these experiments, our method showcases state-of-the-art performance. Our website: urlhttps://sites.google.com/view/offline-policy-learning-pubc.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 37 of its 55 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

37 of 55 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations20 Jul
  2. 2021Implicit Behavioral Cloning1 Sep
  3. 2019BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learning27 Oct

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  7. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  8. 2016OpenAI Gymcited by 1,684
  9. 2016Generative Adversarial Imitation Learningcited by 1,576
  10. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  11. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  12. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
22 more
  1. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  2. 2018DeepMind Control Suitecited by 935
  3. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  4. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  5. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  6. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  7. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  8. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
  9. 2020Critic Regularized Regressioncited by 269
  10. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  11. 2018Learning from positive and unlabeled data: a surveycited by 218
  12. 2017Positive-Unlabeled Learning with Non-Negative Risk Estimatorcited by 217
  13. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
  14. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  15. 2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
  16. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 108
  17. 2018Neural probabilistic motor primitives for humanoid controlcited by 96
  18. 2022A Review of Off-Policy Evaluation in Reinforcement Learningcited by 92
  19. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  20. 2020TriFinger: An Open-Source Robot for Learning Dexteritycited by 31
  21. 2021Solving the Real Robot Challenge using Deep Reinforcement Learningcited by 5
  22. 2022Dexterous Robotic Manipulation using Deep Reinforcement Learning and Knowledge Transfer for Complex Sparse Reward-based Taskscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 1; 2026: 3.

8 papers.

  1. 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 20
  2. 2024Predicting Long-Term Human Behaviors in Discrete Representations via Physics-Guided Diffusioncited by 6
  3. 2023Identifying Expert Behavior in Offline Training Datasets Improves Behavioral Cloning of Robotic Manipulation Policiescited by 5
  4. 2024Offline Reinforcement Learning with Domain-Unlabeled Datacited by 3
  5. 2024Dataset Clustering for Improved Offline Policy Learningcited by 1
  6. 2025SketchPlan: Diffusion Based Drone Planning From Human Sketchescited by 1
  7. 2026How to Mitigate the Distribution Shift Problem in Robotics Control: A Robust and Adaptive Approach Based on Offline to Online Imitation Learningcited by 0
  8. 2026Learning from Mixed-Quality Deployment Experience for Robot Manipulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Offline Learning from Demonstrations and Unlabeled Experience
  2. 2024How to Leverage Diverse Demonstrations in Offline Imitation Learning
  3. 2022Model-based trajectory stitching for improved behavioural cloning and its applications
  4. 2025Robust Offline Imitation Learning Through State-level Trajectory Stitching
  5. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
  6. 2022Reliable Conditioning of Behavioral Cloning for Offline Reinforcement Learning
  7. 2022Implicit Offline Reinforcement Learning via Supervised Learning
  8. 2026When Life Gives You BC, Make Q-functions: Extracting Q-values from Behavior Cloning for On-Robot Reinforcement Learning