mlsanityData to 7 Oct 2026

Implicit Behavioral Cloning

arXiv 2109.00137 ↗First posted 1 Sep 2021cs.RO cs.CV cs.LGcited by 442

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

We find that across a wide range of robot policy learning scenarios, treating supervised policy learning with an implicit model generally performs better, on average, than commonly used explicit models. We present extensive experiments on this finding, and we provide both intuitive insight and theoretical arguments distinguishing the properties of implicit models compared to their explicit counterparts, particularly with respect to approximating complex, potentially discontinuous and multi-valued (set-valued) functions. On robotic policy learning tasks we show that implicit behavioral cloning policies with energy-based models (EBM) often outperform common explicit (Mean Square Error, or Mixture Density) behavioral cloning policies, including on tasks with high-dimensional action spaces and visual image inputs. We find these policies provide competitive results or outperform state-of-the-art offline reinforcement learning methods on the challenging human-expert tasks from the D4RL benchmark suite, despite using no reward information. In the real world, robots with implicit policies can learn complex and remarkably subtle behaviors on contact-rich tasks from human demonstrations, including tasks with high combinatorial complexity and tasks requiring 1mm precision.

Key facts. Cited by 442 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 50 of its 76 references that are in the map. First posted on arXiv 1 Sep 2021.

Builds on

50 of 76 references.

Closest ancestors · 5

Same direction, newest first

  1. 2021Provable Representation Learning for Imitation with Contrastive Fourier Features26 May
  2. 2021On the Sample Complexity of Stability Constrained Imitation Learning18 Feb
  3. 2020Energy-Based Imitation Learning20 Apr
  4. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  5. 2018An Algorithmic Perspective on Imitation Learning16 Nov

Other references · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  6. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  7. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  8. 2016Generative Adversarial Imitation Learningcited by 1,576
  9. 2018World Modelscited by 1,444
  10. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  11. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  12. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
33 more
  1. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  2. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  3. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  4. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  5. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  6. 2019Deep Equilibrium Modelscited by 603
  7. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  8. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 402
  9. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  10. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
  11. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  12. 2021How to Train Your Energy-Based Modelscited by 271
  13. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  14. 2020How Neural Networks Extrapolate: From Feedforward to Graph Neural Networkscited by 218
  15. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  16. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  17. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 131
  18. 2020Improved Contrastive Divergence Training of Energy Based Modelscited by 127
  19. 2019Self-Supervised Correspondence in Visuomotor Policy Learningcited by 106
  20. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  21. 2020Residual Energy-Based Models for Text Generationcited by 95
  22. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85
  23. 2019Deep Declarative Networks: A New Hopecited by 77
  24. 2018SparseMAP: Differentiable Sparse Structured Inferencecited by 62
  25. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  26. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  27. 2020ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representationscited by 38
  28. 2019Model Based Planning with Energy Based Modelscited by 32
  29. 2020The Surprising Effectiveness of Linear Models for Visual Foresight in Object Pile Manipulationcited by 25
  30. 2019Autoregressive Energy Machinescited by 23
  31. 2020How to Train Your Energy-Based Model for Regressioncited by 23
  32. 2021AMP: Adversarial Motion Priors for Stylized Physics-Based Character Controlcited by 18
  33. 2018Concept Learning with Energy-Based Modelscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 48; 2023: 63; 2024: 98; 2025: 118; 2026: 111.

442 papers.

Sort:

By area:

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  2. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  3. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  4. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  5. 2024One Step Diffusion via Shortcut Modelscited by 373
  6. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  7. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  8. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  9. 2023Imitating Human Behaviour with Diffusion Modelscited by 260
  10. 2024Diffusion Policy Policy Optimizationcited by 255
  11. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 244
  12. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 234
38 more
  1. 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policiescited by 229
  2. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  3. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  4. 2025Unified Video Action Modelcited by 195
  5. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  6. 2024Behavior Generation with Latent Actionscited by 159
  7. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 159
  8. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 152
  9. 2021The Surprising Effectiveness of Representation Learning for Visual Imitationcited by 149
  10. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  11. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 135
  12. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  13. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  14. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  15. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
  16. 2022Instruction-driven history-aware policies for robotic manipulationscited by 93
  17. 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 93
  18. 2022Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapescited by 92
  19. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  20. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 86
  21. 2024One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillationcited by 83
  22. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  23. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  24. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  25. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 75
  26. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  27. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  28. 2024EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learningcited by 72
  29. 2024The Ingredients for Robotic Diffusion Transformerscited by 72
  30. 20243D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensingcited by 67
  31. 2024In-Context Imitation Learning via Next-Token Predictioncited by 65
  32. 2022Training and Inference on Any-Order Autoregressive Models the Right Waycited by 64
  33. 2022Learning Multi-Object Dynamics with Compositional Neural Radiance Fieldscited by 63
  34. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  35. 2024Learning Robotic Manipulation Policies from Point Clouds with Conditional Flow Matchingcited by 63
  36. 2024Equivariant Diffusion Policycited by 61
  37. 2023Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Playcited by 60
  38. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60

50 most cited of 442 All 442 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Stable-BC: Controlling Covariate Shift with Stable Behavior Cloning
  2. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policies
  3. 2026Beyond Imitation: Self-Improving Robot Policies via Off-Policy Q-Planning
  4. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
  5. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
  6. 2022Implicit Kinematic Policies: Unifying Joint and Cartesian Action Spaces in End-to-End Robot Learning
  7. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learning
  8. 2026Update-Free On-Policy Steering via Verifiers