mlsanityData to 7 Oct 2026

Behavior Transformers: Cloning k modes with one stone

arXiv 2206.11251 ↗First posted 22 Jun 2022cs.LG cs.AI cs.CVcited by 299

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

While behavior learning has made impressive progress in recent times, it lags behind computer vision and natural language processing due to its inability to leverage large, human-generated datasets. Human behaviors have wide variance, multiple modes, and human demonstrations typically do not come with reward labels. These properties limit the applicability of current methods in Offline RL and Behavioral Cloning to learn from large, pre-collected datasets. In this work, we present Behavior Transformer (BeT), a new technique to model unlabeled demonstration data with multiple modes. BeT retrofits standard transformer architectures with action discretization coupled with a multi-task action correction inspired by offset prediction in object detection. This allows us to leverage the multi-modal modeling ability of modern transformers to predict multi-modal continuous actions. We experimentally evaluate BeT on a variety of robotic manipulation and self-driving behavior datasets. We show that BeT significantly improves over prior state-of-the-art work on solving demonstrated tasks while capturing the major modes present in the pre-collected datasets. Finally, through an extensive ablation study, we analyze the importance of every crucial component in BeT. Videos of behavior generated by BeT are available at https://notmahi.github.io/bet

Key facts. Cited by 299 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 60 of its 97 references that are in the map. First posted on arXiv 22 Jun 2022.

Builds on

60 of 97 references.

Closest ancestors · 8

Same direction, newest first

  1. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation24 Mar
  2. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation2 Dec
  3. 2021Towards More Generalizable One-shot Visual Imitation Learning26 Oct
  4. 2021Playful Interactions for Representation Learning19 Jul
  5. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning25 Oct
  6. 2019Self-Supervised Correspondence in Visuomotor Policy Learning16 Sep
  7. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation7 Nov
  8. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skills26 Feb

Other references · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2015Fast R-CNNcited by 2,930
  10. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  11. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  12. 2016Density estimation using Real NVPcited by 2,029
38 more
  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  8. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  10. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  11. 2016End to End Learning for Self-Driving Carscited by 1,000
  12. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  13. 2019Solving Rubik's Cube with a Robot Handcited by 699
  14. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  15. 2016Language Modeling with Gated Convolutional Networkscited by 688
  16. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  17. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  18. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  19. 2017Convolutional Sequence to Sequence Learningcited by 550
  20. 2018Behavioral Cloning from Observationcited by 482
  21. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  22. 2021Implicit Behavioral Cloningcited by 442
  23. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
  24. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  25. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  26. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  27. 2019Learning Latent Plans from Playcited by 310
  28. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  29. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  30. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  31. 2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 136
  32. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  33. 2020Transformers for One-Shot Visual Imitationcited by 77
  34. 2016Neural Machine Translation in Linear Timecited by 65
  35. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
  36. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  37. 2020Energy-Based Imitation Learningcited by 39
  38. 2021Provable Representation Learning for Imitation with Contrastive Fourier Featurescited by 34

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 53; 2024: 74; 2025: 95; 2026: 66.

299 papers.

Sort:

By area:

  1. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  2. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  3. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  4. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  5. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  6. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  7. 2024One Step Diffusion via Shortcut Modelscited by 373
  8. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  9. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  10. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  11. 2023Imitating Human Behaviour with Diffusion Modelscited by 260
  12. 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 234
38 more
  1. 2023Goal-Conditioned Imitation Learning using Score-based Diffusion Policiescited by 229
  2. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 214
  3. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  4. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  5. 2024Behavior Generation with Latent Actionscited by 159
  6. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 159
  7. 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 151
  8. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  9. 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 137
  10. 2023Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulationcited by 135
  11. 2023Large Language Models for Robotics: A Surveycited by 132
  12. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131
  13. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 129
  14. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
  15. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  16. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  17. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  18. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
  19. 2023On Bringing Robots Homecited by 101
  20. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  21. 2024Large Language Models and Games: A Survey and Roadmapcited by 91
  22. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 82
  23. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 80
  24. 2025Universal Actions for Enhanced Embodied Foundation Modelscited by 80
  25. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  26. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  27. 2023XSkill: Cross Embodiment Skill Discoverycited by 74
  28. 2024The Ingredients for Robotic Diffusion Transformerscited by 72
  29. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learningcited by 70
  30. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 63
  31. 2023Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Playcited by 60
  32. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 60
  33. 2023Multi-Stage Cable Routing through Hierarchical Imitation Learningcited by 59
  34. 2023Waypoint-Based Imitation Learning for Robotic Manipulationcited by 59
  35. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 56
  36. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  37. 2026MEM: Multi-Scale Embodied Memory for Vision Language Action Modelscited by 55
  38. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrationscited by 51

50 most cited of 299 All 299 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025CLASS: Contrastive Learning via Action Sequence Supervision for Robot Manipulation
  2. 2025From Action Labels to Sets: Rethinking Action Supervision for Imitation Learning from Corrective Feedback
  3. 2026A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulation
  4. 2026Scalable Behavior Cloning with Open Data, Training, and Evaluation
  5. 2023Asking for Help: Failure Prediction in Behavioral Cloning through Value Approximation
  6. 2025Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation
  7. 2026Set-Supervised Diffusion Policy: Learning Action-Chunking Diffusion through Corrections
  8. 2026Transformer Transformer: A Unified Model for Motion-Conditioned Robot Co-design