mlsanityData to 7 Oct 2026

MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale

arXiv 2104.08212 ↗First posted 16 Apr 2021cs.RO cs.LGcited by 226

Not on the map. Why?

Cited by 51 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

General-purpose robotic systems must master a large repertoire of diverse skills to be useful in a range of daily tasks. While reinforcement learning provides a powerful framework for acquiring individual behaviors, the time needed to acquire each skill makes the prospect of a generalist robot trained with RL daunting. In this paper, we study how a large-scale collective robotic learning system can acquire a repertoire of behaviors simultaneously, sharing exploration, experience, and representations across tasks. In this framework new tasks can be continuously instantiated from previously learned tasks improving overall performance and capabilities of the system. To instantiate this system, we develop a scalable and intuitive framework for specifying new tasks through user-provided examples of desired outcomes, devise a multi-robot collective learning system for data collection that simultaneously collects experience for multiple tasks, and develop a scalable and generalizable multi-task deep reinforcement learning method, which we call MT-Opt. We demonstrate how MT-Opt can learn a wide range of skills, including semantic picking (i.e., picking an object from a particular category), placing into various fixtures (e.g., placing a food item onto a plate), covering, aligning, and rearranging. We train and evaluate our system on a set of 12 real-world tasks with data collected from 7 robots, and demonstrate the performance of our system both in terms of its ability to generalize to structurally similar new tasks, and acquire distinct new tasks more quickly by leveraging past experience. We recommend viewing the videos at https://karolhausman.github.io/mt-opt/

Key facts. Cited by 226 papers in the map (data through 7 Oct 2026). Builds on 62 of its 90 references that are in the map. First posted on arXiv 16 Apr 2021.

Builds on

62 of 90 references.

References · 62

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  3. 2016Progressive Neural Networkscited by 1,332
  4. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  6. 2017Hindsight Experience Replaycited by 1,096
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  8. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  9. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  10. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  12. 2019Solving Rubik's Cube with a Robot Handcited by 699
38 more
  1. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  2. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  3. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  4. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  5. 2015Policy Distillationcited by 414
  6. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  7. 2016Learning to reinforcement learncited by 369
  8. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
  9. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  10. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  11. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  12. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  13. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 235
  14. 2017One-Shot Imitation Learningcited by 232
  15. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  16. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  17. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 220
  18. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  19. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  20. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 176
  21. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  22. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  23. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 138
  24. 2020Multi-Task Reinforcement Learning with Soft Modularizationcited by 138
  25. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  26. 2018Rotate your Networks: Better Weight Consolidation and Less Catastrophic Forgettingcited by 127
  27. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 121
  28. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  29. 2018ProMP: Proximal Meta-Policy Searchcited by 91
  30. 2018Evolved Policy Gradientscited by 86
  31. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  32. 2019Meta-learning of Sequential Strategiescited by 68
  33. 2018Task-Embedded Control Networks for Few-Shot Imitation Learningcited by 65
  34. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
  35. 2017Deep Generative Dual Memory Network for Continual Learningcited by 59
  36. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 59
  37. 2018Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learningcited by 52
  38. 2017Divide-and-Conquer Reinforcement Learningcited by 46

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 52; 2023: 58; 2024: 51; 2025: 27; 2026: 18.

226 papers.

Sort:

By area:

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  4. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  5. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  6. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  7. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  8. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  9. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 391
  10. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  11. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  12. 2024Robotic Control via Embodied Chain-of-Thought Reasoningcited by 340
38 more
  1. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  2. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
  3. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  4. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  5. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  6. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  7. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  8. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 223
  9. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  10. 2022Multi-Game Decision Transformerscited by 212
  11. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 175
  12. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  13. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  14. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  15. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  16. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  17. 2022Human-to-Robot Imitation in the Wildcited by 157
  18. 2023Structured World Models from Human Videoscited by 144
  19. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
  20. 2023Large Language Models for Robotics: A Surveycited by 132
  21. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 131
  22. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 129
  23. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
  24. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 117
  25. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
  26. 2023Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularizationcited by 114
  27. 2023Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learningcited by 114
  28. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  29. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  30. 2023On Bringing Robots Homecited by 101
  31. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  32. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learningcited by 93
  33. 2022Instruction-driven history-aware policies for robotic manipulationscited by 93
  34. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 86
  35. 2023Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidancecited by 85
  36. 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agentscited by 83
  37. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  38. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 80

50 most cited of 226 All 226 (JSON)